00010000:00010000:1.0:1787826803.063919:0:77480:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e003b lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0840 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.063929:0:77480:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e003b lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0840 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.063934:0:77480:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e003b lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0840 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.063939:0:77480:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e003b lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0840 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826803.063945:0:77480:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff985610df8000 x1874669225525504/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826819 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826803.063953:0:77480:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e003b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0840 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.063959:0:77480:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:1.0:1787826803.063964:0:77480:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:1.0:1787826803.063980:0:77480:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:1.0:1787826803.063985:0:77480:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e003b 00000080:00010000:1.0:1787826803.063990:0:77480:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826803.063993:0:77480:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e003b from it ffffab3ac1dbfbf8 00010000:00010000:1.0:1787826803.063997:0:77480:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e003b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0840 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.064006:0:77480:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e003b lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0840 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826803.064025:0:77480:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787826803.064035:0:77480:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826803.064043:0:77480:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826803.064049:0:77480:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e003b lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0840 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826803.064055:0:77480:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e003b lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0840 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826803.064059:0:77480:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826803.064062:0:77480:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826803.064064:0:77480:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e003b 00000080:00010000:1.0:1787826803.064067:0:77480:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e003b from it ffffab3ac1dbfb38 00010000:00010000:1.0:1787826803.064070:0:77480:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e003b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0840 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826803.064074:0:77480:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e003b lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0840 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826803.064079:0:77480:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787826803.064082:0:77480:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f33a.replay-single-3, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), file ffff9856206d7080, open_flags 8042, mode 81a4 opened 0 00000080:00200000:1.0:1787826803.064088:0:77480:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f33a.replay-single-3, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=open|creat 00000002:00010000:1.0:1787826803.064101:0:77480:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f33a.replay-single-3,[0x2000153f1:0x4:0x0]) in obj [0x200000007:0x1:0x0], intent: open|creat flags 0100103 00010000:00010000:1.0:1787826803.064108:0:77480:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e003b lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x5aa6d504d91a0840 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826803.064148:0:77480:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e003b lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a0840 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826803.064156:0:77480:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826803.064159:0:77480:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e003b lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a0840 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826803.064185:0:77480:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a600/0xb3f83dad1b7e0042 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 77480 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.064191:0:77480:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a600/0xb3f83dad1b7e0042 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 77480 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:1.0:1787826803.064198:0:77480:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:1.0:1787826803.064203:0:77480:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610df8000 x1874669225525760/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:1.0:1787826803.064209:0:77480:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a600/0xb3f83dad1b7e0042 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 77480 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826803.064220:0:77480:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610df8000 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:77480:1874669225525760:192.168.203.114@tcp:101:createmany.0 00000100:00100000:1.0:1787826803.064303:0:77480:0:(client.c:2715:ptlrpc_set_wait()) set ffff985614d8b000 going to sleep for 11 seconds 00000100:00100000:1.0:1787826803.065652:0:77480:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610df8000 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:77480:1874669225525760:192.168.203.114@tcp:101:createmany.0 00010000:00010000:1.0:1787826803.065664:0:77480:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a600/0xb3f83dad1b7e0042 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a085c expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.065679:0:77480:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a600/0xb3f83dad1b7e0042 lrc: 4/0,1 mode: --/PR res: [0x2000153f1:0x4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a085c expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.065688:0:77480:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a600/0xb3f83dad1b7e0042 lrc: 4/0,1 mode: --/PR res: [0x2000153f1:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a085c expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.065711:0:77480:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a600/0xb3f83dad1b7e0042 lrc: 4/0,1 mode: PR/PR res: [0x2000153f1:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a085c expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.065718:0:77480:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a600/0xb3f83dad1b7e0042 lrc: 4/0,1 mode: PR/PR res: [0x2000153f1:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a085c expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.065725:0:77480:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a600/0xb3f83dad1b7e0042 lrc: 5/1,1 mode: PR/PR res: [0x2000153f1:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a085c expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.065731:0:77480:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a600/0xb3f83dad1b7e0042 lrc: 5/1,1 mode: PR/PR res: [0x2000153f1:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a085c expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.065735:0:77480:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a600/0xb3f83dad1b7e0042 lrc: 4/1,0 mode: PR/PR res: [0x2000153f1:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a085c expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826803.065742:0:77480:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff985610df8000 x1874669225525760/t188978561036(188978561036) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/664 e 0 to 0 dl 1787826819 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:1.0:1787826803.065751:0:77480:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff985610df8000 x1874669225525760/t188978561036(188978561036) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/664 e 0 to 0 dl 1787826819 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826803.065758:0:77480:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a600/0xb3f83dad1b7e0042 lrc: 3/1,0 mode: PR/PR res: [0x2000153f1:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a085c expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:1.0:1787826803.065773:0:77480:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a600/0xb3f83dad1b7e0042 lrc: 3/1,0 mode: PR/PR res: [0x2000153f1:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a085c expref: -99 pid: 77480 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.065786:0:77480:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8590021617/4 (0 0) 00000080:00200000:1.0:1787826803.065792:0:77480:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:1.0:1787826803.065801:0:77480:0:(namei.c:1628:ll_create_it()) VFS Op:name=f33a.replay-single-3, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=open|creat 00000080:00200000:1.0:1787826803.065824:0:77480:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x2000153f1:0x4:0x0](ffff985611670908) 00000080:00200000:1.0:1787826803.065884:0:77480:0:(file.c:6854:ll_layout_conf()) [0x2000153f1:0x4:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:1.0:1787826803.065889:0:77480:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x2000153f1:0x4:0x0](ffff985611670908) 00000080:00010000:1.0:1787826803.065892:0:77480:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x2000153f1:0x4:0x0](ffff985611670908) for lock 0xb3f83dad1b7e0042 00000080:00010000:1.0:1787826803.065900:0:77480:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x2000153f1:0x4:0x0](ffff985611670908) 00000080:00200008:1.0:1787826803.065909:0:77480:0:(file.c:1090:ll_file_open()) START file f33a.replay-single-3:[0x2000153f1:0x4:0x0](ffff985611670908), flags 100102 00000002:00100000:1.0:1787826803.065927:0:77480:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff985610df8000 x1874669225525760/t188978561036(188978561036) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826819 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:1.0:1787826803.065954:0:77480:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0042 from it ffff9856054b0660 00010000:00010000:1.0:1787826803.065959:0:77480:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a600/0xb3f83dad1b7e0042 lrc: 3/1,0 mode: PR/PR res: [0x2000153f1:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a085c expref: -99 pid: 77480 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.065968:0:77480:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a600/0xb3f83dad1b7e0042 lrc: 2/0,0 mode: PR/PR res: [0x2000153f1:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a085c expref: -99 pid: 77480 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826803.065984:0:77480:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x2000153f1:0x4:0x0](ffff985611670908), inode mode 81a4 mask 40 00000080:00200008:1.0:1787826803.066002:0:77480:0:(file.c:479:ll_file_release()) START file f33a.replay-single-3:[0x2000153f1:0x4:0x0](ffff985611670908), flags 100002 00000002:00100000:1.0:1787826803.066034:0:77480:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff985610df8000 x1874669225525760/t188978561036(188978561036) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826819 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:1.0:1787826803.066043:0:77480:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:1.0:1787826803.066049:0:77480:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610df9a40 x1874669225526016/t0(0) o35->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:1.0:1787826803.066070:0:77480:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610df9a40 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:77480:1874669225526016:192.168.203.114@tcp:35:createmany.0 00000100:00100000:1.0:1787826803.066111:0:77480:0:(client.c:2715:ptlrpc_set_wait()) set ffff985614d8b540 going to sleep for 11 seconds 00000100:00100000:1.0:1787826803.067032:0:77480:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610df9a40 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:77480:1874669225526016:192.168.203.114@tcp:35:createmany.0 00000002:00100000:1.0:1787826803.067057:0:77480:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff985610df8000 x1874669225525760/t188978561036(188978561036) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826819 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:1.0:1787826803.067100:0:77480:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826803.067112:0:77480:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826803.067153:0:77480:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b800/0xb3f83dad1b7e0049 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 77480 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.067162:0:77480:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b800/0xb3f83dad1b7e0049 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 77480 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.067171:0:77480:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b800/0xb3f83dad1b7e0049 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 77480 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826803.067185:0:77480:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610dfad00 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:77480:1874669225526144:192.168.203.114@tcp:101:createmany.0 00000100:00100000:1.0:1787826803.067227:0:77480:0:(client.c:2715:ptlrpc_set_wait()) set ffff985614d8b540 going to sleep for 11 seconds 00000100:00100000:1.0:1787826803.068140:0:77480:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610dfad00 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:77480:1874669225526144:192.168.203.114@tcp:101:createmany.0 00010000:00010000:1.0:1787826803.068162:0:77480:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b800/0xb3f83dad1b7e0049 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0871 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.068172:0:77480:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b800/0xb3f83dad1b7e0049 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0871 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.068186:0:77480:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b800/0xb3f83dad1b7e0049 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0871 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.068196:0:77480:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b800/0xb3f83dad1b7e0049 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0871 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.068206:0:77480:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b800/0xb3f83dad1b7e0049 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0871 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.068213:0:77480:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b800/0xb3f83dad1b7e0049 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0871 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.068220:0:77480:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b800/0xb3f83dad1b7e0049 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0871 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826803.068230:0:77480:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff985610dfad00 x1874669225526144/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826819 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826803.068242:0:77480:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b800/0xb3f83dad1b7e0049 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0871 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.068251:0:77480:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:1.0:1787826803.068255:0:77480:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:1.0:1787826803.068271:0:77480:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:1.0:1787826803.068276:0:77480:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e0049 00000080:00010000:1.0:1787826803.068280:0:77480:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826803.068283:0:77480:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0049 from it ffffab3ac1dbfbf8 00010000:00010000:1.0:1787826803.068287:0:77480:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b800/0xb3f83dad1b7e0049 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0871 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.068296:0:77480:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b800/0xb3f83dad1b7e0049 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0871 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826803.068313:0:77480:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787826803.068326:0:77480:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826803.068336:0:77480:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826803.068343:0:77480:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b800/0xb3f83dad1b7e0049 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0871 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826803.068352:0:77480:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b800/0xb3f83dad1b7e0049 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0871 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826803.068373:0:77480:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826803.068378:0:77480:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826803.068382:0:77480:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e0049 00000080:00010000:1.0:1787826803.068386:0:77480:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0049 from it ffffab3ac1dbfb38 00010000:00010000:1.0:1787826803.068389:0:77480:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b800/0xb3f83dad1b7e0049 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0871 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826803.068396:0:77480:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b800/0xb3f83dad1b7e0049 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0871 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826803.068404:0:77480:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787826803.068409:0:77480:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f33a.replay-single-4, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), file ffff9856206d7800, open_flags 8042, mode 81a4 opened 0 00000080:00200000:1.0:1787826803.068417:0:77480:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f33a.replay-single-4, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=open|creat 00000002:00010000:1.0:1787826803.068434:0:77480:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f33a.replay-single-4,[0x2000153f1:0x5:0x0]) in obj [0x200000007:0x1:0x0], intent: open|creat flags 0100103 00010000:00010000:1.0:1787826803.068443:0:77480:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b800/0xb3f83dad1b7e0049 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x5aa6d504d91a0871 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826803.068473:0:77480:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b800/0xb3f83dad1b7e0049 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a0871 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826803.068480:0:77480:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826803.068483:0:77480:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b800/0xb3f83dad1b7e0049 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a0871 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826803.068505:0:77480:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a000/0xb3f83dad1b7e0050 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 77480 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.068518:0:77480:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a000/0xb3f83dad1b7e0050 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 77480 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:1.0:1787826803.068527:0:77480:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:1.0:1787826803.068533:0:77480:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610dfad00 x1874669225526400/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:1.0:1787826803.068542:0:77480:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a000/0xb3f83dad1b7e0050 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 77480 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826803.068555:0:77480:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610dfad00 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:77480:1874669225526400:192.168.203.114@tcp:101:createmany.0 00000100:00100000:1.0:1787826803.068634:0:77480:0:(client.c:2715:ptlrpc_set_wait()) set ffff985614d8b540 going to sleep for 11 seconds 00000100:00100000:1.0:1787826803.070135:0:77480:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610dfad00 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:77480:1874669225526400:192.168.203.114@tcp:101:createmany.0 00010000:00010000:1.0:1787826803.070149:0:77480:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a000/0xb3f83dad1b7e0050 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a088d expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.070176:0:77480:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a000/0xb3f83dad1b7e0050 lrc: 4/0,1 mode: --/PR res: [0x2000153f1:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a088d expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.070185:0:77480:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a000/0xb3f83dad1b7e0050 lrc: 4/0,1 mode: --/PR res: [0x2000153f1:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a088d expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.070195:0:77480:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a000/0xb3f83dad1b7e0050 lrc: 4/0,1 mode: PR/PR res: [0x2000153f1:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a088d expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.070204:0:77480:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a000/0xb3f83dad1b7e0050 lrc: 4/0,1 mode: PR/PR res: [0x2000153f1:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a088d expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.070217:0:77480:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a000/0xb3f83dad1b7e0050 lrc: 5/1,1 mode: PR/PR res: [0x2000153f1:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a088d expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.070225:0:77480:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a000/0xb3f83dad1b7e0050 lrc: 5/1,1 mode: PR/PR res: [0x2000153f1:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a088d expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.070232:0:77480:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a000/0xb3f83dad1b7e0050 lrc: 4/1,0 mode: PR/PR res: [0x2000153f1:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a088d expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826803.070243:0:77480:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff985610dfad00 x1874669225526400/t188978561038(188978561038) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/664 e 0 to 0 dl 1787826819 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:1.0:1787826803.070253:0:77480:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff985610dfad00 x1874669225526400/t188978561038(188978561038) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/664 e 0 to 0 dl 1787826819 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826803.070262:0:77480:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a000/0xb3f83dad1b7e0050 lrc: 3/1,0 mode: PR/PR res: [0x2000153f1:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a088d expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:1.0:1787826803.070270:0:77480:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a000/0xb3f83dad1b7e0050 lrc: 3/1,0 mode: PR/PR res: [0x2000153f1:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a088d expref: -99 pid: 77480 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.070277:0:77480:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8590021617/5 (0 0) 00000080:00200000:1.0:1787826803.070281:0:77480:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:1.0:1787826803.070289:0:77480:0:(namei.c:1628:ll_create_it()) VFS Op:name=f33a.replay-single-4, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=open|creat 00000080:00200000:1.0:1787826803.070304:0:77480:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x2000153f1:0x5:0x0](ffff985611671988) 00000080:00200000:1.0:1787826803.070390:0:77480:0:(file.c:6854:ll_layout_conf()) [0x2000153f1:0x5:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:1.0:1787826803.070396:0:77480:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x2000153f1:0x5:0x0](ffff985611671988) 00000080:00010000:1.0:1787826803.070399:0:77480:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x2000153f1:0x5:0x0](ffff985611671988) for lock 0xb3f83dad1b7e0050 00000080:00010000:1.0:1787826803.070406:0:77480:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x2000153f1:0x5:0x0](ffff985611671988) 00000080:00200008:1.0:1787826803.070415:0:77480:0:(file.c:1090:ll_file_open()) START file f33a.replay-single-4:[0x2000153f1:0x5:0x0](ffff985611671988), flags 100102 00000002:00100000:1.0:1787826803.070435:0:77480:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff985610dfad00 x1874669225526400/t188978561038(188978561038) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826819 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:1.0:1787826803.070462:0:77480:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0050 from it ffff9856054b0660 00010000:00010000:1.0:1787826803.070467:0:77480:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a000/0xb3f83dad1b7e0050 lrc: 3/1,0 mode: PR/PR res: [0x2000153f1:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a088d expref: -99 pid: 77480 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.070476:0:77480:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a000/0xb3f83dad1b7e0050 lrc: 2/0,0 mode: PR/PR res: [0x2000153f1:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a088d expref: -99 pid: 77480 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826803.070491:0:77480:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x2000153f1:0x5:0x0](ffff985611671988), inode mode 81a4 mask 40 00000080:00200008:1.0:1787826803.070509:0:77480:0:(file.c:479:ll_file_release()) START file f33a.replay-single-4:[0x2000153f1:0x5:0x0](ffff985611671988), flags 100002 00000002:00100000:1.0:1787826803.070543:0:77480:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff985610dfad00 x1874669225526400/t188978561038(188978561038) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826819 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:1.0:1787826803.070558:0:77480:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:1.0:1787826803.070562:0:77480:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610df83c0 x1874669225526656/t0(0) o35->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:1.0:1787826803.070579:0:77480:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610df83c0 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:77480:1874669225526656:192.168.203.114@tcp:35:createmany.0 00000100:00100000:1.0:1787826803.070632:0:77480:0:(client.c:2715:ptlrpc_set_wait()) set ffff985611091b40 going to sleep for 11 seconds 00000100:00100000:1.0:1787826803.071865:0:77480:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610df83c0 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:77480:1874669225526656:192.168.203.114@tcp:35:createmany.0 00000002:00100000:1.0:1787826803.071890:0:77480:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff985610dfad00 x1874669225526400/t188978561038(188978561038) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826819 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:1.0:1787826803.071935:0:77480:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826803.071948:0:77480:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826803.071986:0:77480:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e0057 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 77480 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.071994:0:77480:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e0057 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 77480 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.072002:0:77480:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e0057 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 77480 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826803.072016:0:77480:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610df92c0 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:77480:1874669225526784:192.168.203.114@tcp:101:createmany.0 00000100:00100000:1.0:1787826803.072064:0:77480:0:(client.c:2715:ptlrpc_set_wait()) set ffff985611091b40 going to sleep for 11 seconds 00000100:00100000:1.0:1787826803.072925:0:77480:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610df92c0 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:77480:1874669225526784:192.168.203.114@tcp:101:createmany.0 00010000:00010000:1.0:1787826803.072940:0:77480:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e0057 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a08a2 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.072956:0:77480:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e0057 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a08a2 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.072966:0:77480:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e0057 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a08a2 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.072976:0:77480:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e0057 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a08a2 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.072986:0:77480:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e0057 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a08a2 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.072993:0:77480:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e0057 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a08a2 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.073000:0:77480:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e0057 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a08a2 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826803.073015:0:77480:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff985610df92c0 x1874669225526784/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826819 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826803.073027:0:77480:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e0057 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a08a2 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.073036:0:77480:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:1.0:1787826803.073042:0:77480:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:1.0:1787826803.073060:0:77480:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:1.0:1787826803.073068:0:77480:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e0057 00000080:00010000:1.0:1787826803.073072:0:77480:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826803.073076:0:77480:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0057 from it ffffab3ac1dbfbf8 00010000:00010000:1.0:1787826803.073080:0:77480:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e0057 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a08a2 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.073088:0:77480:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e0057 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a08a2 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826803.073106:0:77480:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787826803.073120:0:77480:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826803.073143:0:77480:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826803.073152:0:77480:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e0057 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a08a2 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826803.073161:0:77480:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e0057 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a08a2 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826803.073167:0:77480:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826803.073172:0:77480:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826803.073174:0:77480:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e0057 00000080:00010000:1.0:1787826803.073178:0:77480:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0057 from it ffffab3ac1dbfb38 00010000:00010000:1.0:1787826803.073182:0:77480:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e0057 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a08a2 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826803.073189:0:77480:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e0057 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a08a2 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826803.073203:0:77480:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787826803.073208:0:77480:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f33a.replay-single-5, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), file ffff9856206d7080, open_flags 8042, mode 81a4 opened 0 00000080:00200000:1.0:1787826803.073216:0:77480:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f33a.replay-single-5, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=open|creat 00000002:00010000:1.0:1787826803.073234:0:77480:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f33a.replay-single-5,[0x2000153f1:0x6:0x0]) in obj [0x200000007:0x1:0x0], intent: open|creat flags 0100103 00010000:00010000:1.0:1787826803.073243:0:77480:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e0057 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x5aa6d504d91a08a2 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826803.073272:0:77480:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e0057 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a08a2 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826803.073276:0:77480:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826803.073278:0:77480:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e0057 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a08a2 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826803.073299:0:77480:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a200/0xb3f83dad1b7e005e lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 77480 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.073303:0:77480:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a200/0xb3f83dad1b7e005e lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 77480 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:1.0:1787826803.073309:0:77480:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:1.0:1787826803.073314:0:77480:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610df92c0 x1874669225527040/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:1.0:1787826803.073322:0:77480:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a200/0xb3f83dad1b7e005e lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 77480 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826803.073336:0:77480:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610df92c0 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:77480:1874669225527040:192.168.203.114@tcp:101:createmany.0 00000100:00100000:1.0:1787826803.073442:0:77480:0:(client.c:2715:ptlrpc_set_wait()) set ffff985611091b40 going to sleep for 11 seconds 00000100:00100000:1.0:1787826803.075031:0:77480:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610df92c0 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:77480:1874669225527040:192.168.203.114@tcp:101:createmany.0 00010000:00010000:1.0:1787826803.075048:0:77480:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a200/0xb3f83dad1b7e005e lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a08be expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.075067:0:77480:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a200/0xb3f83dad1b7e005e lrc: 4/0,1 mode: --/PR res: [0x2000153f1:0x6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a08be expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.075074:0:77480:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a200/0xb3f83dad1b7e005e lrc: 4/0,1 mode: --/PR res: [0x2000153f1:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a08be expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.075083:0:77480:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a200/0xb3f83dad1b7e005e lrc: 4/0,1 mode: PR/PR res: [0x2000153f1:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a08be expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.075093:0:77480:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a200/0xb3f83dad1b7e005e lrc: 4/0,1 mode: PR/PR res: [0x2000153f1:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a08be expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.075105:0:77480:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a200/0xb3f83dad1b7e005e lrc: 5/1,1 mode: PR/PR res: [0x2000153f1:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a08be expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.075112:0:77480:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a200/0xb3f83dad1b7e005e lrc: 5/1,1 mode: PR/PR res: [0x2000153f1:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a08be expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.075119:0:77480:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a200/0xb3f83dad1b7e005e lrc: 4/1,0 mode: PR/PR res: [0x2000153f1:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a08be expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826803.075129:0:77480:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff985610df92c0 x1874669225527040/t188978561040(188978561040) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/664 e 0 to 0 dl 1787826819 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:1.0:1787826803.075142:0:77480:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff985610df92c0 x1874669225527040/t188978561040(188978561040) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/664 e 0 to 0 dl 1787826819 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826803.075154:0:77480:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a200/0xb3f83dad1b7e005e lrc: 3/1,0 mode: PR/PR res: [0x2000153f1:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a08be expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:1.0:1787826803.075166:0:77480:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a200/0xb3f83dad1b7e005e lrc: 3/1,0 mode: PR/PR res: [0x2000153f1:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a08be expref: -99 pid: 77480 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.075174:0:77480:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8590021617/6 (0 0) 00000080:00200000:1.0:1787826803.075180:0:77480:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:1.0:1787826803.075190:0:77480:0:(namei.c:1628:ll_create_it()) VFS Op:name=f33a.replay-single-5, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=open|creat 00000080:00200000:1.0:1787826803.075216:0:77480:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x2000153f1:0x6:0x0](ffff985611676c08) 00000080:00200000:1.0:1787826803.075312:0:77480:0:(file.c:6854:ll_layout_conf()) [0x2000153f1:0x6:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:1.0:1787826803.075318:0:77480:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x2000153f1:0x6:0x0](ffff985611676c08) 00000080:00010000:1.0:1787826803.075325:0:77480:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x2000153f1:0x6:0x0](ffff985611676c08) for lock 0xb3f83dad1b7e005e 00000080:00010000:1.0:1787826803.075333:0:77480:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x2000153f1:0x6:0x0](ffff985611676c08) 00000080:00200008:1.0:1787826803.075343:0:77480:0:(file.c:1090:ll_file_open()) START file f33a.replay-single-5:[0x2000153f1:0x6:0x0](ffff985611676c08), flags 100102 00000002:00100000:1.0:1787826803.075371:0:77480:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff985610df92c0 x1874669225527040/t188978561040(188978561040) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826819 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:1.0:1787826803.075400:0:77480:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e005e from it ffff9856054b0660 00010000:00010000:1.0:1787826803.075405:0:77480:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a200/0xb3f83dad1b7e005e lrc: 3/1,0 mode: PR/PR res: [0x2000153f1:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a08be expref: -99 pid: 77480 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.075414:0:77480:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a200/0xb3f83dad1b7e005e lrc: 2/0,0 mode: PR/PR res: [0x2000153f1:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a08be expref: -99 pid: 77480 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826803.075437:0:77480:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x2000153f1:0x6:0x0](ffff985611676c08), inode mode 81a4 mask 40 00000080:00200008:1.0:1787826803.075456:0:77480:0:(file.c:479:ll_file_release()) START file f33a.replay-single-5:[0x2000153f1:0x6:0x0](ffff985611676c08), flags 100002 00000002:00100000:1.0:1787826803.075487:0:77480:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff985610df92c0 x1874669225527040/t188978561040(188978561040) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826819 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:1.0:1787826803.075494:0:77480:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:1.0:1787826803.075497:0:77480:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610dfbc00 x1874669225527296/t0(0) o35->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:1.0:1787826803.075510:0:77480:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610dfbc00 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:77480:1874669225527296:192.168.203.114@tcp:35:createmany.0 00000100:00100000:1.0:1787826803.075554:0:77480:0:(client.c:2715:ptlrpc_set_wait()) set ffff985611091840 going to sleep for 11 seconds 00000100:00100000:1.0:1787826803.076737:0:77480:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610dfbc00 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:77480:1874669225527296:192.168.203.114@tcp:35:createmany.0 00000002:00100000:1.0:1787826803.076771:0:77480:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff985610df92c0 x1874669225527040/t188978561040(188978561040) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826819 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:1.0:1787826803.076819:0:77480:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826803.076833:0:77480:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826803.076874:0:77480:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b800/0xb3f83dad1b7e0065 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 77480 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.076884:0:77480:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b800/0xb3f83dad1b7e0065 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 77480 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.076893:0:77480:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b800/0xb3f83dad1b7e0065 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 77480 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826803.076914:0:77480:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610df8b40 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:77480:1874669225527424:192.168.203.114@tcp:101:createmany.0 00000100:00100000:1.0:1787826803.076959:0:77480:0:(client.c:2715:ptlrpc_set_wait()) set ffff985611091840 going to sleep for 11 seconds 00000100:00100000:1.0:1787826803.084265:0:77480:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee12c0 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:77480:1874669225527936:192.168.203.114@tcp:35:createmany.0 00000002:00100000:1.0:1787826803.084306:0:77480:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff985602ee1e00 x1874669225527680/t188978561042(188978561042) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826819 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:1.0:1787826803.084374:0:77480:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826803.084391:0:77480:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826803.084446:0:77480:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e0073 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 77480 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.084457:0:77480:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e0073 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 77480 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.084467:0:77480:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e0073 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 77480 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826803.084490:0:77480:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610df8f00 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:77480:1874669225528064:192.168.203.114@tcp:101:createmany.0 00000100:00100000:1.0:1787826803.084572:0:77480:0:(client.c:2715:ptlrpc_set_wait()) set ffff985611091f00 going to sleep for 11 seconds 00000100:00100000:1.0:1787826803.086874:0:77480:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610df8f00 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:77480:1874669225528064:192.168.203.114@tcp:101:createmany.0 00010000:00010000:1.0:1787826803.086893:0:77480:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e0073 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0904 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.086904:0:77480:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e0073 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0904 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.086916:0:77480:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e0073 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0904 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.086932:0:77480:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e0073 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0904 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.086944:0:77480:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e0073 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0904 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.086953:0:77480:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e0073 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0904 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.086960:0:77480:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e0073 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0904 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826803.086973:0:77480:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff985610df8f00 x1874669225528064/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826819 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826803.086986:0:77480:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e0073 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0904 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.086997:0:77480:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:1.0:1787826803.087004:0:77480:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:1.0:1787826803.087026:0:77480:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:1.0:1787826803.087032:0:77480:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e0073 00000080:00010000:1.0:1787826803.087037:0:77480:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826803.087041:0:77480:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0073 from it ffffab3ac1dbfbf8 00010000:00010000:1.0:1787826803.087044:0:77480:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e0073 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0904 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.087058:0:77480:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e0073 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0904 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826803.087080:0:77480:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787826803.087097:0:77480:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826803.087109:0:77480:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826803.087119:0:77480:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e0073 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0904 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826803.087128:0:77480:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e0073 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0904 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826803.087135:0:77480:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826803.087139:0:77480:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826803.087142:0:77480:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e0073 00000080:00010000:1.0:1787826803.087145:0:77480:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0073 from it ffffab3ac1dbfb38 00010000:00010000:1.0:1787826803.087148:0:77480:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e0073 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0904 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826803.087155:0:77480:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e0073 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0904 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826803.087163:0:77480:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787826803.087167:0:77480:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f33a.replay-single-7, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), file ffff9856206d7e00, open_flags 8042, mode 81a4 opened 0 00000080:00200000:1.0:1787826803.087176:0:77480:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f33a.replay-single-7, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=open|creat 00000002:00010000:1.0:1787826803.087194:0:77480:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f33a.replay-single-7,[0x2000153f1:0x8:0x0]) in obj [0x200000007:0x1:0x0], intent: open|creat flags 0100103 00010000:00010000:1.0:1787826803.087205:0:77480:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e0073 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x5aa6d504d91a0904 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826803.087235:0:77480:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e0073 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a0904 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826803.087242:0:77480:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826803.087245:0:77480:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e0073 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a0904 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826803.087272:0:77480:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b600/0xb3f83dad1b7e007a lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 77480 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.087279:0:77480:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b600/0xb3f83dad1b7e007a lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 77480 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:1.0:1787826803.087289:0:77480:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:1.0:1787826803.087294:0:77480:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610df8f00 x1874669225528320/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:1.0:1787826803.087300:0:77480:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b600/0xb3f83dad1b7e007a lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 77480 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826803.087313:0:77480:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610df8f00 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:77480:1874669225528320:192.168.203.114@tcp:101:createmany.0 00000100:00100000:1.0:1787826803.087432:0:77480:0:(client.c:2715:ptlrpc_set_wait()) set ffff985611091f00 going to sleep for 11 seconds 00000100:00100000:1.0:1787826803.089554:0:77480:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610df8f00 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:77480:1874669225528320:192.168.203.114@tcp:101:createmany.0 00010000:00010000:1.0:1787826803.089575:0:77480:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b600/0xb3f83dad1b7e007a lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0920 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.089602:0:77480:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b600/0xb3f83dad1b7e007a lrc: 4/0,1 mode: --/PR res: [0x2000153f1:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0920 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.089611:0:77480:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b600/0xb3f83dad1b7e007a lrc: 4/0,1 mode: --/PR res: [0x2000153f1:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0920 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.089619:0:77480:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b600/0xb3f83dad1b7e007a lrc: 4/0,1 mode: PR/PR res: [0x2000153f1:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0920 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.089626:0:77480:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b600/0xb3f83dad1b7e007a lrc: 4/0,1 mode: PR/PR res: [0x2000153f1:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0920 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.089635:0:77480:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b600/0xb3f83dad1b7e007a lrc: 5/1,1 mode: PR/PR res: [0x2000153f1:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0920 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.089639:0:77480:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b600/0xb3f83dad1b7e007a lrc: 5/1,1 mode: PR/PR res: [0x2000153f1:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0920 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.089644:0:77480:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b600/0xb3f83dad1b7e007a lrc: 4/1,0 mode: PR/PR res: [0x2000153f1:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0920 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826803.089650:0:77480:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff985610df8f00 x1874669225528320/t188978561044(188978561044) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/664 e 0 to 0 dl 1787826819 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:1.0:1787826803.089658:0:77480:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff985610df8f00 x1874669225528320/t188978561044(188978561044) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/664 e 0 to 0 dl 1787826819 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826803.089670:0:77480:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b600/0xb3f83dad1b7e007a lrc: 3/1,0 mode: PR/PR res: [0x2000153f1:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0920 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:1.0:1787826803.089682:0:77480:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b600/0xb3f83dad1b7e007a lrc: 3/1,0 mode: PR/PR res: [0x2000153f1:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0920 expref: -99 pid: 77480 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.089775:0:77480:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8590021617/8 (0 0) 00000080:00200000:1.0:1787826803.089781:0:77480:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:1.0:1787826803.089793:0:77480:0:(namei.c:1628:ll_create_it()) VFS Op:name=f33a.replay-single-7, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=open|creat 00000080:00200000:1.0:1787826803.089816:0:77480:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x2000153f1:0x8:0x0](ffff9856116721c8) 00000080:00200000:1.0:1787826803.089894:0:77480:0:(file.c:6854:ll_layout_conf()) [0x2000153f1:0x8:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:1.0:1787826803.089901:0:77480:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x2000153f1:0x8:0x0](ffff9856116721c8) 00000080:00010000:1.0:1787826803.089904:0:77480:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x2000153f1:0x8:0x0](ffff9856116721c8) for lock 0xb3f83dad1b7e007a 00000080:00010000:1.0:1787826803.089915:0:77480:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x2000153f1:0x8:0x0](ffff9856116721c8) 00000080:00200008:1.0:1787826803.089930:0:77480:0:(file.c:1090:ll_file_open()) START file f33a.replay-single-7:[0x2000153f1:0x8:0x0](ffff9856116721c8), flags 100102 00000002:00100000:1.0:1787826803.089953:0:77480:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff985610df8f00 x1874669225528320/t188978561044(188978561044) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826819 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:1.0:1787826803.090025:0:77480:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e007a from it ffff98562070dba0 00010000:00010000:1.0:1787826803.090031:0:77480:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b600/0xb3f83dad1b7e007a lrc: 3/1,0 mode: PR/PR res: [0x2000153f1:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a0920 expref: -99 pid: 77480 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.090041:0:77480:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b600/0xb3f83dad1b7e007a lrc: 2/0,0 mode: PR/PR res: [0x2000153f1:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a0920 expref: -99 pid: 77480 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826803.090065:0:77480:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x2000153f1:0x8:0x0](ffff9856116721c8), inode mode 81a4 mask 40 00000080:00200008:1.0:1787826803.090093:0:77480:0:(file.c:479:ll_file_release()) START file f33a.replay-single-7:[0x2000153f1:0x8:0x0](ffff9856116721c8), flags 100002 00000002:00100000:1.0:1787826803.090135:0:77480:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff985610df8f00 x1874669225528320/t188978561044(188978561044) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826819 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:1.0:1787826803.090147:0:77480:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:1.0:1787826803.090154:0:77480:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610dfa580 x1874669225528576/t0(0) o35->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:1.0:1787826803.090178:0:77480:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610dfa580 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:77480:1874669225528576:192.168.203.114@tcp:35:createmany.0 00000100:00100000:1.0:1787826803.090251:0:77480:0:(client.c:2715:ptlrpc_set_wait()) set ffff985611091a80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826803.091394:0:77480:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610dfa580 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:77480:1874669225528576:192.168.203.114@tcp:35:createmany.0 00000002:00100000:1.0:1787826803.091418:0:77480:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff985610df8f00 x1874669225528320/t188978561044(188978561044) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826819 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:1.0:1787826803.091466:0:77480:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826803.091480:0:77480:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826803.091520:0:77480:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b000/0xb3f83dad1b7e0081 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 77480 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.091534:0:77480:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b000/0xb3f83dad1b7e0081 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 77480 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.091544:0:77480:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b000/0xb3f83dad1b7e0081 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 77480 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826803.091559:0:77480:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610dfb480 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:77480:1874669225528704:192.168.203.114@tcp:101:createmany.0 00000100:00100000:1.0:1787826803.091616:0:77480:0:(client.c:2715:ptlrpc_set_wait()) set ffff985611091a80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826803.093122:0:77480:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610dfb480 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:77480:1874669225528704:192.168.203.114@tcp:101:createmany.0 00010000:00010000:1.0:1787826803.093136:0:77480:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b000/0xb3f83dad1b7e0081 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0935 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.093144:0:77480:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b000/0xb3f83dad1b7e0081 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0935 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.093156:0:77480:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b000/0xb3f83dad1b7e0081 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0935 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.093163:0:77480:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b000/0xb3f83dad1b7e0081 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0935 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.093171:0:77480:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b000/0xb3f83dad1b7e0081 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0935 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.093177:0:77480:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b000/0xb3f83dad1b7e0081 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0935 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.093181:0:77480:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b000/0xb3f83dad1b7e0081 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0935 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826803.093188:0:77480:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff985610dfb480 x1874669225528704/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826819 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826803.093197:0:77480:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b000/0xb3f83dad1b7e0081 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0935 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.093203:0:77480:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:1.0:1787826803.093207:0:77480:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:1.0:1787826803.093224:0:77480:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:1.0:1787826803.093228:0:77480:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e0081 00000080:00010000:1.0:1787826803.093231:0:77480:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826803.093233:0:77480:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0081 from it ffffab3ac1dbfbf8 00010000:00010000:1.0:1787826803.093236:0:77480:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b000/0xb3f83dad1b7e0081 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0935 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.093242:0:77480:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b000/0xb3f83dad1b7e0081 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0935 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826803.093259:0:77480:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787826803.093269:0:77480:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826803.093278:0:77480:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826803.093283:0:77480:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b000/0xb3f83dad1b7e0081 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0935 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826803.093289:0:77480:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b000/0xb3f83dad1b7e0081 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0935 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826803.093294:0:77480:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826803.093297:0:77480:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826803.093299:0:77480:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e0081 00000080:00010000:1.0:1787826803.093302:0:77480:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0081 from it ffffab3ac1dbfb38 00010000:00010000:1.0:1787826803.093304:0:77480:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b000/0xb3f83dad1b7e0081 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0935 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826803.093308:0:77480:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b000/0xb3f83dad1b7e0081 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0935 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826803.093314:0:77480:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787826803.093317:0:77480:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f33a.replay-single-8, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), file ffff9856206d7680, open_flags 8042, mode 81a4 opened 0 00000080:00200000:1.0:1787826803.093323:0:77480:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f33a.replay-single-8, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=open|creat 00000002:00010000:1.0:1787826803.093341:0:77480:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f33a.replay-single-8,[0x2000153f1:0x9:0x0]) in obj [0x200000007:0x1:0x0], intent: open|creat flags 0100103 00010000:00010000:1.0:1787826803.093351:0:77480:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b000/0xb3f83dad1b7e0081 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x5aa6d504d91a0935 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826803.093404:0:77480:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b000/0xb3f83dad1b7e0081 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a0935 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826803.093409:0:77480:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826803.093411:0:77480:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b000/0xb3f83dad1b7e0081 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a0935 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826803.093433:0:77480:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b800/0xb3f83dad1b7e0088 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 77480 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.093438:0:77480:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b800/0xb3f83dad1b7e0088 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 77480 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:1.0:1787826803.093445:0:77480:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:1.0:1787826803.093449:0:77480:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610dfb480 x1874669225528960/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:1.0:1787826803.093455:0:77480:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b800/0xb3f83dad1b7e0088 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 77480 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826803.093465:0:77480:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610dfb480 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:77480:1874669225528960:192.168.203.114@tcp:101:createmany.0 00000100:00100000:1.0:1787826803.093548:0:77480:0:(client.c:2715:ptlrpc_set_wait()) set ffff985611091a80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826803.095524:0:77480:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610dfb480 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:77480:1874669225528960:192.168.203.114@tcp:101:createmany.0 00010000:00010000:1.0:1787826803.095541:0:77480:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b800/0xb3f83dad1b7e0088 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0951 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.095566:0:77480:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b800/0xb3f83dad1b7e0088 lrc: 4/0,1 mode: --/PR res: [0x2000153f1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0951 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.095573:0:77480:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b800/0xb3f83dad1b7e0088 lrc: 4/0,1 mode: --/PR res: [0x2000153f1:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0951 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.095581:0:77480:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b800/0xb3f83dad1b7e0088 lrc: 4/0,1 mode: PR/PR res: [0x2000153f1:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0951 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.095590:0:77480:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b800/0xb3f83dad1b7e0088 lrc: 4/0,1 mode: PR/PR res: [0x2000153f1:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0951 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.095600:0:77480:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b800/0xb3f83dad1b7e0088 lrc: 5/1,1 mode: PR/PR res: [0x2000153f1:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0951 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.095608:0:77480:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b800/0xb3f83dad1b7e0088 lrc: 5/1,1 mode: PR/PR res: [0x2000153f1:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0951 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.095615:0:77480:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b800/0xb3f83dad1b7e0088 lrc: 4/1,0 mode: PR/PR res: [0x2000153f1:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0951 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826803.095625:0:77480:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff985610dfb480 x1874669225528960/t188978561046(188978561046) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/664 e 0 to 0 dl 1787826819 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:1.0:1787826803.095633:0:77480:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff985610dfb480 x1874669225528960/t188978561046(188978561046) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/664 e 0 to 0 dl 1787826819 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826803.095641:0:77480:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b800/0xb3f83dad1b7e0088 lrc: 3/1,0 mode: PR/PR res: [0x2000153f1:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0951 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:1.0:1787826803.095652:0:77480:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b800/0xb3f83dad1b7e0088 lrc: 3/1,0 mode: PR/PR res: [0x2000153f1:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0951 expref: -99 pid: 77480 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.095661:0:77480:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8590021617/9 (0 0) 00000080:00200000:1.0:1787826803.095666:0:77480:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:1.0:1787826803.095678:0:77480:0:(namei.c:1628:ll_create_it()) VFS Op:name=f33a.replay-single-8, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=open|creat 00000080:00200000:1.0:1787826803.095732:0:77480:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x2000153f1:0x9:0x0](ffff985611675348) 00000080:00200000:1.0:1787826803.095800:0:77480:0:(file.c:6854:ll_layout_conf()) [0x2000153f1:0x9:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:1.0:1787826803.095805:0:77480:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x2000153f1:0x9:0x0](ffff985611675348) 00000080:00010000:1.0:1787826803.095808:0:77480:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x2000153f1:0x9:0x0](ffff985611675348) for lock 0xb3f83dad1b7e0088 00000080:00010000:1.0:1787826803.095816:0:77480:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x2000153f1:0x9:0x0](ffff985611675348) 00000080:00200008:1.0:1787826803.095828:0:77480:0:(file.c:1090:ll_file_open()) START file f33a.replay-single-8:[0x2000153f1:0x9:0x0](ffff985611675348), flags 100102 00000002:00100000:1.0:1787826803.095846:0:77480:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff985610dfb480 x1874669225528960/t188978561046(188978561046) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826819 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:1.0:1787826803.095887:0:77480:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0088 from it ffff9856094b6a80 00010000:00010000:1.0:1787826803.095891:0:77480:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b800/0xb3f83dad1b7e0088 lrc: 3/1,0 mode: PR/PR res: [0x2000153f1:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a0951 expref: -99 pid: 77480 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.095899:0:77480:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b800/0xb3f83dad1b7e0088 lrc: 2/0,0 mode: PR/PR res: [0x2000153f1:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a0951 expref: -99 pid: 77480 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826803.095916:0:77480:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x2000153f1:0x9:0x0](ffff985611675348), inode mode 81a4 mask 40 00000080:00200008:1.0:1787826803.095947:0:77480:0:(file.c:479:ll_file_release()) START file f33a.replay-single-8:[0x2000153f1:0x9:0x0](ffff985611675348), flags 100002 00000002:00100000:1.0:1787826803.095989:0:77480:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff985610dfb480 x1874669225528960/t188978561046(188978561046) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826819 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:1.0:1787826803.096004:0:77480:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:1.0:1787826803.096009:0:77480:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610dfa940 x1874669225529216/t0(0) o35->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:1.0:1787826803.096026:0:77480:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610dfa940 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:77480:1874669225529216:192.168.203.114@tcp:35:createmany.0 00000100:00100000:1.0:1787826803.096083:0:77480:0:(client.c:2715:ptlrpc_set_wait()) set ffff9856110916c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826803.097373:0:77480:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610dfa940 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:77480:1874669225529216:192.168.203.114@tcp:35:createmany.0 00000002:00100000:1.0:1787826803.097387:0:77480:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff985610dfb480 x1874669225528960/t188978561046(188978561046) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826819 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:1.0:1787826803.097428:0:77480:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826803.097438:0:77480:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826803.097471:0:77480:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e008f lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 77480 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.097479:0:77480:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e008f lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 77480 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.097497:0:77480:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e008f lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 77480 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826803.097511:0:77480:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610df9e00 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:77480:1874669225529344:192.168.203.114@tcp:101:createmany.0 00000100:00100000:1.0:1787826803.097552:0:77480:0:(client.c:2715:ptlrpc_set_wait()) set ffff9856110916c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826803.098590:0:77480:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610df9e00 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:77480:1874669225529344:192.168.203.114@tcp:101:createmany.0 00010000:00010000:1.0:1787826803.098600:0:77480:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e008f lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0966 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.098611:0:77480:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e008f lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0966 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.098619:0:77480:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e008f lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0966 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.098624:0:77480:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e008f lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0966 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.098631:0:77480:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e008f lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0966 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.098636:0:77480:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e008f lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0966 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.098640:0:77480:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e008f lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0966 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826803.098646:0:77480:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff985610df9e00 x1874669225529344/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826819 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826803.098654:0:77480:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e008f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0966 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.098660:0:77480:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:1.0:1787826803.098663:0:77480:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:1.0:1787826803.098677:0:77480:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:1.0:1787826803.098681:0:77480:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e008f 00000080:00010000:1.0:1787826803.098684:0:77480:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826803.098686:0:77480:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e008f from it ffffab3ac1dbfbf8 00010000:00010000:1.0:1787826803.098720:0:77480:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e008f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0966 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.098730:0:77480:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e008f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0966 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826803.098750:0:77480:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787826803.098766:0:77480:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826803.098775:0:77480:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826803.098783:0:77480:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e008f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0966 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826803.098792:0:77480:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e008f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0966 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826803.098799:0:77480:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826803.098803:0:77480:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826803.098807:0:77480:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e008f 00000080:00010000:1.0:1787826803.098812:0:77480:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e008f from it ffffab3ac1dbfb38 00010000:00010000:1.0:1787826803.098815:0:77480:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e008f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0966 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826803.098824:0:77480:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e008f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0966 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826803.098836:0:77480:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787826803.098840:0:77480:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f33a.replay-single-9, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), file ffff9856206d6600, open_flags 8042, mode 81a4 opened 0 00000080:00200000:1.0:1787826803.098846:0:77480:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f33a.replay-single-9, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=open|creat 00000002:00010000:1.0:1787826803.098860:0:77480:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f33a.replay-single-9,[0x2000153f1:0xa:0x0]) in obj [0x200000007:0x1:0x0], intent: open|creat flags 0100103 00010000:00010000:1.0:1787826803.098867:0:77480:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e008f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x5aa6d504d91a0966 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826803.098906:0:77480:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e008f lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a0966 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826803.098914:0:77480:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826803.098917:0:77480:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e008f lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a0966 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826803.098943:0:77480:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b000/0xb3f83dad1b7e0096 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 77480 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.098951:0:77480:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b000/0xb3f83dad1b7e0096 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 77480 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:1.0:1787826803.098959:0:77480:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:1.0:1787826803.098966:0:77480:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610df9e00 x1874669225529600/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:1.0:1787826803.098975:0:77480:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b000/0xb3f83dad1b7e0096 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 77480 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826803.098987:0:77480:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610df9e00 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:77480:1874669225529600:192.168.203.114@tcp:101:createmany.0 00000100:00100000:1.0:1787826803.099072:0:77480:0:(client.c:2715:ptlrpc_set_wait()) set ffff9856110916c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826803.100684:0:77480:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610df9e00 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:77480:1874669225529600:192.168.203.114@tcp:101:createmany.0 00010000:00010000:1.0:1787826803.100719:0:77480:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b000/0xb3f83dad1b7e0096 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0982 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.100739:0:77480:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b000/0xb3f83dad1b7e0096 lrc: 4/0,1 mode: --/PR res: [0x2000153f1:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0982 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.100746:0:77480:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b000/0xb3f83dad1b7e0096 lrc: 4/0,1 mode: --/PR res: [0x2000153f1:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0982 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.100755:0:77480:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b000/0xb3f83dad1b7e0096 lrc: 4/0,1 mode: PR/PR res: [0x2000153f1:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0982 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.100766:0:77480:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b000/0xb3f83dad1b7e0096 lrc: 4/0,1 mode: PR/PR res: [0x2000153f1:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0982 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.100774:0:77480:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b000/0xb3f83dad1b7e0096 lrc: 5/1,1 mode: PR/PR res: [0x2000153f1:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0982 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.100779:0:77480:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b000/0xb3f83dad1b7e0096 lrc: 5/1,1 mode: PR/PR res: [0x2000153f1:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0982 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.100783:0:77480:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b000/0xb3f83dad1b7e0096 lrc: 4/1,0 mode: PR/PR res: [0x2000153f1:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0982 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826803.100789:0:77480:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff985610df9e00 x1874669225529600/t188978561048(188978561048) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/664 e 0 to 0 dl 1787826819 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:1.0:1787826803.100803:0:77480:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff985610df9e00 x1874669225529600/t188978561048(188978561048) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/664 e 0 to 0 dl 1787826819 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826803.100811:0:77480:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b000/0xb3f83dad1b7e0096 lrc: 3/1,0 mode: PR/PR res: [0x2000153f1:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0982 expref: -99 pid: 77480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:1.0:1787826803.100818:0:77480:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b000/0xb3f83dad1b7e0096 lrc: 3/1,0 mode: PR/PR res: [0x2000153f1:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0982 expref: -99 pid: 77480 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.100824:0:77480:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8590021617/10 (0 0) 00000080:00200000:1.0:1787826803.100827:0:77480:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:1.0:1787826803.100835:0:77480:0:(namei.c:1628:ll_create_it()) VFS Op:name=f33a.replay-single-9, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=open|creat 00000080:00200000:1.0:1787826803.100849:0:77480:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x2000153f1:0xa:0x0](ffff985611677448) 00000080:00200000:1.0:1787826803.100913:0:77480:0:(file.c:6854:ll_layout_conf()) [0x2000153f1:0xa:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:1.0:1787826803.100918:0:77480:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x2000153f1:0xa:0x0](ffff985611677448) 00000080:00010000:1.0:1787826803.100921:0:77480:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x2000153f1:0xa:0x0](ffff985611677448) for lock 0xb3f83dad1b7e0096 00000080:00010000:1.0:1787826803.100928:0:77480:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x2000153f1:0xa:0x0](ffff985611677448) 00000080:00200008:1.0:1787826803.100938:0:77480:0:(file.c:1090:ll_file_open()) START file f33a.replay-single-9:[0x2000153f1:0xa:0x0](ffff985611677448), flags 100102 00000002:00100000:1.0:1787826803.100956:0:77480:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff985610df9e00 x1874669225529600/t188978561048(188978561048) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826819 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:1.0:1787826803.100983:0:77480:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0096 from it ffff985610e3fc00 00010000:00010000:1.0:1787826803.100988:0:77480:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b000/0xb3f83dad1b7e0096 lrc: 3/1,0 mode: PR/PR res: [0x2000153f1:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a0982 expref: -99 pid: 77480 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826803.100996:0:77480:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9b000/0xb3f83dad1b7e0096 lrc: 2/0,0 mode: PR/PR res: [0x2000153f1:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a0982 expref: -99 pid: 77480 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826803.101015:0:77480:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x2000153f1:0xa:0x0](ffff985611677448), inode mode 81a4 mask 40 00000080:00200008:1.0:1787826803.101036:0:77480:0:(file.c:479:ll_file_release()) START file f33a.replay-single-9:[0x2000153f1:0xa:0x0](ffff985611677448), flags 100002 00000002:00100000:1.0:1787826803.101067:0:77480:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff985610df9e00 x1874669225529600/t188978561048(188978561048) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826819 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:1.0:1787826803.101076:0:77480:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:1.0:1787826803.101081:0:77480:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610df8780 x1874669225529856/t0(0) o35->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:1.0:1787826803.101096:0:77480:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610df8780 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:77480:1874669225529856:192.168.203.114@tcp:35:createmany.0 00000100:00100000:1.0:1787826803.101149:0:77480:0:(client.c:2715:ptlrpc_set_wait()) set ffff985611091d80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826803.102141:0:77480:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610df8780 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:77480:1874669225529856:192.168.203.114@tcp:35:createmany.0 00000002:00100000:1.0:1787826803.102160:0:77480:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff985610df9e00 x1874669225529600/t188978561048(188978561048) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826819 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000100:00100000:1.0:1787826806.005014:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610dfa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669225530112:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787826806.005144:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e321c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669225530368:192.168.203.114@tcp:400:kworker.0 00000100:00080000:1.0:1787826806.006602:0:2384:0:(recover.c:205:ptlrpc_request_handle_notconn()) import lustre-MDT0000-mdc-ffff985602bc9800 of lustre-MDT0000_UUID@192.168.203.114@tcp abruptly disconnected: reconnecting 00000100:00080000:1.0:1787826806.006609:0:2384:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from FULL to DISCONN 00000100:02000400:1.0:1787826806.006617:0:2384:0:(import.c:182:ptlrpc_set_import_discon()) lustre-MDT0000-mdc-ffff985602bc9800: Connection to lustre-MDT0000 (at 192.168.203.114@tcp) was lost; in progress operations using this service will wait for recovery to complete 00000100:00080000:1.0:1787826806.006623:0:2384:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787826806.006627:0:2384:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 2950 00000100:00080000:1.0:1787826806.006637:0:2384:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787826806.006651:0:2384:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787826806.006675:0:2384:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610e32580 x1874669225530624/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826806.006721:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610dfa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669225530112:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787826806.006734:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e321c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669225530368:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787826811.124990:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e643c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669225530880:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787826811.126369:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669225531392:192.168.203.114@tcp:250:ptlrpcd_01_01.0 00000100:00080000:1.0:1787826811.127685:0:2382:0:(import.c:1132:ptlrpc_connect_interpret()) MGC192.168.203.114@tcp: connect to target with instance 0 10000000:01000000:1.0:1787826811.127731:0:2382:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808005 00000100:02000400:1.0:1787826811.127741:0:2382:0:(import.c:1284:ptlrpc_connect_interpret()) Evicted from MGS (at 192.168.203.114@tcp) after server handle changed from 0x5aa6d504d91a049d to 0x5aa6d504d91a0c8b 00000100:00080000:1.0:1787826811.127746:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to EVICTED 00000100:00080000:1.0:1787826811.127751:0:2382:0:(import.c:1674:ptlrpc_import_recovery_state_machine()) evicted from MGS@192.168.203.114@tcp; invalidating 00000100:00100000:1.0:1787826811.134236:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e643c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669225530880:192.168.203.114@tcp:38:kworker.0 10000000:01000000:1.0:1787826812.340821:0:15166:0:(mgc_request.c:611:do_requeue()) updating log lustre-cliir 10000000:01000000:1.0:1787826812.340834:0:15166:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-cliir-ffff985602bc9800 from 180 10000000:01000000:1.0:1787826812.340842:0:15166:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-cliir (res 0x65727473756c) 00010000:00010000:1.0:1787826812.340917:0:15166:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.114@tcp lock: ffff985610e9b000/0xb3f83dad1b7e009d lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826812.340932:0:15166:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.114@tcp lock: ffff985610e9b000/0xb3f83dad1b7e009d lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826812.340945:0:15166:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.114@tcp lock: ffff985610e9b000/0xb3f83dad1b7e009d lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826812.340983:0:15166:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e312c0 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669225531520:192.168.203.114@tcp:101:ll_cfg_requeue.0 00000100:00100000:1.0:1787826812.341093:0:15166:0:(client.c:2715:ptlrpc_set_wait()) set ffff985611091c00 going to sleep for 11 seconds 00000100:00100000:1.0:1787826812.342532:0:15166:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e312c0 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669225531520:192.168.203.114@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787826812.342563:0:15166:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.114@tcp lock: ffff985610e9b000/0xb3f83dad1b7e009d lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x5aa6d504d91a0d56 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826812.342576:0:15166:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.114@tcp lock: ffff985610e9b000/0xb3f83dad1b7e009d lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x5aa6d504d91a0d56 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826812.342645:0:15166:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e312c0 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669225531648:192.168.203.114@tcp:256:ll_cfg_requeue.0 00000100:00100000:1.0:1787826812.342747:0:15166:0:(client.c:2715:ptlrpc_set_wait()) set ffff985611091c00 going to sleep for 11 seconds 00000100:00100000:1.0:1787826812.344737:0:15166:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e312c0 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669225531648:192.168.203.114@tcp:256:ll_cfg_requeue.0 10000000:01000000:1.0:1787826812.344830:0:15166:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.114@tcp: configuration from log 'lustre-cliir' succeeded (0). 00010000:00010000:1.0:1787826812.344840:0:15166:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.114@tcp lock: ffff985610e9b000/0xb3f83dad1b7e009d lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x5aa6d504d91a0d56 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826812.344850:0:15166:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.114@tcp lock: ffff985610e9b000/0xb3f83dad1b7e009d lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x5aa6d504d91a0d56 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787826812.344858:0:15166:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 10000000:01000000:1.0:1787826812.344860:0:15166:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff985602bc9800 from 52 10000000:01000000:1.0:1787826812.344863:0:15166:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00010000:00010000:1.0:1787826812.344896:0:15166:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.114@tcp lock: ffff985610e9b800/0xb3f83dad1b7e00a4 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826812.344903:0:15166:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.114@tcp lock: ffff985610e9b800/0xb3f83dad1b7e00a4 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826812.344913:0:15166:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.114@tcp lock: ffff985610e9b800/0xb3f83dad1b7e00a4 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826812.344932:0:15166:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e312c0 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669225531776:192.168.203.114@tcp:101:ll_cfg_requeue.0 00000100:00100000:1.0:1787826812.345009:0:15166:0:(client.c:2715:ptlrpc_set_wait()) set ffff985611091c00 going to sleep for 11 seconds 00000100:00100000:1.0:1787826812.345870:0:15166:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e312c0 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669225531776:192.168.203.114@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787826812.345898:0:15166:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.114@tcp lock: ffff985610e9b800/0xb3f83dad1b7e00a4 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x5aa6d504d91a0d5d expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826812.345908:0:15166:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.114@tcp lock: ffff985610e9b800/0xb3f83dad1b7e00a4 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x5aa6d504d91a0d5d expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826812.345952:0:15166:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e312c0 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669225531904:192.168.203.114@tcp:501:ll_cfg_requeue.0 00000100:00100000:1.0:1787826812.345982:0:15166:0:(client.c:2715:ptlrpc_set_wait()) set ffff985611091c00 going to sleep for 11 seconds 00000100:00100000:1.0:1787826812.346984:0:15166:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e312c0 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669225531904:192.168.203.114@tcp:501:ll_cfg_requeue.0 00000100:00100000:1.0:1787826812.347024:0:15166:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e312c0 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669225532032:192.168.203.114@tcp:503:ll_cfg_requeue.0 00000100:00100000:1.0:1787826812.347083:0:15166:0:(client.c:2715:ptlrpc_set_wait()) set ffff985611091c00 going to sleep for 11 seconds 00000100:00100000:1.0:1787826812.348066:0:15166:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e312c0 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669225532032:192.168.203.114@tcp:503:ll_cfg_requeue.0 00000040:00080000:1.0:1787826812.348824:0:77821:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x3:0x0] index 64768 count 51 00000100:00100000:1.0:1787826812.350459:0:15166:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e7a940 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669225532160:192.168.203.114@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787826812.350489:0:15166:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.114@tcp lock: ffff985609eb0c00/0xb3f83dad1b7e00ab lrc: 4/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x5aa6d504d91a0d64 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826812.350503:0:15166:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.114@tcp lock: ffff985609eb0c00/0xb3f83dad1b7e00ab lrc: 4/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x5aa6d504d91a0d64 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826812.350585:0:15166:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e312c0 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669225532288:192.168.203.114@tcp:501:ll_cfg_requeue.0 00000100:00100000:1.0:1787826812.350658:0:15166:0:(client.c:2715:ptlrpc_set_wait()) set ffff985611091780 going to sleep for 11 seconds 00000100:00100000:1.0:1787826812.351599:0:15166:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e312c0 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669225532288:192.168.203.114@tcp:501:ll_cfg_requeue.0 00000100:00100000:1.0:1787826812.351636:0:15166:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e312c0 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669225532416:192.168.203.114@tcp:503:ll_cfg_requeue.0 00000100:00100000:1.0:1787826812.351729:0:15166:0:(client.c:2715:ptlrpc_set_wait()) set ffff985611091780 going to sleep for 11 seconds 00000100:00100000:1.0:1787826812.352720:0:15166:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e312c0 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669225532416:192.168.203.114@tcp:503:ll_cfg_requeue.0 00000020:01000000:1.0:1787826812.353497:0:15166:0:(obd_config.c:2145:class_config_parse_llog()) Processed log params gen 7-6 (rc=0) 10000000:01000000:1.0:1787826812.353520:0:15166:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.114@tcp: configuration from log 'params' succeeded (0). 00010000:00010000:1.0:1787826812.353534:0:15166:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.114@tcp lock: ffff985609eb0c00/0xb3f83dad1b7e00ab lrc: 3/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x5aa6d504d91a0d64 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826812.353546:0:15166:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.114@tcp lock: ffff985609eb0c00/0xb3f83dad1b7e00ab lrc: 2/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x5aa6d504d91a0d64 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787826812.353557:0:15166:0:(mgc_request.c:611:do_requeue()) updating log lustre-sptlrpc 10000000:01000000:1.0:1787826812.353559:0:15166:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-sptlrpc-ffff985606a30000 from 1 10000000:01000000:1.0:1787826812.353563:0:15166:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-sptlrpc (res 0x65727473756c) 00010000:00010000:1.0:1787826812.353596:0:15166:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.114@tcp lock: ffff985610e9b600/0xb3f83dad1b7e00b2 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826812.353605:0:15166:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.114@tcp lock: ffff985610e9b600/0xb3f83dad1b7e00b2 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826812.353621:0:15166:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.114@tcp lock: ffff985610e9b600/0xb3f83dad1b7e00b2 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826812.353639:0:15166:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e312c0 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669225532544:192.168.203.114@tcp:101:ll_cfg_requeue.0 00000100:00100000:1.0:1787826812.353781:0:15166:0:(client.c:2715:ptlrpc_set_wait()) set ffff985611091780 going to sleep for 11 seconds 00000100:00100000:1.0:1787826812.354687:0:15166:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e312c0 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669225532544:192.168.203.114@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787826812.354739:0:15166:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.114@tcp lock: ffff985610e9b600/0xb3f83dad1b7e00b2 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0x5aa6d504d91a0d6b expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826812.354750:0:15166:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.114@tcp lock: ffff985610e9b600/0xb3f83dad1b7e00b2 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1000000000000 nid: local remote: 0x5aa6d504d91a0d6b expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826812.354799:0:15166:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e312c0 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669225532672:192.168.203.114@tcp:501:ll_cfg_requeue.0 00000100:00100000:1.0:1787826812.354847:0:15166:0:(client.c:2715:ptlrpc_set_wait()) set ffff985611091780 going to sleep for 11 seconds 00000100:00100000:1.0:1787826812.355748:0:15166:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e312c0 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669225532672:192.168.203.114@tcp:501:ll_cfg_requeue.0 10000000:01000000:1.0:1787826812.355770:0:15166:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.114@tcp: configuration from log 'lustre-sptlrpc' failed (-2). 00010000:00010000:1.0:1787826812.355776:0:15166:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.114@tcp lock: ffff985610e9b600/0xb3f83dad1b7e00b2 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0x5aa6d504d91a0d6b expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826812.355784:0:15166:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.114@tcp lock: ffff985610e9b600/0xb3f83dad1b7e00b2 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0x5aa6d504d91a0d6b expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826812.724882:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200008:1.0:1787826814.230349:0:77862:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:1.0:1787826814.230360:0:77862:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826814.230379:0:77862:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826814.230423:0:77862:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ba00/0xb3f83dad1b7e00b9 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 77862 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826814.230430:0:77862:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ba00/0xb3f83dad1b7e00b9 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 77862 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826814.230437:0:77862:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ba00/0xb3f83dad1b7e00b9 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 77862 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826814.230454:0:77862:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e312c0 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:77862:1874669225532800:192.168.203.114@tcp:101:lfs.0 00000100:00100000:1.0:1787826814.230515:0:77862:0:(client.c:2715:ptlrpc_set_wait()) set ffff985611091e40 going to sleep for 11 seconds 00000100:00100000:1.0:1787826814.239000:0:77862:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e312c0 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:77862:1874669225532800:192.168.203.114@tcp:101:lfs.0 00010000:00010000:1.0:1787826814.239026:0:77862:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ba00/0xb3f83dad1b7e00b9 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0d79 expref: -99 pid: 77862 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826814.239038:0:77862:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ba00/0xb3f83dad1b7e00b9 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0d79 expref: -99 pid: 77862 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826814.239051:0:77862:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ba00/0xb3f83dad1b7e00b9 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0d79 expref: -99 pid: 77862 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826814.239062:0:77862:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ba00/0xb3f83dad1b7e00b9 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0d79 expref: -99 pid: 77862 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826814.239073:0:77862:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ba00/0xb3f83dad1b7e00b9 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0d79 expref: -99 pid: 77862 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826814.239082:0:77862:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ba00/0xb3f83dad1b7e00b9 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0d79 expref: -99 pid: 77862 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826814.239090:0:77862:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ba00/0xb3f83dad1b7e00b9 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0d79 expref: -99 pid: 77862 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826814.239102:0:77862:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff985610e312c0 x1874669225532800/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826830 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826814.239117:0:77862:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ba00/0xb3f83dad1b7e00b9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0d79 expref: -99 pid: 77862 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826814.239129:0:77862:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:1.0:1787826814.239135:0:77862:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:1.0:1787826814.239162:0:77862:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:1.0:1787826814.239167:0:77862:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e00b9 00000080:00010000:1.0:1787826814.239173:0:77862:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826814.239176:0:77862:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e00b9 from it ffffab3ac0c97cb8 00010000:00010000:1.0:1787826814.239180:0:77862:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ba00/0xb3f83dad1b7e00b9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0d79 expref: -99 pid: 77862 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826814.239189:0:77862:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ba00/0xb3f83dad1b7e00b9 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0d79 expref: -99 pid: 77862 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:1.0:1787826814.239931:0:77862:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 0, flags 0, foreign 0 00000080:00200000:1.0:1787826814.239938:0:77862:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826814.239955:0:77862:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826814.239966:0:77862:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ba00/0xb3f83dad1b7e00b9 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0d79 expref: -99 pid: 77862 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826814.239983:0:77862:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ba00/0xb3f83dad1b7e00b9 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0d79 expref: -99 pid: 77862 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826814.239990:0:77862:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826814.239995:0:77862:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826814.239998:0:77862:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e00b9 00000080:00010000:1.0:1787826814.240002:0:77862:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e00b9 from it ffffab3ac0c97cb8 00010000:00010000:1.0:1787826814.240006:0:77862:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ba00/0xb3f83dad1b7e00b9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0d79 expref: -99 pid: 77862 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826814.240013:0:77862:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ba00/0xb3f83dad1b7e00b9 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0d79 expref: -99 pid: 77862 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826814.240103:0:77862:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826814.240112:0:77862:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826814.240119:0:77862:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ba00/0xb3f83dad1b7e00b9 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0d79 expref: -99 pid: 77862 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826814.240128:0:77862:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ba00/0xb3f83dad1b7e00b9 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0d79 expref: -99 pid: 77862 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826814.240135:0:77862:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826814.240139:0:77862:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826814.240141:0:77862:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e00b9 00000080:00010000:1.0:1787826814.240145:0:77862:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e00b9 from it ffffab3ac0c97bb0 00010000:00010000:1.0:1787826814.240148:0:77862:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ba00/0xb3f83dad1b7e00b9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0d79 expref: -99 pid: 77862 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826814.240156:0:77862:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ba00/0xb3f83dad1b7e00b9 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0d79 expref: -99 pid: 77862 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787826814.240164:0:77862:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 44 00000080:00200008:1.0:1787826814.240171:0:77862:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 100000 00000080:00200080:1.0:1787826814.240299:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00800000:00000080:1.0:1787826814.240315:0:77862:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00800000:00000004:1.0:1787826814.240323:0:77862:0:(obd_class.h:1091:obd_statfs()) lustre-MDT0000-mdc-ffff985602bc9800: age 2951, max_age 2964 00000100:00100000:1.0:1787826814.240357:0:77862:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e312c0 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:77862:1874669225532928:192.168.203.114@tcp:41:lfs.0 00000100:00100000:1.0:1787826814.240436:0:77862:0:(client.c:2715:ptlrpc_set_wait()) set ffff985611091e40 going to sleep for 11 seconds 00000100:00100000:1.0:1787826814.241596:0:77862:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e312c0 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:77862:1874669225532928:192.168.203.114@tcp:41:lfs.0 00800000:00000004:1.0:1787826814.241615:0:77862:0:(obd_class.h:1117:obd_statfs()) lustre-MDT0000-mdc-ffff985602bc9800: update ffff985606a337f0 cache blocks 317388/353529 objects 1023694/1024000 00000080:00200080:1.0:1787826814.241670:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00800000:00000080:1.0:1787826814.241681:0:77862:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00800000:00000004:1.0:1787826814.241710:0:77862:0:(obd_class.h:1091:obd_statfs()) lustre-MDT0001-mdc-ffff985602bc9800: age 2950, max_age 2964 00000100:00100000:1.0:1787826814.241732:0:77862:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e312c0 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:77862:1874669225533056:192.168.203.114@tcp:41:lfs.0 00000100:00100000:1.0:1787826814.241773:0:77862:0:(client.c:2715:ptlrpc_set_wait()) set ffff985611091e40 going to sleep for 11 seconds 00000100:00100000:1.0:1787826814.243350:0:77862:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e312c0 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:77862:1874669225533056:192.168.203.114@tcp:41:lfs.0 00800000:00000004:1.0:1787826814.243364:0:77862:0:(obd_class.h:1117:obd_statfs()) lustre-MDT0001-mdc-ffff985602bc9800: update ffff98560952b7f0 cache blocks 317433/353529 objects 1023741/1024000 00000080:00200080:1.0:1787826814.243405:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00800000:00000080:1.0:1787826814.243415:0:77862:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.243422:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00800000:00000080:1.0:1787826814.243426:0:77862:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.243431:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00800000:00000080:1.0:1787826814.243435:0:77862:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.243439:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00800000:00000080:1.0:1787826814.243443:0:77862:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.243449:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00800000:00000080:1.0:1787826814.243453:0:77862:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.243458:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00800000:00000080:1.0:1787826814.243462:0:77862:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.243467:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00800000:00000080:1.0:1787826814.243471:0:77862:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.243474:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00800000:00000080:1.0:1787826814.243478:0:77862:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.243481:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00800000:00000080:1.0:1787826814.243484:0:77862:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.243487:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00800000:00000080:1.0:1787826814.243489:0:77862:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.243492:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00800000:00000080:1.0:1787826814.243494:0:77862:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.243497:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00800000:00000080:1.0:1787826814.243501:0:77862:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.243505:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00800000:00000080:1.0:1787826814.243509:0:77862:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.243513:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00800000:00000080:1.0:1787826814.243516:0:77862:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.243521:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00800000:00000080:1.0:1787826814.243525:0:77862:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.243530:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00800000:00000080:1.0:1787826814.243534:0:77862:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.243540:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00800000:00000080:1.0:1787826814.243545:0:77862:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.243551:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00800000:00000080:1.0:1787826814.243554:0:77862:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.243557:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00800000:00000080:1.0:1787826814.243559:0:77862:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.243561:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00800000:00000080:1.0:1787826814.243564:0:77862:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.243567:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00800000:00000080:1.0:1787826814.243569:0:77862:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.243571:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00800000:00000080:1.0:1787826814.243572:0:77862:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.243574:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00800000:00000080:1.0:1787826814.243576:0:77862:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.243578:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00800000:00000080:1.0:1787826814.243580:0:77862:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.243582:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00800000:00000080:1.0:1787826814.243584:0:77862:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.243587:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00800000:00000080:1.0:1787826814.243589:0:77862:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.243591:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00800000:00000080:1.0:1787826814.243593:0:77862:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.243596:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00800000:00000080:1.0:1787826814.243599:0:77862:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.243602:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00800000:00000080:1.0:1787826814.243604:0:77862:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.243607:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00800000:00000080:1.0:1787826814.243610:0:77862:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.243613:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00800000:00000080:1.0:1787826814.243617:0:77862:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.243621:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00800000:00000080:1.0:1787826814.243633:0:77862:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.243638:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00800000:00000080:1.0:1787826814.243642:0:77862:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.243647:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00800000:00000080:1.0:1787826814.243651:0:77862:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.243655:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00800000:00000080:1.0:1787826814.243659:0:77862:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.243663:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00800000:00000080:1.0:1787826814.243667:0:77862:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.243672:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00800000:00000080:1.0:1787826814.243675:0:77862:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.243680:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00800000:00000080:1.0:1787826814.243684:0:77862:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.243688:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00800000:00000080:1.0:1787826814.243722:0:77862:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.243729:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00800000:00000080:1.0:1787826814.243735:0:77862:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.243739:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00800000:00000080:1.0:1787826814.243748:0:77862:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.243753:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00800000:00000080:1.0:1787826814.243757:0:77862:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.243760:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00800000:00000080:1.0:1787826814.243763:0:77862:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.243767:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00800000:00000080:1.0:1787826814.243770:0:77862:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.243773:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00800000:00000080:1.0:1787826814.243777:0:77862:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.243780:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00800000:00000080:1.0:1787826814.243784:0:77862:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.243798:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00800000:00000080:1.0:1787826814.243802:0:77862:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.243806:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00800000:00000080:1.0:1787826814.243810:0:77862:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.243814:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00800000:00000080:1.0:1787826814.243817:0:77862:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.243820:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00800000:00000080:1.0:1787826814.243823:0:77862:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.243827:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00800000:00000080:1.0:1787826814.243831:0:77862:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.243834:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00800000:00000080:1.0:1787826814.243838:0:77862:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.243842:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00800000:00000080:1.0:1787826814.243846:0:77862:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.243850:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00800000:00000080:1.0:1787826814.243853:0:77862:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.243857:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00800000:00000080:1.0:1787826814.243861:0:77862:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.243865:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00800000:00000080:1.0:1787826814.243868:0:77862:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.243872:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00800000:00000080:1.0:1787826814.243875:0:77862:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.243879:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00800000:00000080:1.0:1787826814.243882:0:77862:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.243886:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00800000:00000080:1.0:1787826814.243890:0:77862:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.243895:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00800000:00000080:1.0:1787826814.243899:0:77862:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.243904:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00800000:00000080:1.0:1787826814.243908:0:77862:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.243913:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00800000:00000080:1.0:1787826814.243917:0:77862:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.243922:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00800000:00000080:1.0:1787826814.243926:0:77862:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.243930:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00020000:00000080:1.0:1787826814.243936:0:77862:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00020000:00000004:1.0:1787826814.243943:0:77862:0:(obd_class.h:1091:obd_statfs()) lustre-OST0000-osc-ffff985602bc9800: age 2950, max_age 2964 00000100:00100000:1.0:1787826814.243969:0:77862:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e312c0 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:77862:1874669225533184:192.168.203.114@tcp:13:lfs.0 00000100:00100000:1.0:1787826814.244013:0:77862:0:(client.c:2715:ptlrpc_set_wait()) set ffff985611091e40 going to sleep for 11 seconds 00000100:00100000:1.0:1787826814.244983:0:77862:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e312c0 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:77862:1874669225533184:192.168.203.114@tcp:13:lfs.0 00020000:00000004:1.0:1787826814.244997:0:77862:0:(obd_class.h:1117:obd_statfs()) lustre-OST0000-osc-ffff985602bc9800: update ffff985609528450 cache blocks 901357/958279 objects 261728/262144 00000080:00200080:1.0:1787826814.245035:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00020000:00000080:1.0:1787826814.245043:0:77862:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00020000:00000004:1.0:1787826814.245047:0:77862:0:(obd_class.h:1091:obd_statfs()) lustre-OST0001-osc-ffff985602bc9800: age 2950, max_age 2964 00000100:00100000:1.0:1787826814.245064:0:77862:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e312c0 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:77862:1874669225533312:192.168.203.114@tcp:13:lfs.0 00000100:00100000:1.0:1787826814.245105:0:77862:0:(client.c:2715:ptlrpc_set_wait()) set ffff985611091e40 going to sleep for 11 seconds 00000100:00100000:1.0:1787826814.246009:0:77862:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e312c0 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:77862:1874669225533312:192.168.203.114@tcp:13:lfs.0 00020000:00000004:1.0:1787826814.246023:0:77862:0:(obd_class.h:1117:obd_statfs()) lustre-OST0001-osc-ffff985602bc9800: update ffff98560952d1c0 cache blocks 901363/958279 objects 261735/262144 00000080:00200080:1.0:1787826814.246058:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00020000:00000080:1.0:1787826814.246067:0:77862:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.246072:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00020000:00000080:1.0:1787826814.246074:0:77862:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.246077:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00020000:00000080:1.0:1787826814.246080:0:77862:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.246083:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00020000:00000080:1.0:1787826814.246085:0:77862:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.246087:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00020000:00000080:1.0:1787826814.246089:0:77862:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.246091:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00020000:00000080:1.0:1787826814.246094:0:77862:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.246098:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00020000:00000080:1.0:1787826814.246109:0:77862:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.246114:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00020000:00000080:1.0:1787826814.246118:0:77862:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.246123:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00020000:00000080:1.0:1787826814.246127:0:77862:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.246131:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00020000:00000080:1.0:1787826814.246136:0:77862:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.246141:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00020000:00000080:1.0:1787826814.246145:0:77862:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.246148:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00020000:00000080:1.0:1787826814.246151:0:77862:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.246154:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00020000:00000080:1.0:1787826814.246156:0:77862:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.246158:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00020000:00000080:1.0:1787826814.246161:0:77862:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.246163:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00020000:00000080:1.0:1787826814.246165:0:77862:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.246168:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00020000:00000080:1.0:1787826814.246170:0:77862:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.246173:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00020000:00000080:1.0:1787826814.246178:0:77862:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.246183:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00020000:00000080:1.0:1787826814.246187:0:77862:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.246192:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00020000:00000080:1.0:1787826814.246195:0:77862:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.246201:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00020000:00000080:1.0:1787826814.246206:0:77862:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.246211:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00020000:00000080:1.0:1787826814.246216:0:77862:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.246221:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00020000:00000080:1.0:1787826814.246224:0:77862:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.246229:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00020000:00000080:1.0:1787826814.246233:0:77862:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.246236:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00020000:00000080:1.0:1787826814.246240:0:77862:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.246245:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00020000:00000080:1.0:1787826814.246250:0:77862:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.246255:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00020000:00000080:1.0:1787826814.246259:0:77862:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.246266:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00020000:00000080:1.0:1787826814.246271:0:77862:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.246288:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00020000:00000080:1.0:1787826814.246294:0:77862:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.246300:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00020000:00000080:1.0:1787826814.246304:0:77862:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.246307:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00020000:00000080:1.0:1787826814.246309:0:77862:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.246312:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00020000:00000080:1.0:1787826814.246315:0:77862:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.246317:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00020000:00000080:1.0:1787826814.246320:0:77862:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.246322:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00020000:00000080:1.0:1787826814.246324:0:77862:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.246326:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00020000:00000080:1.0:1787826814.246328:0:77862:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.246330:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00020000:00000080:1.0:1787826814.246331:0:77862:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.246333:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00020000:00000080:1.0:1787826814.246335:0:77862:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.246337:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00020000:00000080:1.0:1787826814.246339:0:77862:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.246341:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00020000:00000080:1.0:1787826814.246342:0:77862:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.246344:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00020000:00000080:1.0:1787826814.246346:0:77862:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.246348:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00020000:00000080:1.0:1787826814.246349:0:77862:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.246351:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00020000:00000080:1.0:1787826814.246353:0:77862:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.246355:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00020000:00000080:1.0:1787826814.246357:0:77862:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.246366:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00020000:00000080:1.0:1787826814.246370:0:77862:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.246374:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00020000:00000080:1.0:1787826814.246377:0:77862:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.246381:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00020000:00000080:1.0:1787826814.246384:0:77862:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.246388:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00020000:00000080:1.0:1787826814.246392:0:77862:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.246396:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00020000:00000080:1.0:1787826814.246399:0:77862:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.246403:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00020000:00000080:1.0:1787826814.246407:0:77862:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.246411:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00020000:00000080:1.0:1787826814.246414:0:77862:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.246417:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00020000:00000080:1.0:1787826814.246418:0:77862:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.246421:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00020000:00000080:1.0:1787826814.246422:0:77862:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.246424:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00020000:00000080:1.0:1787826814.246426:0:77862:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.246428:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00020000:00000080:1.0:1787826814.246430:0:77862:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.246432:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00020000:00000080:1.0:1787826814.246433:0:77862:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.246435:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00020000:00000080:1.0:1787826814.246437:0:77862:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.246439:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00020000:00000080:1.0:1787826814.246440:0:77862:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.246442:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00020000:00000080:1.0:1787826814.246445:0:77862:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.246447:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00020000:00000080:1.0:1787826814.246449:0:77862:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.246451:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00020000:00000080:1.0:1787826814.246453:0:77862:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.246455:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00020000:00000080:1.0:1787826814.246457:0:77862:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.246459:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00020000:00000080:1.0:1787826814.246460:0:77862:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.246462:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00020000:00000080:1.0:1787826814.246464:0:77862:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200080:1.0:1787826814.246466:0:77862:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff4a02c470 00020000:00000080:1.0:1787826814.246467:0:77862:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681800 uarg=0000000000000000 00000080:00200008:1.0:1787826814.246476:0:77862:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 100000 00800000:00000080:3.0:1787826815.544915:0:77914:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605620000 uarg=0000000000000000 00000080:00200080:3.0:1787826815.544917:0:77914:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff26037220 00800000:00000080:3.0:1787826815.544919:0:77914:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605620000 uarg=0000000000000000 00000080:00200080:3.0:1787826815.544922:0:77914:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff26037220 00800000:00000080:3.0:1787826815.544923:0:77914:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605620000 uarg=0000000000000000 00000080:00200080:3.0:1787826815.544926:0:77914:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff26037220 00800000:00000080:3.0:1787826815.544927:0:77914:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605620000 uarg=0000000000000000 00000080:00200080:3.0:1787826815.544929:0:77914:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff26037220 00800000:00000080:3.0:1787826815.544931:0:77914:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605620000 uarg=0000000000000000 00000080:00200080:3.0:1787826815.544933:0:77914:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff26037220 00800000:00000080:3.0:1787826815.544934:0:77914:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605620000 uarg=0000000000000000 00000080:00200080:3.0:1787826815.544936:0:77914:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff26037220 00800000:00000080:3.0:1787826815.544938:0:77914:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605620000 uarg=0000000000000000 00000080:00200080:3.0:1787826815.544940:0:77914:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff26037220 00020000:00000080:3.0:1787826815.544944:0:77914:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605620000 uarg=0000000000000000 00020000:00000004:3.0:1787826815.544948:0:77914:0:(obd_class.h:1091:obd_statfs()) lustre-OST0000-osc-ffff985602bc9800: age 2965, max_age 2965 00020000:00000004:3.0:1787826815.544950:0:77914:0:(obd_class.h:1131:obd_statfs()) lustre-OST0000-osc-ffff985602bc9800: use ffff985609528450 cache blocks 901357/958279 objects 261728/262144 00000080:00200080:3.0:1787826815.544969:0:77914:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff26037220 00020000:00000080:3.0:1787826815.544973:0:77914:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605620000 uarg=0000000000000000 00020000:00000004:3.0:1787826815.544975:0:77914:0:(obd_class.h:1091:obd_statfs()) lustre-OST0001-osc-ffff985602bc9800: age 2965, max_age 2965 00020000:00000004:3.0:1787826815.544976:0:77914:0:(obd_class.h:1131:obd_statfs()) lustre-OST0001-osc-ffff985602bc9800: use ffff98560952d1c0 cache blocks 901363/958279 objects 261735/262144 00000080:00200080:3.0:1787826815.544982:0:77914:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff26037220 00020000:00000080:3.0:1787826815.544985:0:77914:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605620000 uarg=0000000000000000 00000080:00200080:3.0:1787826815.544988:0:77914:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff26037220 00020000:00000080:3.0:1787826815.544990:0:77914:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605620000 uarg=0000000000000000 00000080:00200080:3.0:1787826815.544994:0:77914:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff26037220 00020000:00000080:3.0:1787826815.544999:0:77914:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605620000 uarg=0000000000000000 00000080:00200080:3.0:1787826815.545004:0:77914:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff26037220 00020000:00000080:3.0:1787826815.545008:0:77914:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605620000 uarg=0000000000000000 00000080:00200080:3.0:1787826815.545012:0:77914:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff26037220 00020000:00000080:3.0:1787826815.545016:0:77914:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605620000 uarg=0000000000000000 00000080:00200080:3.0:1787826815.545021:0:77914:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff26037220 00020000:00000080:3.0:1787826815.545026:0:77914:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605620000 uarg=0000000000000000 00000080:00200080:3.0:1787826815.545031:0:77914:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff26037220 00020000:00000080:3.0:1787826815.545036:0:77914:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605620000 uarg=0000000000000000 00000080:00200080:3.0:1787826815.545040:0:77914:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff26037220 00020000:00000080:3.0:1787826815.545045:0:77914:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605620000 uarg=0000000000000000 00000080:00200080:3.0:1787826815.545049:0:77914:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff26037220 00020000:00000080:3.0:1787826815.545052:0:77914:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605620000 uarg=0000000000000000 00000080:00200080:3.0:1787826815.545057:0:77914:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff26037220 00020000:00000080:3.0:1787826815.545061:0:77914:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605620000 uarg=0000000000000000 00000080:00200080:3.0:1787826815.545066:0:77914:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff26037220 00020000:00000080:3.0:1787826815.545070:0:77914:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605620000 uarg=0000000000000000 00000080:00200080:3.0:1787826815.545075:0:77914:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff26037220 00020000:00000080:3.0:1787826815.545079:0:77914:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605620000 uarg=0000000000000000 00000080:00200080:3.0:1787826815.545084:0:77914:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff26037220 00020000:00000080:3.0:1787826815.545088:0:77914:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605620000 uarg=0000000000000000 00000080:00200080:3.0:1787826815.545092:0:77914:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff26037220 00020000:00000080:3.0:1787826815.545097:0:77914:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605620000 uarg=0000000000000000 00000080:00200080:3.0:1787826815.545102:0:77914:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff26037220 00020000:00000080:3.0:1787826815.545107:0:77914:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605620000 uarg=0000000000000000 00000080:00200080:3.0:1787826815.545112:0:77914:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff26037220 00020000:00000080:3.0:1787826815.545117:0:77914:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605620000 uarg=0000000000000000 00000080:00200080:3.0:1787826815.545122:0:77914:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff26037220 00020000:00000080:3.0:1787826815.545126:0:77914:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605620000 uarg=0000000000000000 00000080:00200080:3.0:1787826815.545131:0:77914:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff26037220 00020000:00000080:3.0:1787826815.545134:0:77914:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605620000 uarg=0000000000000000 00000080:00200080:3.0:1787826815.545139:0:77914:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff26037220 00020000:00000080:3.0:1787826815.545142:0:77914:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605620000 uarg=0000000000000000 00000080:00200080:3.0:1787826815.545148:0:77914:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff26037220 00020000:00000080:3.0:1787826815.545152:0:77914:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605620000 uarg=0000000000000000 00000080:00200080:3.0:1787826815.545157:0:77914:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff26037220 00020000:00000080:3.0:1787826815.545160:0:77914:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605620000 uarg=0000000000000000 00000080:00200080:3.0:1787826815.545164:0:77914:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff26037220 00020000:00000080:3.0:1787826815.545169:0:77914:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605620000 uarg=0000000000000000 00000080:00200080:3.0:1787826815.545173:0:77914:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff26037220 00020000:00000080:3.0:1787826815.545177:0:77914:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605620000 uarg=0000000000000000 00000080:00200080:3.0:1787826815.545181:0:77914:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff26037220 00020000:00000080:3.0:1787826815.545185:0:77914:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605620000 uarg=0000000000000000 00000080:00200080:3.0:1787826815.545189:0:77914:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff26037220 00020000:00000080:3.0:1787826815.545192:0:77914:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605620000 uarg=0000000000000000 00000080:00200080:3.0:1787826815.545196:0:77914:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff26037220 00020000:00000080:3.0:1787826815.545200:0:77914:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605620000 uarg=0000000000000000 00000080:00200080:3.0:1787826815.545204:0:77914:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff26037220 00020000:00000080:3.0:1787826815.545206:0:77914:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605620000 uarg=0000000000000000 00000080:00200080:3.0:1787826815.545208:0:77914:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff26037220 00020000:00000080:3.0:1787826815.545211:0:77914:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605620000 uarg=0000000000000000 00000080:00200080:3.0:1787826815.545213:0:77914:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff26037220 00020000:00000080:3.0:1787826815.545215:0:77914:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605620000 uarg=0000000000000000 00000080:00200080:3.0:1787826815.545217:0:77914:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff26037220 00020000:00000080:3.0:1787826815.545219:0:77914:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605620000 uarg=0000000000000000 00000080:00200080:3.0:1787826815.545221:0:77914:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff26037220 00020000:00000080:3.0:1787826815.545223:0:77914:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605620000 uarg=0000000000000000 00000080:00200080:3.0:1787826815.545225:0:77914:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff26037220 00020000:00000080:3.0:1787826815.545226:0:77914:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605620000 uarg=0000000000000000 00000080:00200080:3.0:1787826815.545228:0:77914:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff26037220 00020000:00000080:3.0:1787826815.545230:0:77914:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605620000 uarg=0000000000000000 00000080:00200080:3.0:1787826815.545232:0:77914:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff26037220 00020000:00000080:3.0:1787826815.545233:0:77914:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605620000 uarg=0000000000000000 00000080:00200080:3.0:1787826815.545235:0:77914:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff26037220 00020000:00000080:3.0:1787826815.545237:0:77914:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605620000 uarg=0000000000000000 00000080:00200080:3.0:1787826815.545239:0:77914:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff26037220 00020000:00000080:3.0:1787826815.545241:0:77914:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605620000 uarg=0000000000000000 00000080:00200080:3.0:1787826815.545243:0:77914:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff26037220 00020000:00000080:3.0:1787826815.545244:0:77914:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605620000 uarg=0000000000000000 00000080:00200080:3.0:1787826815.545246:0:77914:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff26037220 00020000:00000080:3.0:1787826815.545248:0:77914:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605620000 uarg=0000000000000000 00000080:00200080:3.0:1787826815.545250:0:77914:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff26037220 00020000:00000080:3.0:1787826815.545252:0:77914:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605620000 uarg=0000000000000000 00000080:00200080:3.0:1787826815.545256:0:77914:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff26037220 00020000:00000080:3.0:1787826815.545259:0:77914:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605620000 uarg=0000000000000000 00000080:00200080:3.0:1787826815.545263:0:77914:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff26037220 00020000:00000080:3.0:1787826815.545266:0:77914:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605620000 uarg=0000000000000000 00000080:00200080:3.0:1787826815.545285:0:77914:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff26037220 00020000:00000080:3.0:1787826815.545290:0:77914:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605620000 uarg=0000000000000000 00000080:00200080:3.0:1787826815.545294:0:77914:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff26037220 00020000:00000080:3.0:1787826815.545298:0:77914:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605620000 uarg=0000000000000000 00000080:00200080:3.0:1787826815.545302:0:77914:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff26037220 00020000:00000080:3.0:1787826815.545306:0:77914:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605620000 uarg=0000000000000000 00000080:00200080:3.0:1787826815.545310:0:77914:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff26037220 00020000:00000080:3.0:1787826815.545314:0:77914:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605620000 uarg=0000000000000000 00000080:00200080:3.0:1787826815.545317:0:77914:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff26037220 00020000:00000080:3.0:1787826815.545319:0:77914:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605620000 uarg=0000000000000000 00000080:00200080:3.0:1787826815.545321:0:77914:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff26037220 00020000:00000080:3.0:1787826815.545323:0:77914:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605620000 uarg=0000000000000000 00000080:00200080:3.0:1787826815.545325:0:77914:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff26037220 00020000:00000080:3.0:1787826815.545327:0:77914:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605620000 uarg=0000000000000000 00000080:00200080:3.0:1787826815.545329:0:77914:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff26037220 00020000:00000080:3.0:1787826815.545330:0:77914:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605620000 uarg=0000000000000000 00000080:00200080:3.0:1787826815.545332:0:77914:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff26037220 00020000:00000080:3.0:1787826815.545334:0:77914:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605620000 uarg=0000000000000000 00000080:00200080:3.0:1787826815.545336:0:77914:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff26037220 00020000:00000080:3.0:1787826815.545338:0:77914:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605620000 uarg=0000000000000000 00000080:00200080:3.0:1787826815.545340:0:77914:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff26037220 00020000:00000080:3.0:1787826815.545341:0:77914:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605620000 uarg=0000000000000000 00000080:00200080:3.0:1787826815.545343:0:77914:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff26037220 00020000:00000080:3.0:1787826815.545346:0:77914:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605620000 uarg=0000000000000000 00000080:00200080:3.0:1787826815.545348:0:77914:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff26037220 00020000:00000080:3.0:1787826815.545350:0:77914:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605620000 uarg=0000000000000000 00000080:00200080:3.0:1787826815.545352:0:77914:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff26037220 00020000:00000080:3.0:1787826815.545354:0:77914:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605620000 uarg=0000000000000000 00000080:00200080:3.0:1787826815.545356:0:77914:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff26037220 00020000:00000080:3.0:1787826815.545358:0:77914:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605620000 uarg=0000000000000000 00000080:00200080:3.0:1787826815.545362:0:77914:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff26037220 00020000:00000080:3.0:1787826815.545365:0:77914:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605620000 uarg=0000000000000000 00000080:00200080:3.0:1787826815.545368:0:77914:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff26037220 00020000:00000080:3.0:1787826815.545371:0:77914:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605620000 uarg=0000000000000000 00000080:00200080:3.0:1787826815.545374:0:77914:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff26037220 00020000:00000080:3.0:1787826815.545377:0:77914:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605620000 uarg=0000000000000000 00000080:00200080:3.0:1787826815.545381:0:77914:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff26037220 00020000:00000080:3.0:1787826815.545383:0:77914:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605620000 uarg=0000000000000000 00000080:00200080:3.0:1787826815.545387:0:77914:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff26037220 00020000:00000080:3.0:1787826815.545390:0:77914:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605620000 uarg=0000000000000000 00000080:00200080:3.0:1787826815.545395:0:77914:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff26037220 00020000:00000080:3.0:1787826815.545398:0:77914:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605620000 uarg=0000000000000000 00000080:00200080:3.0:1787826815.545402:0:77914:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff26037220 00020000:00000080:3.0:1787826815.545405:0:77914:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605620000 uarg=0000000000000000 00000080:00200008:3.0:1787826815.545413:0:77914:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 100000 00000100:00100000:3.0:1787826816.244907:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:3c946890-15ca-497a-b838-fe1cba61f4dd:2386:1874669225533440:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787826816.244995:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669225533696:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787826816.246477:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee2d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669225533952:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787826816.246493:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669225533568:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787826816.246503:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669225533824:192.168.203.114@tcp:400:kworker.0 00010000:00010000:1.0:1787826816.820810:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826818.868938:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826820.916940:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826821.369383:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2385:1874669225534080:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787826821.712514:0:78051:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee2580 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78051:1874669225534720:192.168.203.114@tcp:700:createmany.0 00000002:00010000:3.0:1787826821.712549:0:78051:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f33a.replay-single--0,[0x200015bc1:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: open|creat flags 0100103 00010000:00010000:3.0:1787826821.712567:0:78051:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ba00/0xb3f83dad1b7e00b9 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x5aa6d504d91a0d79 expref: -99 pid: 77862 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 1 00010000:00010000:3.0:1787826821.712626:0:78051:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ba00/0xb3f83dad1b7e00b9 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a0d79 expref: -99 pid: 77862 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 1 00010000:00010000:3.0:1787826821.712635:0:78051:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826821.712638:0:78051:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ba00/0xb3f83dad1b7e00b9 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a0d79 expref: -99 pid: 77862 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 1 00010000:00010000:3.0:1787826821.712686:0:78051:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06400/0xb3f83dad1b7e00c0 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 78051 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.712779:0:78051:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06400/0xb3f83dad1b7e00c0 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78051 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:3.0:1787826821.712793:0:78051:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:3.0:1787826821.712804:0:78051:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610e661c0 x1874669225534976/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:3.0:1787826821.712813:0:78051:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06400/0xb3f83dad1b7e00c0 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78051 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826821.712831:0:78051:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e661c0 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78051:1874669225534976:192.168.203.114@tcp:101:createmany.0 00000100:00100000:3.0:1787826821.713070:0:78051:0:(client.c:2715:ptlrpc_set_wait()) set ffff985610f13840 going to sleep for 11 seconds 00000100:00100000:3.0:1787826821.715415:0:78051:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e661c0 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78051:1874669225534976:192.168.203.114@tcp:101:createmany.0 00010000:00010000:3.0:1787826821.715443:0:78051:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06400/0xb3f83dad1b7e00c0 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0db8 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.715473:0:78051:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06400/0xb3f83dad1b7e00c0 lrc: 4/0,1 mode: --/PR res: [0x200015bc1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0db8 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.715481:0:78051:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06400/0xb3f83dad1b7e00c0 lrc: 4/0,1 mode: --/PR res: [0x200015bc1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0db8 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.715494:0:78051:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06400/0xb3f83dad1b7e00c0 lrc: 4/0,1 mode: PR/PR res: [0x200015bc1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0db8 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.715505:0:78051:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06400/0xb3f83dad1b7e00c0 lrc: 4/0,1 mode: PR/PR res: [0x200015bc1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0db8 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.715518:0:78051:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06400/0xb3f83dad1b7e00c0 lrc: 5/1,1 mode: PR/PR res: [0x200015bc1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0db8 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.715525:0:78051:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06400/0xb3f83dad1b7e00c0 lrc: 5/1,1 mode: PR/PR res: [0x200015bc1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0db8 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.715534:0:78051:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06400/0xb3f83dad1b7e00c0 lrc: 4/1,0 mode: PR/PR res: [0x200015bc1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0db8 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826821.715545:0:78051:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff985610e661c0 x1874669225534976/t193273528321(193273528321) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/664 e 0 to 0 dl 1787826837 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:3.0:1787826821.715558:0:78051:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff985610e661c0 x1874669225534976/t193273528321(193273528321) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/664 e 0 to 0 dl 1787826837 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826821.715571:0:78051:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06400/0xb3f83dad1b7e00c0 lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0db8 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:3.0:1787826821.715585:0:78051:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06400/0xb3f83dad1b7e00c0 lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0db8 expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.715597:0:78051:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8590023617/1 (0 0) 00000080:00200000:3.0:1787826821.715603:0:78051:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:3.0:1787826821.715618:0:78051:0:(namei.c:1628:ll_create_it()) VFS Op:name=f33a.replay-single--0, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=open|creat 00000080:00200000:3.0:1787826821.715665:0:78051:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200015bc1:0x1:0x0](ffff98561158ba88) 00000080:00200000:3.0:1787826821.715774:0:78051:0:(file.c:6854:ll_layout_conf()) [0x200015bc1:0x1:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:3.0:1787826821.715779:0:78051:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x200015bc1:0x1:0x0](ffff98561158ba88) 00000080:00010000:3.0:1787826821.715783:0:78051:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200015bc1:0x1:0x0](ffff98561158ba88) for lock 0xb3f83dad1b7e00c0 00000080:00010000:3.0:1787826821.715793:0:78051:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200015bc1:0x1:0x0](ffff98561158ba88) 00000080:00200008:3.0:1787826821.715808:0:78051:0:(file.c:1090:ll_file_open()) START file f33a.replay-single--0:[0x200015bc1:0x1:0x0](ffff98561158ba88), flags 100102 00000002:00100000:3.0:1787826821.715830:0:78051:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff985610e661c0 x1874669225534976/t193273528321(193273528321) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826837 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:3.0:1787826821.715867:0:78051:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e00c0 from it ffff9856095bb720 00010000:00010000:3.0:1787826821.715871:0:78051:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06400/0xb3f83dad1b7e00c0 lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a0db8 expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.715881:0:78051:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06400/0xb3f83dad1b7e00c0 lrc: 2/0,0 mode: PR/PR res: [0x200015bc1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a0db8 expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826821.715902:0:78051:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200015bc1:0x1:0x0](ffff98561158ba88), inode mode 81a4 mask 40 00000080:00200008:3.0:1787826821.715935:0:78051:0:(file.c:479:ll_file_release()) START file f33a.replay-single--0:[0x200015bc1:0x1:0x0](ffff98561158ba88), flags 100002 00000002:00100000:3.0:1787826821.715979:0:78051:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff985610e661c0 x1874669225534976/t193273528321(193273528321) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826837 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:3.0:1787826821.715989:0:78051:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:3.0:1787826821.715993:0:78051:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610e64000 x1874669225535232/t0(0) o35->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:3.0:1787826821.716014:0:78051:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e64000 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78051:1874669225535232:192.168.203.114@tcp:35:createmany.0 00000100:00100000:3.0:1787826821.716083:0:78051:0:(client.c:2715:ptlrpc_set_wait()) set ffff985610f13240 going to sleep for 11 seconds 00000100:00100000:3.0:1787826821.717942:0:78051:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e64000 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78051:1874669225535232:192.168.203.114@tcp:35:createmany.0 00000002:00100000:3.0:1787826821.717977:0:78051:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff985610e661c0 x1874669225534976/t193273528321(193273528321) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826837 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:3.0:1787826821.718044:0:78051:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:3.0:1787826821.718060:0:78051:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826821.718108:0:78051:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07a00/0xb3f83dad1b7e00c7 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 78051 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.718119:0:78051:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07a00/0xb3f83dad1b7e00c7 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78051 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.718138:0:78051:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07a00/0xb3f83dad1b7e00c7 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78051 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826821.718152:0:78051:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e64b40 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78051:1874669225535360:192.168.203.114@tcp:101:createmany.0 00000100:00100000:3.0:1787826821.718204:0:78051:0:(client.c:2715:ptlrpc_set_wait()) set ffff985610f13240 going to sleep for 11 seconds 00000100:00100000:3.0:1787826821.719430:0:78051:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e64b40 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78051:1874669225535360:192.168.203.114@tcp:101:createmany.0 00010000:00010000:3.0:1787826821.719448:0:78051:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07a00/0xb3f83dad1b7e00c7 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0dcd expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.719460:0:78051:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07a00/0xb3f83dad1b7e00c7 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0dcd expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.719473:0:78051:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07a00/0xb3f83dad1b7e00c7 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0dcd expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.719484:0:78051:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07a00/0xb3f83dad1b7e00c7 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0dcd expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.719496:0:78051:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07a00/0xb3f83dad1b7e00c7 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0dcd expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.719505:0:78051:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07a00/0xb3f83dad1b7e00c7 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0dcd expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.719519:0:78051:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07a00/0xb3f83dad1b7e00c7 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0dcd expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826821.719531:0:78051:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff985610e64b40 x1874669225535360/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826837 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826821.719545:0:78051:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07a00/0xb3f83dad1b7e00c7 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0dcd expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.719556:0:78051:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:3.0:1787826821.719562:0:78051:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:3.0:1787826821.719587:0:78051:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:3.0:1787826821.719594:0:78051:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e00c7 00000080:00010000:3.0:1787826821.719599:0:78051:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:3.0:1787826821.719603:0:78051:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e00c7 from it ffffab3ac1cafbf8 00010000:00010000:3.0:1787826821.719606:0:78051:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07a00/0xb3f83dad1b7e00c7 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0dcd expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.719623:0:78051:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07a00/0xb3f83dad1b7e00c7 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0dcd expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826821.719646:0:78051:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:3.0:1787826821.719683:0:78051:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:3.0:1787826821.719747:0:78051:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826821.719757:0:78051:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07a00/0xb3f83dad1b7e00c7 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0dcd expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826821.719774:0:78051:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07a00/0xb3f83dad1b7e00c7 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0dcd expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826821.719782:0:78051:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826821.719786:0:78051:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:3.0:1787826821.719790:0:78051:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e00c7 00000080:00010000:3.0:1787826821.719794:0:78051:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e00c7 from it ffffab3ac1cafb38 00010000:00010000:3.0:1787826821.719797:0:78051:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07a00/0xb3f83dad1b7e00c7 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0dcd expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826821.719805:0:78051:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07a00/0xb3f83dad1b7e00c7 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0dcd expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:3.0:1787826821.719813:0:78051:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 3 00000080:00200000:3.0:1787826821.719819:0:78051:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f33a.replay-single--1, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), file ffff985611f24c00, open_flags 8042, mode 81a4 opened 0 00000080:00200000:3.0:1787826821.719829:0:78051:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f33a.replay-single--1, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=open|creat 00000002:00010000:3.0:1787826821.719859:0:78051:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f33a.replay-single--1,[0x200015bc1:0x2:0x0]) in obj [0x200000007:0x1:0x0], intent: open|creat flags 0100103 00010000:00010000:3.0:1787826821.719873:0:78051:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07a00/0xb3f83dad1b7e00c7 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x5aa6d504d91a0dcd expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826821.719920:0:78051:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07a00/0xb3f83dad1b7e00c7 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a0dcd expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826821.719927:0:78051:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826821.719930:0:78051:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07a00/0xb3f83dad1b7e00c7 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a0dcd expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826821.719972:0:78051:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07800/0xb3f83dad1b7e00ce lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 78051 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.719980:0:78051:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07800/0xb3f83dad1b7e00ce lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78051 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:3.0:1787826821.719990:0:78051:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:3.0:1787826821.719997:0:78051:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610e64b40 x1874669225535616/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:3.0:1787826821.720007:0:78051:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07800/0xb3f83dad1b7e00ce lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78051 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826821.720024:0:78051:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e64b40 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78051:1874669225535616:192.168.203.114@tcp:101:createmany.0 00000100:00100000:3.0:1787826821.720149:0:78051:0:(client.c:2715:ptlrpc_set_wait()) set ffff985610f13240 going to sleep for 11 seconds 00000100:00100000:3.0:1787826821.722630:0:78051:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e64b40 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78051:1874669225535616:192.168.203.114@tcp:101:createmany.0 00010000:00010000:3.0:1787826821.722655:0:78051:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07800/0xb3f83dad1b7e00ce lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0de9 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.722681:0:78051:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07800/0xb3f83dad1b7e00ce lrc: 4/0,1 mode: --/PR res: [0x200015bc1:0x2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0de9 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.722688:0:78051:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07800/0xb3f83dad1b7e00ce lrc: 4/0,1 mode: --/PR res: [0x200015bc1:0x2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0de9 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.722737:0:78051:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07800/0xb3f83dad1b7e00ce lrc: 4/0,1 mode: PR/PR res: [0x200015bc1:0x2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0de9 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.722756:0:78051:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07800/0xb3f83dad1b7e00ce lrc: 4/0,1 mode: PR/PR res: [0x200015bc1:0x2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0de9 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.722771:0:78051:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07800/0xb3f83dad1b7e00ce lrc: 5/1,1 mode: PR/PR res: [0x200015bc1:0x2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0de9 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.722778:0:78051:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07800/0xb3f83dad1b7e00ce lrc: 5/1,1 mode: PR/PR res: [0x200015bc1:0x2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0de9 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.722783:0:78051:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07800/0xb3f83dad1b7e00ce lrc: 4/1,0 mode: PR/PR res: [0x200015bc1:0x2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0de9 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826821.722791:0:78051:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff985610e64b40 x1874669225535616/t193273528323(193273528323) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/664 e 0 to 0 dl 1787826837 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:3.0:1787826821.722800:0:78051:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff985610e64b40 x1874669225535616/t193273528323(193273528323) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/664 e 0 to 0 dl 1787826837 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826821.722811:0:78051:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07800/0xb3f83dad1b7e00ce lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0de9 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:3.0:1787826821.722821:0:78051:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07800/0xb3f83dad1b7e00ce lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0de9 expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.722829:0:78051:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8590023617/2 (0 0) 00000080:00200000:3.0:1787826821.722835:0:78051:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:3.0:1787826821.722847:0:78051:0:(namei.c:1628:ll_create_it()) VFS Op:name=f33a.replay-single--1, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=open|creat 00000080:00200000:3.0:1787826821.722885:0:78051:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200015bc1:0x2:0x0](ffff98561158f448) 00000080:00200000:3.0:1787826821.722980:0:78051:0:(file.c:6854:ll_layout_conf()) [0x200015bc1:0x2:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:3.0:1787826821.722986:0:78051:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x200015bc1:0x2:0x0](ffff98561158f448) 00000080:00010000:3.0:1787826821.722990:0:78051:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200015bc1:0x2:0x0](ffff98561158f448) for lock 0xb3f83dad1b7e00ce 00000080:00010000:3.0:1787826821.723001:0:78051:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200015bc1:0x2:0x0](ffff98561158f448) 00000080:00200008:3.0:1787826821.723022:0:78051:0:(file.c:1090:ll_file_open()) START file f33a.replay-single--1:[0x200015bc1:0x2:0x0](ffff98561158f448), flags 100102 00000002:00100000:3.0:1787826821.723046:0:78051:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff985610e64b40 x1874669225535616/t193273528323(193273528323) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826837 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:3.0:1787826821.723087:0:78051:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e00ce from it ffff985610c4a420 00010000:00010000:3.0:1787826821.723093:0:78051:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07800/0xb3f83dad1b7e00ce lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a0de9 expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.723102:0:78051:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07800/0xb3f83dad1b7e00ce lrc: 2/0,0 mode: PR/PR res: [0x200015bc1:0x2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a0de9 expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826821.723135:0:78051:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200015bc1:0x2:0x0](ffff98561158f448), inode mode 81a4 mask 40 00000080:00200008:3.0:1787826821.723180:0:78051:0:(file.c:479:ll_file_release()) START file f33a.replay-single--1:[0x200015bc1:0x2:0x0](ffff98561158f448), flags 100002 00000002:00100000:3.0:1787826821.723239:0:78051:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff985610e64b40 x1874669225535616/t193273528323(193273528323) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826837 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:3.0:1787826821.723251:0:78051:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:3.0:1787826821.723256:0:78051:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610e66580 x1874669225535872/t0(0) o35->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:3.0:1787826821.723278:0:78051:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e66580 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78051:1874669225535872:192.168.203.114@tcp:35:createmany.0 00000100:00100000:3.0:1787826821.723365:0:78051:0:(client.c:2715:ptlrpc_set_wait()) set ffff985610f13780 going to sleep for 11 seconds 00000100:00100000:3.0:1787826821.724530:0:78051:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e66580 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78051:1874669225535872:192.168.203.114@tcp:35:createmany.0 00000002:00100000:3.0:1787826821.724557:0:78051:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff985610e64b40 x1874669225535616/t193273528323(193273528323) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826837 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:3.0:1787826821.724627:0:78051:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:3.0:1787826821.724645:0:78051:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826821.724744:0:78051:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e00d5 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 78051 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.724756:0:78051:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e00d5 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78051 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.724769:0:78051:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e00d5 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78051 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826821.724788:0:78051:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e67840 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78051:1874669225536000:192.168.203.114@tcp:101:createmany.0 00000100:00100000:3.0:1787826821.724841:0:78051:0:(client.c:2715:ptlrpc_set_wait()) set ffff985610f13780 going to sleep for 11 seconds 00000100:00100000:3.0:1787826821.726184:0:78051:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e67840 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78051:1874669225536000:192.168.203.114@tcp:101:createmany.0 00010000:00010000:3.0:1787826821.726208:0:78051:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e00d5 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0dfe expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.726235:0:78051:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e00d5 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0dfe expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.726249:0:78051:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e00d5 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0dfe expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.726262:0:78051:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e00d5 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0dfe expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.726279:0:78051:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e00d5 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0dfe expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.726289:0:78051:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e00d5 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0dfe expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.726297:0:78051:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e00d5 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0dfe expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826821.726310:0:78051:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff985610e67840 x1874669225536000/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826837 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826821.726322:0:78051:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e00d5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0dfe expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.726331:0:78051:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:3.0:1787826821.726336:0:78051:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:3.0:1787826821.726362:0:78051:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:3.0:1787826821.726369:0:78051:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e00d5 00000080:00010000:3.0:1787826821.726375:0:78051:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:3.0:1787826821.726378:0:78051:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e00d5 from it ffffab3ac1cafbf8 00010000:00010000:3.0:1787826821.726381:0:78051:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e00d5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0dfe expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.726390:0:78051:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e00d5 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0dfe expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826821.726422:0:78051:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:3.0:1787826821.726447:0:78051:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:3.0:1787826821.726461:0:78051:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826821.726470:0:78051:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e00d5 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0dfe expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826821.726482:0:78051:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e00d5 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0dfe expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826821.726489:0:78051:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826821.726492:0:78051:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:3.0:1787826821.726494:0:78051:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e00d5 00000080:00010000:3.0:1787826821.726498:0:78051:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e00d5 from it ffffab3ac1cafb38 00010000:00010000:3.0:1787826821.726500:0:78051:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e00d5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0dfe expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826821.726504:0:78051:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e00d5 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0dfe expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:3.0:1787826821.726510:0:78051:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 3 00000080:00200000:3.0:1787826821.726514:0:78051:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f33a.replay-single--2, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), file ffff985611f24480, open_flags 8042, mode 81a4 opened 0 00000080:00200000:3.0:1787826821.726522:0:78051:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f33a.replay-single--2, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=open|creat 00000002:00010000:3.0:1787826821.726545:0:78051:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f33a.replay-single--2,[0x200015bc1:0x3:0x0]) in obj [0x200000007:0x1:0x0], intent: open|creat flags 0100103 00010000:00010000:3.0:1787826821.726559:0:78051:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e00d5 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x5aa6d504d91a0dfe expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826821.726612:0:78051:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e00d5 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a0dfe expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826821.726619:0:78051:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826821.726622:0:78051:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e00d5 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a0dfe expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826821.726663:0:78051:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06c00/0xb3f83dad1b7e00dc lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 78051 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.726670:0:78051:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06c00/0xb3f83dad1b7e00dc lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78051 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:3.0:1787826821.726681:0:78051:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:3.0:1787826821.726687:0:78051:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610e67840 x1874669225536256/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:3.0:1787826821.726733:0:78051:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06c00/0xb3f83dad1b7e00dc lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78051 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826821.726754:0:78051:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e67840 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78051:1874669225536256:192.168.203.114@tcp:101:createmany.0 00000100:00100000:3.0:1787826821.726900:0:78051:0:(client.c:2715:ptlrpc_set_wait()) set ffff985610f13780 going to sleep for 11 seconds 00000100:00100000:3.0:1787826821.728767:0:78051:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e67840 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78051:1874669225536256:192.168.203.114@tcp:101:createmany.0 00010000:00010000:3.0:1787826821.728798:0:78051:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06c00/0xb3f83dad1b7e00dc lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0e1a expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.728828:0:78051:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06c00/0xb3f83dad1b7e00dc lrc: 4/0,1 mode: --/PR res: [0x200015bc1:0x3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0e1a expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.728843:0:78051:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06c00/0xb3f83dad1b7e00dc lrc: 4/0,1 mode: --/PR res: [0x200015bc1:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0e1a expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.728854:0:78051:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06c00/0xb3f83dad1b7e00dc lrc: 4/0,1 mode: PR/PR res: [0x200015bc1:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0e1a expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.728865:0:78051:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06c00/0xb3f83dad1b7e00dc lrc: 4/0,1 mode: PR/PR res: [0x200015bc1:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0e1a expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.728878:0:78051:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06c00/0xb3f83dad1b7e00dc lrc: 5/1,1 mode: PR/PR res: [0x200015bc1:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0e1a expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.728887:0:78051:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06c00/0xb3f83dad1b7e00dc lrc: 5/1,1 mode: PR/PR res: [0x200015bc1:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0e1a expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.728894:0:78051:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06c00/0xb3f83dad1b7e00dc lrc: 4/1,0 mode: PR/PR res: [0x200015bc1:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0e1a expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826821.728910:0:78051:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff985610e67840 x1874669225536256/t193273528325(193273528325) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/664 e 0 to 0 dl 1787826837 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:3.0:1787826821.728926:0:78051:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff985610e67840 x1874669225536256/t193273528325(193273528325) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/664 e 0 to 0 dl 1787826837 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826821.728940:0:78051:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06c00/0xb3f83dad1b7e00dc lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0e1a expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:3.0:1787826821.728963:0:78051:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06c00/0xb3f83dad1b7e00dc lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0e1a expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.728979:0:78051:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8590023617/3 (0 0) 00000080:00200000:3.0:1787826821.728985:0:78051:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:3.0:1787826821.729000:0:78051:0:(namei.c:1628:ll_create_it()) VFS Op:name=f33a.replay-single--2, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=open|creat 00000080:00200000:3.0:1787826821.729029:0:78051:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200015bc1:0x3:0x0](ffff985611588908) 00000080:00200000:3.0:1787826821.729125:0:78051:0:(file.c:6854:ll_layout_conf()) [0x200015bc1:0x3:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:3.0:1787826821.729130:0:78051:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x200015bc1:0x3:0x0](ffff985611588908) 00000080:00010000:3.0:1787826821.729134:0:78051:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200015bc1:0x3:0x0](ffff985611588908) for lock 0xb3f83dad1b7e00dc 00000080:00010000:3.0:1787826821.729144:0:78051:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200015bc1:0x3:0x0](ffff985611588908) 00000080:00200008:3.0:1787826821.729160:0:78051:0:(file.c:1090:ll_file_open()) START file f33a.replay-single--2:[0x200015bc1:0x3:0x0](ffff985611588908), flags 100102 00000002:00100000:3.0:1787826821.729183:0:78051:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff985610e67840 x1874669225536256/t193273528325(193273528325) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826837 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:3.0:1787826821.729242:0:78051:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e00dc from it ffff985610c4a420 00010000:00010000:3.0:1787826821.729247:0:78051:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06c00/0xb3f83dad1b7e00dc lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a0e1a expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.729256:0:78051:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06c00/0xb3f83dad1b7e00dc lrc: 2/0,0 mode: PR/PR res: [0x200015bc1:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a0e1a expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826821.729279:0:78051:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200015bc1:0x3:0x0](ffff985611588908), inode mode 81a4 mask 40 00000080:00200008:3.0:1787826821.729314:0:78051:0:(file.c:479:ll_file_release()) START file f33a.replay-single--2:[0x200015bc1:0x3:0x0](ffff985611588908), flags 100002 00000002:00100000:3.0:1787826821.729361:0:78051:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff985610e67840 x1874669225536256/t193273528325(193273528325) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826837 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:3.0:1787826821.729372:0:78051:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:3.0:1787826821.729378:0:78051:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610e670c0 x1874669225536512/t0(0) o35->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:3.0:1787826821.729409:0:78051:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e670c0 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78051:1874669225536512:192.168.203.114@tcp:35:createmany.0 00000100:00100000:3.0:1787826821.729482:0:78051:0:(client.c:2715:ptlrpc_set_wait()) set ffff985610f13900 going to sleep for 11 seconds 00000100:00100000:3.0:1787826821.730582:0:78051:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e670c0 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78051:1874669225536512:192.168.203.114@tcp:35:createmany.0 00000002:00100000:3.0:1787826821.730610:0:78051:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff985610e67840 x1874669225536256/t193273528325(193273528325) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826837 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:3.0:1787826821.730726:0:78051:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:3.0:1787826821.730749:0:78051:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826821.730808:0:78051:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07a00/0xb3f83dad1b7e00e3 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 78051 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.730819:0:78051:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07a00/0xb3f83dad1b7e00e3 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78051 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.730832:0:78051:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07a00/0xb3f83dad1b7e00e3 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78051 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826821.730852:0:78051:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e652c0 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78051:1874669225536640:192.168.203.114@tcp:101:createmany.0 00000100:00100000:3.0:1787826821.730937:0:78051:0:(client.c:2715:ptlrpc_set_wait()) set ffff985610f13900 going to sleep for 11 seconds 00000100:00100000:3.0:1787826821.732285:0:78051:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e652c0 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78051:1874669225536640:192.168.203.114@tcp:101:createmany.0 00010000:00010000:3.0:1787826821.732306:0:78051:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07a00/0xb3f83dad1b7e00e3 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0e2f expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.732318:0:78051:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07a00/0xb3f83dad1b7e00e3 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0e2f expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.732335:0:78051:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07a00/0xb3f83dad1b7e00e3 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0e2f expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.732345:0:78051:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07a00/0xb3f83dad1b7e00e3 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0e2f expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.732357:0:78051:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07a00/0xb3f83dad1b7e00e3 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0e2f expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.732365:0:78051:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07a00/0xb3f83dad1b7e00e3 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0e2f expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.732372:0:78051:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07a00/0xb3f83dad1b7e00e3 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0e2f expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826821.732383:0:78051:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff985610e652c0 x1874669225536640/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826837 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826821.732397:0:78051:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07a00/0xb3f83dad1b7e00e3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0e2f expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.732409:0:78051:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:3.0:1787826821.732415:0:78051:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:3.0:1787826821.732441:0:78051:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:3.0:1787826821.732448:0:78051:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e00e3 00000080:00010000:3.0:1787826821.732453:0:78051:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:3.0:1787826821.732456:0:78051:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e00e3 from it ffffab3ac1cafbf8 00010000:00010000:3.0:1787826821.732460:0:78051:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07a00/0xb3f83dad1b7e00e3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0e2f expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.732469:0:78051:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07a00/0xb3f83dad1b7e00e3 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0e2f expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826821.732490:0:78051:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:3.0:1787826821.732512:0:78051:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:3.0:1787826821.732527:0:78051:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826821.732536:0:78051:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07a00/0xb3f83dad1b7e00e3 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0e2f expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826821.732546:0:78051:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07a00/0xb3f83dad1b7e00e3 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0e2f expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826821.732553:0:78051:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826821.732557:0:78051:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:3.0:1787826821.732560:0:78051:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e00e3 00000080:00010000:3.0:1787826821.732564:0:78051:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e00e3 from it ffffab3ac1cafb38 00010000:00010000:3.0:1787826821.732567:0:78051:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07a00/0xb3f83dad1b7e00e3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0e2f expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826821.732574:0:78051:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07a00/0xb3f83dad1b7e00e3 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0e2f expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:3.0:1787826821.732580:0:78051:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 3 00000080:00200000:3.0:1787826821.732585:0:78051:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f33a.replay-single--3, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), file ffff985611f25e00, open_flags 8042, mode 81a4 opened 0 00000080:00200000:3.0:1787826821.732591:0:78051:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f33a.replay-single--3, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=open|creat 00000002:00010000:3.0:1787826821.732609:0:78051:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f33a.replay-single--3,[0x200015bc1:0x4:0x0]) in obj [0x200000007:0x1:0x0], intent: open|creat flags 0100103 00010000:00010000:3.0:1787826821.732619:0:78051:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07a00/0xb3f83dad1b7e00e3 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x5aa6d504d91a0e2f expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826821.732659:0:78051:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07a00/0xb3f83dad1b7e00e3 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a0e2f expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826821.732667:0:78051:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826821.732670:0:78051:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07a00/0xb3f83dad1b7e00e3 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a0e2f expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826821.732734:0:78051:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07200/0xb3f83dad1b7e00ea lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 78051 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.732742:0:78051:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07200/0xb3f83dad1b7e00ea lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78051 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:3.0:1787826821.732759:0:78051:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:3.0:1787826821.732767:0:78051:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610e652c0 x1874669225536896/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:3.0:1787826821.732775:0:78051:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07200/0xb3f83dad1b7e00ea lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78051 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826821.732791:0:78051:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e652c0 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78051:1874669225536896:192.168.203.114@tcp:101:createmany.0 00000100:00100000:3.0:1787826821.732899:0:78051:0:(client.c:2715:ptlrpc_set_wait()) set ffff985610f13900 going to sleep for 11 seconds 00000100:00100000:3.0:1787826821.734571:0:78051:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e652c0 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78051:1874669225536896:192.168.203.114@tcp:101:createmany.0 00010000:00010000:3.0:1787826821.734585:0:78051:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07200/0xb3f83dad1b7e00ea lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0e4b expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.734608:0:78051:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07200/0xb3f83dad1b7e00ea lrc: 4/0,1 mode: --/PR res: [0x200015bc1:0x4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0e4b expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.734616:0:78051:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07200/0xb3f83dad1b7e00ea lrc: 4/0,1 mode: --/PR res: [0x200015bc1:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0e4b expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.734623:0:78051:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07200/0xb3f83dad1b7e00ea lrc: 4/0,1 mode: PR/PR res: [0x200015bc1:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0e4b expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.734632:0:78051:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07200/0xb3f83dad1b7e00ea lrc: 4/0,1 mode: PR/PR res: [0x200015bc1:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0e4b expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.734641:0:78051:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07200/0xb3f83dad1b7e00ea lrc: 5/1,1 mode: PR/PR res: [0x200015bc1:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0e4b expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.734648:0:78051:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07200/0xb3f83dad1b7e00ea lrc: 5/1,1 mode: PR/PR res: [0x200015bc1:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0e4b expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.734655:0:78051:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07200/0xb3f83dad1b7e00ea lrc: 4/1,0 mode: PR/PR res: [0x200015bc1:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0e4b expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826821.734664:0:78051:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff985610e652c0 x1874669225536896/t193273528327(193273528327) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/664 e 0 to 0 dl 1787826837 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:3.0:1787826821.734675:0:78051:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff985610e652c0 x1874669225536896/t193273528327(193273528327) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/664 e 0 to 0 dl 1787826837 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826821.734686:0:78051:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07200/0xb3f83dad1b7e00ea lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0e4b expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:3.0:1787826821.734721:0:78051:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07200/0xb3f83dad1b7e00ea lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0e4b expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.734730:0:78051:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8590023617/4 (0 0) 00000080:00200000:3.0:1787826821.734734:0:78051:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:3.0:1787826821.734743:0:78051:0:(namei.c:1628:ll_create_it()) VFS Op:name=f33a.replay-single--3, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=open|creat 00000080:00200000:3.0:1787826821.734765:0:78051:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200015bc1:0x4:0x0](ffff985611589148) 00000080:00200000:3.0:1787826821.734830:0:78051:0:(file.c:6854:ll_layout_conf()) [0x200015bc1:0x4:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:3.0:1787826821.734835:0:78051:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x200015bc1:0x4:0x0](ffff985611589148) 00000080:00010000:3.0:1787826821.734838:0:78051:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200015bc1:0x4:0x0](ffff985611589148) for lock 0xb3f83dad1b7e00ea 00000080:00010000:3.0:1787826821.734844:0:78051:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200015bc1:0x4:0x0](ffff985611589148) 00000080:00200008:3.0:1787826821.734855:0:78051:0:(file.c:1090:ll_file_open()) START file f33a.replay-single--3:[0x200015bc1:0x4:0x0](ffff985611589148), flags 100102 00000002:00100000:3.0:1787826821.734872:0:78051:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff985610e652c0 x1874669225536896/t193273528327(193273528327) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826837 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:3.0:1787826821.734900:0:78051:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e00ea from it ffff985610c4a420 00010000:00010000:3.0:1787826821.734904:0:78051:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07200/0xb3f83dad1b7e00ea lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a0e4b expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.734912:0:78051:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07200/0xb3f83dad1b7e00ea lrc: 2/0,0 mode: PR/PR res: [0x200015bc1:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a0e4b expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826821.734925:0:78051:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200015bc1:0x4:0x0](ffff985611589148), inode mode 81a4 mask 40 00000080:00200008:3.0:1787826821.734961:0:78051:0:(file.c:479:ll_file_release()) START file f33a.replay-single--3:[0x200015bc1:0x4:0x0](ffff985611589148), flags 100002 00000002:00100000:3.0:1787826821.734990:0:78051:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff985610e652c0 x1874669225536896/t193273528327(193273528327) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826837 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:3.0:1787826821.735004:0:78051:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:3.0:1787826821.735008:0:78051:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610e66d00 x1874669225537152/t0(0) o35->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:3.0:1787826821.735022:0:78051:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e66d00 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78051:1874669225537152:192.168.203.114@tcp:35:createmany.0 00000100:00100000:3.0:1787826821.735066:0:78051:0:(client.c:2715:ptlrpc_set_wait()) set ffff985610f13000 going to sleep for 11 seconds 00000100:00100000:3.0:1787826821.736166:0:78051:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e66d00 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78051:1874669225537152:192.168.203.114@tcp:35:createmany.0 00000002:00100000:3.0:1787826821.736183:0:78051:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff985610e652c0 x1874669225536896/t193273528327(193273528327) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826837 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:3.0:1787826821.736244:0:78051:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:3.0:1787826821.736260:0:78051:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826821.736284:0:78051:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7e00f1 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 78051 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.736290:0:78051:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7e00f1 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78051 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.736295:0:78051:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7e00f1 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78051 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826821.736303:0:78051:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e64780 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78051:1874669225537280:192.168.203.114@tcp:101:createmany.0 00000100:00100000:3.0:1787826821.736337:0:78051:0:(client.c:2715:ptlrpc_set_wait()) set ffff985610f13000 going to sleep for 11 seconds 00000100:00100000:3.0:1787826821.737258:0:78051:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e64780 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78051:1874669225537280:192.168.203.114@tcp:101:createmany.0 00010000:00010000:3.0:1787826821.737270:0:78051:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7e00f1 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0e60 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.737285:0:78051:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7e00f1 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0e60 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.737294:0:78051:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7e00f1 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0e60 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.737302:0:78051:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7e00f1 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0e60 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.737312:0:78051:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7e00f1 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0e60 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.737320:0:78051:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7e00f1 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0e60 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.737327:0:78051:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7e00f1 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0e60 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826821.737333:0:78051:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff985610e64780 x1874669225537280/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826837 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826821.737340:0:78051:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7e00f1 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0e60 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.737353:0:78051:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:3.0:1787826821.737358:0:78051:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:3.0:1787826821.737374:0:78051:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:3.0:1787826821.737379:0:78051:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e00f1 00000080:00010000:3.0:1787826821.737383:0:78051:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:3.0:1787826821.737385:0:78051:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e00f1 from it ffffab3ac1cafbf8 00010000:00010000:3.0:1787826821.737389:0:78051:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7e00f1 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0e60 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.737397:0:78051:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7e00f1 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0e60 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826821.737415:0:78051:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:3.0:1787826821.737428:0:78051:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:3.0:1787826821.737436:0:78051:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826821.737444:0:78051:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7e00f1 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0e60 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826821.737452:0:78051:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7e00f1 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0e60 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826821.737458:0:78051:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826821.737462:0:78051:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:3.0:1787826821.737465:0:78051:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e00f1 00000080:00010000:3.0:1787826821.737469:0:78051:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e00f1 from it ffffab3ac1cafb38 00010000:00010000:3.0:1787826821.737472:0:78051:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7e00f1 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0e60 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826821.737479:0:78051:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7e00f1 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0e60 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:3.0:1787826821.737491:0:78051:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 3 00000080:00200000:3.0:1787826821.737494:0:78051:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f33a.replay-single--4, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), file ffff985611f24c00, open_flags 8042, mode 81a4 opened 0 00000080:00200000:3.0:1787826821.737499:0:78051:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f33a.replay-single--4, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=open|creat 00000002:00010000:3.0:1787826821.737511:0:78051:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f33a.replay-single--4,[0x200015bc1:0x5:0x0]) in obj [0x200000007:0x1:0x0], intent: open|creat flags 0100103 00010000:00010000:3.0:1787826821.737517:0:78051:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7e00f1 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x5aa6d504d91a0e60 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826821.737539:0:78051:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7e00f1 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a0e60 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826821.737544:0:78051:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826821.737545:0:78051:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7e00f1 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a0e60 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826821.737567:0:78051:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07400/0xb3f83dad1b7e00f8 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 78051 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.737573:0:78051:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07400/0xb3f83dad1b7e00f8 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78051 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:3.0:1787826821.737581:0:78051:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:3.0:1787826821.737587:0:78051:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610e64780 x1874669225537536/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:3.0:1787826821.737596:0:78051:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07400/0xb3f83dad1b7e00f8 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78051 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826821.737608:0:78051:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e64780 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78051:1874669225537536:192.168.203.114@tcp:101:createmany.0 00000100:00100000:3.0:1787826821.737727:0:78051:0:(client.c:2715:ptlrpc_set_wait()) set ffff985610f13000 going to sleep for 11 seconds 00000100:00100000:3.0:1787826821.739112:0:78051:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e64780 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78051:1874669225537536:192.168.203.114@tcp:101:createmany.0 00010000:00010000:3.0:1787826821.739123:0:78051:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07400/0xb3f83dad1b7e00f8 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0e7c expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.739136:0:78051:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07400/0xb3f83dad1b7e00f8 lrc: 4/0,1 mode: --/PR res: [0x200015bc1:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0e7c expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.739141:0:78051:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07400/0xb3f83dad1b7e00f8 lrc: 4/0,1 mode: --/PR res: [0x200015bc1:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0e7c expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.739147:0:78051:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07400/0xb3f83dad1b7e00f8 lrc: 4/0,1 mode: PR/PR res: [0x200015bc1:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0e7c expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.739152:0:78051:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07400/0xb3f83dad1b7e00f8 lrc: 4/0,1 mode: PR/PR res: [0x200015bc1:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0e7c expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.739158:0:78051:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07400/0xb3f83dad1b7e00f8 lrc: 5/1,1 mode: PR/PR res: [0x200015bc1:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0e7c expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.739162:0:78051:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07400/0xb3f83dad1b7e00f8 lrc: 5/1,1 mode: PR/PR res: [0x200015bc1:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0e7c expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.739166:0:78051:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07400/0xb3f83dad1b7e00f8 lrc: 4/1,0 mode: PR/PR res: [0x200015bc1:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0e7c expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826821.739172:0:78051:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff985610e64780 x1874669225537536/t193273528329(193273528329) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/664 e 0 to 0 dl 1787826837 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:3.0:1787826821.739180:0:78051:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff985610e64780 x1874669225537536/t193273528329(193273528329) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/664 e 0 to 0 dl 1787826837 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826821.739187:0:78051:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07400/0xb3f83dad1b7e00f8 lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0e7c expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:3.0:1787826821.739193:0:78051:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07400/0xb3f83dad1b7e00f8 lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0e7c expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.739198:0:78051:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8590023617/5 (0 0) 00000080:00200000:3.0:1787826821.739201:0:78051:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:3.0:1787826821.739208:0:78051:0:(namei.c:1628:ll_create_it()) VFS Op:name=f33a.replay-single--4, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=open|creat 00000080:00200000:3.0:1787826821.739241:0:78051:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200015bc1:0x5:0x0](ffff98561158db88) 00000080:00200000:3.0:1787826821.739299:0:78051:0:(file.c:6854:ll_layout_conf()) [0x200015bc1:0x5:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:3.0:1787826821.739304:0:78051:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x200015bc1:0x5:0x0](ffff98561158db88) 00000080:00010000:3.0:1787826821.739307:0:78051:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200015bc1:0x5:0x0](ffff98561158db88) for lock 0xb3f83dad1b7e00f8 00000080:00010000:3.0:1787826821.739315:0:78051:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200015bc1:0x5:0x0](ffff98561158db88) 00000080:00200008:3.0:1787826821.739325:0:78051:0:(file.c:1090:ll_file_open()) START file f33a.replay-single--4:[0x200015bc1:0x5:0x0](ffff98561158db88), flags 100102 00000002:00100000:3.0:1787826821.739341:0:78051:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff985610e64780 x1874669225537536/t193273528329(193273528329) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826837 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:3.0:1787826821.739365:0:78051:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e00f8 from it ffff985610c4a420 00010000:00010000:3.0:1787826821.739370:0:78051:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07400/0xb3f83dad1b7e00f8 lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a0e7c expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.739378:0:78051:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07400/0xb3f83dad1b7e00f8 lrc: 2/0,0 mode: PR/PR res: [0x200015bc1:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a0e7c expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826821.739397:0:78051:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200015bc1:0x5:0x0](ffff98561158db88), inode mode 81a4 mask 40 00000080:00200008:3.0:1787826821.739417:0:78051:0:(file.c:479:ll_file_release()) START file f33a.replay-single--4:[0x200015bc1:0x5:0x0](ffff98561158db88), flags 100002 00000002:00100000:3.0:1787826821.739448:0:78051:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff985610e64780 x1874669225537536/t193273528329(193273528329) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826837 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:3.0:1787826821.739456:0:78051:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:3.0:1787826821.739460:0:78051:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610e67c00 x1874669225537792/t0(0) o35->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:3.0:1787826821.739475:0:78051:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e67c00 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78051:1874669225537792:192.168.203.114@tcp:35:createmany.0 00000100:00100000:3.0:1787826821.739524:0:78051:0:(client.c:2715:ptlrpc_set_wait()) set ffff985610f13540 going to sleep for 11 seconds 00000100:00100000:3.0:1787826821.740764:0:78051:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e67c00 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78051:1874669225537792:192.168.203.114@tcp:35:createmany.0 00000002:00100000:3.0:1787826821.740801:0:78051:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff985610e64780 x1874669225537536/t193273528329(193273528329) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826837 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:3.0:1787826821.740867:0:78051:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:3.0:1787826821.740886:0:78051:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826821.740938:0:78051:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e00ff lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 78051 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.740945:0:78051:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e00ff lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78051 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.740952:0:78051:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e00ff lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78051 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826821.740967:0:78051:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e67480 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78051:1874669225537920:192.168.203.114@tcp:101:createmany.0 00000100:00100000:3.0:1787826821.741046:0:78051:0:(client.c:2715:ptlrpc_set_wait()) set ffff985610f13540 going to sleep for 11 seconds 00000100:00100000:3.0:1787826821.742572:0:78051:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e67480 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78051:1874669225537920:192.168.203.114@tcp:101:createmany.0 00010000:00010000:3.0:1787826821.742585:0:78051:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e00ff lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0e91 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.742591:0:78051:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e00ff lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0e91 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.742597:0:78051:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e00ff lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0e91 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.742604:0:78051:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e00ff lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0e91 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.742609:0:78051:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e00ff lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0e91 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.742614:0:78051:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e00ff lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0e91 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.742618:0:78051:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e00ff lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0e91 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826821.742624:0:78051:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff985610e67480 x1874669225537920/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826837 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826821.742631:0:78051:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e00ff lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0e91 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.742642:0:78051:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:3.0:1787826821.742646:0:78051:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:3.0:1787826821.742659:0:78051:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:3.0:1787826821.742662:0:78051:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e00ff 00000080:00010000:3.0:1787826821.742665:0:78051:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:3.0:1787826821.742667:0:78051:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e00ff from it ffffab3ac1cafbf8 00010000:00010000:3.0:1787826821.742669:0:78051:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e00ff lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0e91 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.742675:0:78051:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e00ff lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0e91 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826821.742687:0:78051:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:3.0:1787826821.742717:0:78051:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:3.0:1787826821.742725:0:78051:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826821.742731:0:78051:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e00ff lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0e91 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826821.742736:0:78051:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e00ff lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0e91 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826821.742740:0:78051:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826821.742742:0:78051:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:3.0:1787826821.742744:0:78051:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e00ff 00000080:00010000:3.0:1787826821.742747:0:78051:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e00ff from it ffffab3ac1cafb38 00010000:00010000:3.0:1787826821.742748:0:78051:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e00ff lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0e91 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826821.742753:0:78051:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e00ff lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0e91 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:3.0:1787826821.742758:0:78051:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 3 00000080:00200000:3.0:1787826821.742761:0:78051:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f33a.replay-single--5, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), file ffff985611f24480, open_flags 8042, mode 81a4 opened 0 00000080:00200000:3.0:1787826821.742765:0:78051:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f33a.replay-single--5, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=open|creat 00000002:00010000:3.0:1787826821.742777:0:78051:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f33a.replay-single--5,[0x200015bc1:0x6:0x0]) in obj [0x200000007:0x1:0x0], intent: open|creat flags 0100103 00010000:00010000:3.0:1787826821.742783:0:78051:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e00ff lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x5aa6d504d91a0e91 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826821.742808:0:78051:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e00ff lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a0e91 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826821.742813:0:78051:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826821.742815:0:78051:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e00ff lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a0e91 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826821.742833:0:78051:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07a00/0xb3f83dad1b7e0106 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 78051 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.742837:0:78051:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07a00/0xb3f83dad1b7e0106 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78051 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:3.0:1787826821.742843:0:78051:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:3.0:1787826821.742847:0:78051:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610e67480 x1874669225538176/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:3.0:1787826821.742860:0:78051:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07a00/0xb3f83dad1b7e0106 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78051 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826821.742872:0:78051:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e67480 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78051:1874669225538176:192.168.203.114@tcp:101:createmany.0 00000100:00100000:3.0:1787826821.742948:0:78051:0:(client.c:2715:ptlrpc_set_wait()) set ffff985610f13540 going to sleep for 11 seconds 00000100:00100000:3.0:1787826821.744952:0:78051:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e67480 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78051:1874669225538176:192.168.203.114@tcp:101:createmany.0 00010000:00010000:3.0:1787826821.744973:0:78051:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07a00/0xb3f83dad1b7e0106 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0ead expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.744989:0:78051:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07a00/0xb3f83dad1b7e0106 lrc: 4/0,1 mode: --/PR res: [0x200015bc1:0x6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0ead expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.744994:0:78051:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07a00/0xb3f83dad1b7e0106 lrc: 4/0,1 mode: --/PR res: [0x200015bc1:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0ead expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.745000:0:78051:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07a00/0xb3f83dad1b7e0106 lrc: 4/0,1 mode: PR/PR res: [0x200015bc1:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0ead expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.745005:0:78051:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07a00/0xb3f83dad1b7e0106 lrc: 4/0,1 mode: PR/PR res: [0x200015bc1:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0ead expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.745011:0:78051:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07a00/0xb3f83dad1b7e0106 lrc: 5/1,1 mode: PR/PR res: [0x200015bc1:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0ead expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.745015:0:78051:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07a00/0xb3f83dad1b7e0106 lrc: 5/1,1 mode: PR/PR res: [0x200015bc1:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0ead expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.745020:0:78051:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07a00/0xb3f83dad1b7e0106 lrc: 4/1,0 mode: PR/PR res: [0x200015bc1:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0ead expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826821.745029:0:78051:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff985610e67480 x1874669225538176/t193273528331(193273528331) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/664 e 0 to 0 dl 1787826837 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:3.0:1787826821.745037:0:78051:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff985610e67480 x1874669225538176/t193273528331(193273528331) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/664 e 0 to 0 dl 1787826837 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826821.745045:0:78051:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07a00/0xb3f83dad1b7e0106 lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0ead expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:3.0:1787826821.745051:0:78051:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07a00/0xb3f83dad1b7e0106 lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0ead expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.745056:0:78051:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8590023617/6 (0 0) 00000080:00200000:3.0:1787826821.745059:0:78051:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:3.0:1787826821.745065:0:78051:0:(namei.c:1628:ll_create_it()) VFS Op:name=f33a.replay-single--5, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=open|creat 00000080:00200000:3.0:1787826821.745079:0:78051:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200015bc1:0x6:0x0](ffff98561158d348) 00000080:00200000:3.0:1787826821.745125:0:78051:0:(file.c:6854:ll_layout_conf()) [0x200015bc1:0x6:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:3.0:1787826821.745129:0:78051:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x200015bc1:0x6:0x0](ffff98561158d348) 00000080:00010000:3.0:1787826821.745132:0:78051:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200015bc1:0x6:0x0](ffff98561158d348) for lock 0xb3f83dad1b7e0106 00000080:00010000:3.0:1787826821.745138:0:78051:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200015bc1:0x6:0x0](ffff98561158d348) 00000080:00200008:3.0:1787826821.745147:0:78051:0:(file.c:1090:ll_file_open()) START file f33a.replay-single--5:[0x200015bc1:0x6:0x0](ffff98561158d348), flags 100102 00000002:00100000:3.0:1787826821.745164:0:78051:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff985610e67480 x1874669225538176/t193273528331(193273528331) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826837 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:3.0:1787826821.745184:0:78051:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0106 from it ffff985610c4a420 00010000:00010000:3.0:1787826821.745187:0:78051:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07a00/0xb3f83dad1b7e0106 lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a0ead expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.745196:0:78051:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07a00/0xb3f83dad1b7e0106 lrc: 2/0,0 mode: PR/PR res: [0x200015bc1:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a0ead expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826821.745206:0:78051:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200015bc1:0x6:0x0](ffff98561158d348), inode mode 81a4 mask 40 00000080:00200008:3.0:1787826821.745223:0:78051:0:(file.c:479:ll_file_release()) START file f33a.replay-single--5:[0x200015bc1:0x6:0x0](ffff98561158d348), flags 100002 00000002:00100000:3.0:1787826821.745256:0:78051:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff985610e67480 x1874669225538176/t193273528331(193273528331) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826837 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:3.0:1787826821.745262:0:78051:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:3.0:1787826821.745265:0:78051:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610e643c0 x1874669225538432/t0(0) o35->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:3.0:1787826821.745274:0:78051:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e643c0 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78051:1874669225538432:192.168.203.114@tcp:35:createmany.0 00000100:00100000:3.0:1787826821.745306:0:78051:0:(client.c:2715:ptlrpc_set_wait()) set ffff985610f13c00 going to sleep for 11 seconds 00000100:00100000:3.0:1787826821.746903:0:78051:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e643c0 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78051:1874669225538432:192.168.203.114@tcp:35:createmany.0 00000002:00100000:3.0:1787826821.746923:0:78051:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff985610e67480 x1874669225538176/t193273528331(193273528331) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826837 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:3.0:1787826821.746973:0:78051:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:3.0:1787826821.746985:0:78051:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826821.747017:0:78051:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7e010d lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 78051 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.747025:0:78051:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7e010d lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78051 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.747034:0:78051:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7e010d lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78051 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826821.747048:0:78051:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e65a40 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78051:1874669225538560:192.168.203.114@tcp:101:createmany.0 00000100:00100000:3.0:1787826821.747097:0:78051:0:(client.c:2715:ptlrpc_set_wait()) set ffff985610f13c00 going to sleep for 11 seconds 00000100:00100000:3.0:1787826821.748299:0:78051:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e65a40 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78051:1874669225538560:192.168.203.114@tcp:101:createmany.0 00010000:00010000:3.0:1787826821.748323:0:78051:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7e010d lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0ec2 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.748336:0:78051:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7e010d lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0ec2 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.748347:0:78051:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7e010d lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0ec2 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.748357:0:78051:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7e010d lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0ec2 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.748368:0:78051:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7e010d lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0ec2 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.748377:0:78051:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7e010d lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0ec2 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.748384:0:78051:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7e010d lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0ec2 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826821.748395:0:78051:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff985610e65a40 x1874669225538560/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826837 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826821.748415:0:78051:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7e010d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0ec2 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.748428:0:78051:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:3.0:1787826821.748434:0:78051:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:3.0:1787826821.748463:0:78051:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:3.0:1787826821.748468:0:78051:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e010d 00000080:00010000:3.0:1787826821.748473:0:78051:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:3.0:1787826821.748477:0:78051:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e010d from it ffffab3ac1cafbf8 00010000:00010000:3.0:1787826821.748480:0:78051:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7e010d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0ec2 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.748488:0:78051:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7e010d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0ec2 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826821.748510:0:78051:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:3.0:1787826821.748533:0:78051:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:3.0:1787826821.748546:0:78051:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826821.748556:0:78051:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7e010d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0ec2 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826821.748565:0:78051:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7e010d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0ec2 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826821.748571:0:78051:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826821.748575:0:78051:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:3.0:1787826821.748578:0:78051:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e010d 00000080:00010000:3.0:1787826821.748582:0:78051:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e010d from it ffffab3ac1cafb38 00010000:00010000:3.0:1787826821.748585:0:78051:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7e010d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0ec2 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826821.748591:0:78051:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7e010d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0ec2 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:3.0:1787826821.748598:0:78051:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 3 00000080:00200000:3.0:1787826821.748603:0:78051:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f33a.replay-single--6, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), file ffff985611f25e00, open_flags 8042, mode 81a4 opened 0 00000080:00200000:3.0:1787826821.748612:0:78051:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f33a.replay-single--6, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=open|creat 00000002:00010000:3.0:1787826821.748636:0:78051:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f33a.replay-single--6,[0x200015bc1:0x7:0x0]) in obj [0x200000007:0x1:0x0], intent: open|creat flags 0100103 00010000:00010000:3.0:1787826821.748647:0:78051:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7e010d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x5aa6d504d91a0ec2 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826821.748737:0:78051:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7e010d lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a0ec2 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826821.748746:0:78051:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826821.748753:0:78051:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7e010d lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a0ec2 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826821.748807:0:78051:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e0114 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 78051 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.748817:0:78051:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e0114 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78051 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:3.0:1787826821.748831:0:78051:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:3.0:1787826821.748839:0:78051:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610e65a40 x1874669225538816/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:3.0:1787826821.748853:0:78051:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e0114 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78051 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826821.748877:0:78051:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e65a40 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78051:1874669225538816:192.168.203.114@tcp:101:createmany.0 00000100:00100000:3.0:1787826821.749054:0:78051:0:(client.c:2715:ptlrpc_set_wait()) set ffff985610f13c00 going to sleep for 11 seconds 00000100:00100000:3.0:1787826821.750977:0:78051:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e65a40 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78051:1874669225538816:192.168.203.114@tcp:101:createmany.0 00010000:00010000:3.0:1787826821.750999:0:78051:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e0114 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0ede expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.751024:0:78051:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e0114 lrc: 4/0,1 mode: --/PR res: [0x200015bc1:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0ede expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.751030:0:78051:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e0114 lrc: 4/0,1 mode: --/PR res: [0x200015bc1:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0ede expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.751037:0:78051:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e0114 lrc: 4/0,1 mode: PR/PR res: [0x200015bc1:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0ede expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.751043:0:78051:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e0114 lrc: 4/0,1 mode: PR/PR res: [0x200015bc1:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0ede expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.751053:0:78051:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e0114 lrc: 5/1,1 mode: PR/PR res: [0x200015bc1:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0ede expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.751058:0:78051:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e0114 lrc: 5/1,1 mode: PR/PR res: [0x200015bc1:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0ede expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.751067:0:78051:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e0114 lrc: 4/1,0 mode: PR/PR res: [0x200015bc1:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0ede expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826821.751074:0:78051:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff985610e65a40 x1874669225538816/t193273528333(193273528333) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/664 e 0 to 0 dl 1787826837 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:3.0:1787826821.751083:0:78051:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff985610e65a40 x1874669225538816/t193273528333(193273528333) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/664 e 0 to 0 dl 1787826837 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826821.751092:0:78051:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e0114 lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0ede expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:3.0:1787826821.751101:0:78051:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e0114 lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0ede expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.751108:0:78051:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8590023617/7 (0 0) 00000080:00200000:3.0:1787826821.751113:0:78051:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:3.0:1787826821.751125:0:78051:0:(namei.c:1628:ll_create_it()) VFS Op:name=f33a.replay-single--6, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=open|creat 00000080:00200000:3.0:1787826821.751152:0:78051:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200015bc1:0x7:0x0](ffff98561158a1c8) 00000080:00200000:3.0:1787826821.751244:0:78051:0:(file.c:6854:ll_layout_conf()) [0x200015bc1:0x7:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:3.0:1787826821.751249:0:78051:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x200015bc1:0x7:0x0](ffff98561158a1c8) 00000080:00010000:3.0:1787826821.751252:0:78051:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200015bc1:0x7:0x0](ffff98561158a1c8) for lock 0xb3f83dad1b7e0114 00000080:00010000:3.0:1787826821.751261:0:78051:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200015bc1:0x7:0x0](ffff98561158a1c8) 00000080:00200008:3.0:1787826821.751277:0:78051:0:(file.c:1090:ll_file_open()) START file f33a.replay-single--6:[0x200015bc1:0x7:0x0](ffff98561158a1c8), flags 100102 00000002:00100000:3.0:1787826821.751296:0:78051:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff985610e65a40 x1874669225538816/t193273528333(193273528333) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826837 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:3.0:1787826821.751331:0:78051:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0114 from it ffff985610c4a420 00010000:00010000:3.0:1787826821.751334:0:78051:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e0114 lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a0ede expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.751340:0:78051:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e0114 lrc: 2/0,0 mode: PR/PR res: [0x200015bc1:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a0ede expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826821.751360:0:78051:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200015bc1:0x7:0x0](ffff98561158a1c8), inode mode 81a4 mask 40 00000080:00200008:3.0:1787826821.751394:0:78051:0:(file.c:479:ll_file_release()) START file f33a.replay-single--6:[0x200015bc1:0x7:0x0](ffff98561158a1c8), flags 100002 00000002:00100000:3.0:1787826821.751437:0:78051:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff985610e65a40 x1874669225538816/t193273528333(193273528333) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826837 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:3.0:1787826821.751447:0:78051:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:3.0:1787826821.751452:0:78051:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610e64f00 x1874669225539072/t0(0) o35->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:3.0:1787826821.751471:0:78051:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e64f00 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78051:1874669225539072:192.168.203.114@tcp:35:createmany.0 00000100:00100000:3.0:1787826821.751530:0:78051:0:(client.c:2715:ptlrpc_set_wait()) set ffff985610f13300 going to sleep for 11 seconds 00000100:00100000:3.0:1787826821.752676:0:78051:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e64f00 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78051:1874669225539072:192.168.203.114@tcp:35:createmany.0 00000002:00100000:3.0:1787826821.752727:0:78051:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff985610e65a40 x1874669225538816/t193273528333(193273528333) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826837 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:3.0:1787826821.752799:0:78051:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:3.0:1787826821.752815:0:78051:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826821.752866:0:78051:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7e011b lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 78051 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.752880:0:78051:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7e011b lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78051 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.752890:0:78051:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7e011b lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78051 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826821.752905:0:78051:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e65e00 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78051:1874669225539200:192.168.203.114@tcp:101:createmany.0 00000100:00100000:3.0:1787826821.752957:0:78051:0:(client.c:2715:ptlrpc_set_wait()) set ffff985610f13300 going to sleep for 11 seconds 00000100:00100000:3.0:1787826821.754308:0:78051:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e65e00 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78051:1874669225539200:192.168.203.114@tcp:101:createmany.0 00010000:00010000:3.0:1787826821.754328:0:78051:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7e011b lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0ef3 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.754339:0:78051:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7e011b lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0ef3 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.754351:0:78051:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7e011b lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0ef3 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.754363:0:78051:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7e011b lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0ef3 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.754374:0:78051:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7e011b lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0ef3 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.754383:0:78051:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7e011b lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0ef3 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.754397:0:78051:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7e011b lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0ef3 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826821.754409:0:78051:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff985610e65e00 x1874669225539200/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826837 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826821.754423:0:78051:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7e011b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0ef3 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.754435:0:78051:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:3.0:1787826821.754441:0:78051:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:3.0:1787826821.754468:0:78051:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:3.0:1787826821.754474:0:78051:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e011b 00000080:00010000:3.0:1787826821.754480:0:78051:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:3.0:1787826821.754483:0:78051:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e011b from it ffffab3ac1cafbf8 00010000:00010000:3.0:1787826821.754487:0:78051:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7e011b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0ef3 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.754497:0:78051:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7e011b lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0ef3 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826821.754521:0:78051:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:3.0:1787826821.754541:0:78051:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:3.0:1787826821.754553:0:78051:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826821.754561:0:78051:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7e011b lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0ef3 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826821.754576:0:78051:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7e011b lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0ef3 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826821.754583:0:78051:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826821.754588:0:78051:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:3.0:1787826821.754590:0:78051:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e011b 00000080:00010000:3.0:1787826821.754594:0:78051:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e011b from it ffffab3ac1cafb38 00010000:00010000:3.0:1787826821.754597:0:78051:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7e011b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0ef3 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826821.754605:0:78051:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7e011b lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0ef3 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:3.0:1787826821.754612:0:78051:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 3 00000080:00200000:3.0:1787826821.754618:0:78051:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f33a.replay-single--7, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), file ffff985611f24c00, open_flags 8042, mode 81a4 opened 0 00000080:00200000:3.0:1787826821.754627:0:78051:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f33a.replay-single--7, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=open|creat 00000002:00010000:3.0:1787826821.754649:0:78051:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f33a.replay-single--7,[0x200015bc1:0x8:0x0]) in obj [0x200000007:0x1:0x0], intent: open|creat flags 0100103 00010000:00010000:3.0:1787826821.754660:0:78051:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7e011b lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x5aa6d504d91a0ef3 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826821.754727:0:78051:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7e011b lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a0ef3 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826821.754735:0:78051:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826821.754738:0:78051:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7e011b lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a0ef3 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826821.754770:0:78051:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e0122 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 78051 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.754775:0:78051:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e0122 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78051 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:3.0:1787826821.754782:0:78051:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:3.0:1787826821.754787:0:78051:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610e65e00 x1874669225539456/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:3.0:1787826821.754793:0:78051:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e0122 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78051 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826821.754807:0:78051:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e65e00 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78051:1874669225539456:192.168.203.114@tcp:101:createmany.0 00000100:00100000:3.0:1787826821.754975:0:78051:0:(client.c:2715:ptlrpc_set_wait()) set ffff985610f13300 going to sleep for 11 seconds 00000100:00100000:3.0:1787826821.757174:0:78051:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e65e00 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78051:1874669225539456:192.168.203.114@tcp:101:createmany.0 00010000:00010000:3.0:1787826821.757200:0:78051:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e0122 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0f0f expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.757239:0:78051:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e0122 lrc: 4/0,1 mode: --/PR res: [0x200015bc1:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0f0f expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.757249:0:78051:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e0122 lrc: 4/0,1 mode: --/PR res: [0x200015bc1:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0f0f expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.757258:0:78051:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e0122 lrc: 4/0,1 mode: PR/PR res: [0x200015bc1:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0f0f expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.757283:0:78051:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e0122 lrc: 4/0,1 mode: PR/PR res: [0x200015bc1:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0f0f expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.757296:0:78051:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e0122 lrc: 5/1,1 mode: PR/PR res: [0x200015bc1:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0f0f expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.757305:0:78051:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e0122 lrc: 5/1,1 mode: PR/PR res: [0x200015bc1:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0f0f expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.757312:0:78051:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e0122 lrc: 4/1,0 mode: PR/PR res: [0x200015bc1:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0f0f expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826821.757329:0:78051:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff985610e65e00 x1874669225539456/t193273528335(193273528335) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/664 e 0 to 0 dl 1787826837 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:3.0:1787826821.757342:0:78051:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff985610e65e00 x1874669225539456/t193273528335(193273528335) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/664 e 0 to 0 dl 1787826837 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826821.757355:0:78051:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e0122 lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0f0f expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:3.0:1787826821.757367:0:78051:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e0122 lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0f0f expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.757376:0:78051:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8590023617/8 (0 0) 00000080:00200000:3.0:1787826821.757382:0:78051:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:3.0:1787826821.757396:0:78051:0:(namei.c:1628:ll_create_it()) VFS Op:name=f33a.replay-single--7, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=open|creat 00000080:00200000:3.0:1787826821.757438:0:78051:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200015bc1:0x8:0x0](ffff98561158aa08) 00000080:00200000:3.0:1787826821.757523:0:78051:0:(file.c:6854:ll_layout_conf()) [0x200015bc1:0x8:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:3.0:1787826821.757528:0:78051:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x200015bc1:0x8:0x0](ffff98561158aa08) 00000080:00010000:3.0:1787826821.757531:0:78051:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200015bc1:0x8:0x0](ffff98561158aa08) for lock 0xb3f83dad1b7e0122 00000080:00010000:3.0:1787826821.757543:0:78051:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200015bc1:0x8:0x0](ffff98561158aa08) 00000080:00200008:3.0:1787826821.757559:0:78051:0:(file.c:1090:ll_file_open()) START file f33a.replay-single--7:[0x200015bc1:0x8:0x0](ffff98561158aa08), flags 100102 00000002:00100000:3.0:1787826821.757579:0:78051:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff985610e65e00 x1874669225539456/t193273528335(193273528335) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826837 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:3.0:1787826821.757612:0:78051:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0122 from it ffff985610c4a420 00010000:00010000:3.0:1787826821.757617:0:78051:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e0122 lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a0f0f expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.757626:0:78051:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e0122 lrc: 2/0,0 mode: PR/PR res: [0x200015bc1:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a0f0f expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826821.757647:0:78051:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200015bc1:0x8:0x0](ffff98561158aa08), inode mode 81a4 mask 40 00000080:00200008:3.0:1787826821.757682:0:78051:0:(file.c:479:ll_file_release()) START file f33a.replay-single--7:[0x200015bc1:0x8:0x0](ffff98561158aa08), flags 100002 00000002:00100000:3.0:1787826821.757744:0:78051:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff985610e65e00 x1874669225539456/t193273528335(193273528335) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826837 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:3.0:1787826821.757754:0:78051:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:3.0:1787826821.757759:0:78051:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610e66940 x1874669225539712/t0(0) o35->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:3.0:1787826821.757788:0:78051:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e66940 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78051:1874669225539712:192.168.203.114@tcp:35:createmany.0 00000100:00100000:3.0:1787826821.757858:0:78051:0:(client.c:2715:ptlrpc_set_wait()) set ffff985610f13a80 going to sleep for 11 seconds 00000100:00100000:3.0:1787826821.759511:0:78051:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e66940 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78051:1874669225539712:192.168.203.114@tcp:35:createmany.0 00000002:00100000:3.0:1787826821.759536:0:78051:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff985610e65e00 x1874669225539456/t193273528335(193273528335) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826837 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:3.0:1787826821.759599:0:78051:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:3.0:1787826821.759613:0:78051:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826821.759673:0:78051:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06600/0xb3f83dad1b7e0129 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 78051 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.759680:0:78051:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06600/0xb3f83dad1b7e0129 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78051 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.759687:0:78051:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06600/0xb3f83dad1b7e0129 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78051 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826821.759734:0:78051:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e4e580 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78051:1874669225539840:192.168.203.114@tcp:101:createmany.0 00000100:00100000:3.0:1787826821.759786:0:78051:0:(client.c:2715:ptlrpc_set_wait()) set ffff985610f13a80 going to sleep for 11 seconds 00000100:00100000:3.0:1787826821.760998:0:78051:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e4e580 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78051:1874669225539840:192.168.203.114@tcp:101:createmany.0 00010000:00010000:3.0:1787826821.761015:0:78051:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06600/0xb3f83dad1b7e0129 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0f24 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.761024:0:78051:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06600/0xb3f83dad1b7e0129 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0f24 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.761037:0:78051:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06600/0xb3f83dad1b7e0129 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0f24 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.761049:0:78051:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06600/0xb3f83dad1b7e0129 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0f24 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.761067:0:78051:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06600/0xb3f83dad1b7e0129 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0f24 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.761076:0:78051:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06600/0xb3f83dad1b7e0129 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0f24 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.761084:0:78051:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06600/0xb3f83dad1b7e0129 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0f24 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826821.761095:0:78051:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff985610e4e580 x1874669225539840/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826837 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826821.761108:0:78051:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06600/0xb3f83dad1b7e0129 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0f24 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.761120:0:78051:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:3.0:1787826821.761125:0:78051:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:3.0:1787826821.761151:0:78051:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:3.0:1787826821.761158:0:78051:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e0129 00000080:00010000:3.0:1787826821.761164:0:78051:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:3.0:1787826821.761167:0:78051:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0129 from it ffffab3ac1cafbf8 00010000:00010000:3.0:1787826821.761171:0:78051:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06600/0xb3f83dad1b7e0129 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0f24 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.761182:0:78051:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06600/0xb3f83dad1b7e0129 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0f24 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826821.761209:0:78051:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:3.0:1787826821.761245:0:78051:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:3.0:1787826821.761257:0:78051:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826821.761266:0:78051:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06600/0xb3f83dad1b7e0129 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0f24 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826821.761277:0:78051:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06600/0xb3f83dad1b7e0129 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0f24 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826821.761284:0:78051:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826821.761288:0:78051:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:3.0:1787826821.761291:0:78051:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e0129 00000080:00010000:3.0:1787826821.761295:0:78051:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0129 from it ffffab3ac1cafb38 00010000:00010000:3.0:1787826821.761298:0:78051:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06600/0xb3f83dad1b7e0129 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0f24 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826821.761305:0:78051:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06600/0xb3f83dad1b7e0129 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0f24 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:3.0:1787826821.761314:0:78051:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 3 00000080:00200000:3.0:1787826821.761319:0:78051:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f33a.replay-single--8, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), file ffff985611f25080, open_flags 8042, mode 81a4 opened 0 00000080:00200000:3.0:1787826821.761329:0:78051:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f33a.replay-single--8, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=open|creat 00000002:00010000:3.0:1787826821.761351:0:78051:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f33a.replay-single--8,[0x200015bc1:0x9:0x0]) in obj [0x200000007:0x1:0x0], intent: open|creat flags 0100103 00010000:00010000:3.0:1787826821.761363:0:78051:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06600/0xb3f83dad1b7e0129 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x5aa6d504d91a0f24 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826821.761412:0:78051:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06600/0xb3f83dad1b7e0129 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a0f24 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826821.761418:0:78051:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826821.761420:0:78051:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06600/0xb3f83dad1b7e0129 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a0f24 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826821.761448:0:78051:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7e0130 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 78051 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.761455:0:78051:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7e0130 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78051 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:3.0:1787826821.761464:0:78051:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:3.0:1787826821.761472:0:78051:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610e4e580 x1874669225540096/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:3.0:1787826821.761482:0:78051:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7e0130 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78051 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826821.761497:0:78051:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e4e580 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78051:1874669225540096:192.168.203.114@tcp:101:createmany.0 00000100:00100000:3.0:1787826821.761633:0:78051:0:(client.c:2715:ptlrpc_set_wait()) set ffff985610f13a80 going to sleep for 11 seconds 00000100:00100000:3.0:1787826821.763806:0:78051:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e4e580 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78051:1874669225540096:192.168.203.114@tcp:101:createmany.0 00010000:00010000:3.0:1787826821.763833:0:78051:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7e0130 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0f40 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.763860:0:78051:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7e0130 lrc: 4/0,1 mode: --/PR res: [0x200015bc1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0f40 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.763875:0:78051:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7e0130 lrc: 4/0,1 mode: --/PR res: [0x200015bc1:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0f40 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.763885:0:78051:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7e0130 lrc: 4/0,1 mode: PR/PR res: [0x200015bc1:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0f40 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.763896:0:78051:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7e0130 lrc: 4/0,1 mode: PR/PR res: [0x200015bc1:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0f40 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.763917:0:78051:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7e0130 lrc: 5/1,1 mode: PR/PR res: [0x200015bc1:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0f40 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.763926:0:78051:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7e0130 lrc: 5/1,1 mode: PR/PR res: [0x200015bc1:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0f40 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.763934:0:78051:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7e0130 lrc: 4/1,0 mode: PR/PR res: [0x200015bc1:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0f40 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826821.763947:0:78051:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff985610e4e580 x1874669225540096/t193273528337(193273528337) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/664 e 0 to 0 dl 1787826837 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:3.0:1787826821.763960:0:78051:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff985610e4e580 x1874669225540096/t193273528337(193273528337) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/664 e 0 to 0 dl 1787826837 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826821.763986:0:78051:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7e0130 lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0f40 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:3.0:1787826821.763998:0:78051:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7e0130 lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0f40 expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.764010:0:78051:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8590023617/9 (0 0) 00000080:00200000:3.0:1787826821.764016:0:78051:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:3.0:1787826821.764036:0:78051:0:(namei.c:1628:ll_create_it()) VFS Op:name=f33a.replay-single--8, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=open|creat 00000080:00200000:3.0:1787826821.764066:0:78051:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200015bc1:0x9:0x0](ffff9856115880c8) 00000080:00200000:3.0:1787826821.764153:0:78051:0:(file.c:6854:ll_layout_conf()) [0x200015bc1:0x9:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:3.0:1787826821.764159:0:78051:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x200015bc1:0x9:0x0](ffff9856115880c8) 00000080:00010000:3.0:1787826821.764162:0:78051:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200015bc1:0x9:0x0](ffff9856115880c8) for lock 0xb3f83dad1b7e0130 00000080:00010000:3.0:1787826821.764172:0:78051:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200015bc1:0x9:0x0](ffff9856115880c8) 00000080:00200008:3.0:1787826821.764204:0:78051:0:(file.c:1090:ll_file_open()) START file f33a.replay-single--8:[0x200015bc1:0x9:0x0](ffff9856115880c8), flags 100102 00000002:00100000:3.0:1787826821.764235:0:78051:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff985610e4e580 x1874669225540096/t193273528337(193273528337) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826837 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:3.0:1787826821.764275:0:78051:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0130 from it ffff985610c4a420 00010000:00010000:3.0:1787826821.764280:0:78051:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7e0130 lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a0f40 expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.764289:0:78051:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7e0130 lrc: 2/0,0 mode: PR/PR res: [0x200015bc1:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a0f40 expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826821.764309:0:78051:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200015bc1:0x9:0x0](ffff9856115880c8), inode mode 81a4 mask 40 00000080:00200008:3.0:1787826821.764345:0:78051:0:(file.c:479:ll_file_release()) START file f33a.replay-single--8:[0x200015bc1:0x9:0x0](ffff9856115880c8), flags 100002 00000002:00100000:3.0:1787826821.764395:0:78051:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff985610e4e580 x1874669225540096/t193273528337(193273528337) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826837 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:3.0:1787826821.764404:0:78051:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:3.0:1787826821.764410:0:78051:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610e4e1c0 x1874669225540352/t0(0) o35->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:3.0:1787826821.764435:0:78051:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e4e1c0 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78051:1874669225540352:192.168.203.114@tcp:35:createmany.0 00000100:00100000:3.0:1787826821.764510:0:78051:0:(client.c:2715:ptlrpc_set_wait()) set ffff985610f139c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787826821.765991:0:78051:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e4e1c0 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78051:1874669225540352:192.168.203.114@tcp:35:createmany.0 00000002:00100000:3.0:1787826821.766012:0:78051:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff985610e4e580 x1874669225540096/t193273528337(193273528337) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826837 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:3.0:1787826821.766073:0:78051:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:3.0:1787826821.766087:0:78051:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826821.766125:0:78051:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07000/0xb3f83dad1b7e0137 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 78051 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.766132:0:78051:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07000/0xb3f83dad1b7e0137 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78051 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.766140:0:78051:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07000/0xb3f83dad1b7e0137 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78051 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826821.766163:0:78051:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e4e940 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78051:1874669225540480:192.168.203.114@tcp:101:createmany.0 00000100:00100000:3.0:1787826821.766211:0:78051:0:(client.c:2715:ptlrpc_set_wait()) set ffff985610f139c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787826821.767214:0:78051:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e4e940 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78051:1874669225540480:192.168.203.114@tcp:101:createmany.0 00010000:00010000:3.0:1787826821.767240:0:78051:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07000/0xb3f83dad1b7e0137 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0f55 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.767250:0:78051:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07000/0xb3f83dad1b7e0137 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0f55 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.767269:0:78051:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07000/0xb3f83dad1b7e0137 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0f55 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.767280:0:78051:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07000/0xb3f83dad1b7e0137 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0f55 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.767291:0:78051:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07000/0xb3f83dad1b7e0137 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0f55 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.767299:0:78051:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07000/0xb3f83dad1b7e0137 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0f55 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.767308:0:78051:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07000/0xb3f83dad1b7e0137 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0f55 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826821.767317:0:78051:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff985610e4e940 x1874669225540480/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826837 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826821.767331:0:78051:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07000/0xb3f83dad1b7e0137 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0f55 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.767342:0:78051:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:3.0:1787826821.767348:0:78051:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:3.0:1787826821.767374:0:78051:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:3.0:1787826821.767379:0:78051:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e0137 00000080:00010000:3.0:1787826821.767383:0:78051:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:3.0:1787826821.767386:0:78051:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0137 from it ffffab3ac1cafbf8 00010000:00010000:3.0:1787826821.767389:0:78051:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07000/0xb3f83dad1b7e0137 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0f55 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.767395:0:78051:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07000/0xb3f83dad1b7e0137 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0f55 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826821.767414:0:78051:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:3.0:1787826821.767432:0:78051:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:3.0:1787826821.767442:0:78051:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826821.767451:0:78051:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07000/0xb3f83dad1b7e0137 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0f55 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826821.767460:0:78051:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07000/0xb3f83dad1b7e0137 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0f55 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826821.767467:0:78051:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826821.767471:0:78051:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:3.0:1787826821.767475:0:78051:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e0137 00000080:00010000:3.0:1787826821.767478:0:78051:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0137 from it ffffab3ac1cafb38 00010000:00010000:3.0:1787826821.767482:0:78051:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07000/0xb3f83dad1b7e0137 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0f55 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826821.767489:0:78051:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07000/0xb3f83dad1b7e0137 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0f55 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:3.0:1787826821.767497:0:78051:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 3 00000080:00200000:3.0:1787826821.767502:0:78051:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f33a.replay-single--9, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), file ffff985611f25e00, open_flags 8042, mode 81a4 opened 0 00000080:00200000:3.0:1787826821.767511:0:78051:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f33a.replay-single--9, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=open|creat 00000002:00010000:3.0:1787826821.767541:0:78051:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f33a.replay-single--9,[0x200015bc1:0xa:0x0]) in obj [0x200000007:0x1:0x0], intent: open|creat flags 0100103 00010000:00010000:3.0:1787826821.767553:0:78051:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07000/0xb3f83dad1b7e0137 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x5aa6d504d91a0f55 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826821.767601:0:78051:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07000/0xb3f83dad1b7e0137 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a0f55 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826821.767608:0:78051:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826821.767611:0:78051:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07000/0xb3f83dad1b7e0137 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a0f55 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826821.767643:0:78051:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06000/0xb3f83dad1b7e013e lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 78051 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.767650:0:78051:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06000/0xb3f83dad1b7e013e lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78051 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:3.0:1787826821.767659:0:78051:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:3.0:1787826821.767666:0:78051:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610e4e940 x1874669225540736/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:3.0:1787826821.767675:0:78051:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06000/0xb3f83dad1b7e013e lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78051 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826821.767726:0:78051:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e4e940 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78051:1874669225540736:192.168.203.114@tcp:101:createmany.0 00000100:00100000:3.0:1787826821.767850:0:78051:0:(client.c:2715:ptlrpc_set_wait()) set ffff985610f139c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787826821.769514:0:78051:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e4e940 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78051:1874669225540736:192.168.203.114@tcp:101:createmany.0 00010000:00010000:3.0:1787826821.769533:0:78051:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06000/0xb3f83dad1b7e013e lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0f71 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.769573:0:78051:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06000/0xb3f83dad1b7e013e lrc: 4/0,1 mode: --/PR res: [0x200015bc1:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0f71 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.769582:0:78051:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06000/0xb3f83dad1b7e013e lrc: 4/0,1 mode: --/PR res: [0x200015bc1:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0f71 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.769591:0:78051:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06000/0xb3f83dad1b7e013e lrc: 4/0,1 mode: PR/PR res: [0x200015bc1:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0f71 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.769599:0:78051:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06000/0xb3f83dad1b7e013e lrc: 4/0,1 mode: PR/PR res: [0x200015bc1:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0f71 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.769608:0:78051:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06000/0xb3f83dad1b7e013e lrc: 5/1,1 mode: PR/PR res: [0x200015bc1:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0f71 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.769613:0:78051:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06000/0xb3f83dad1b7e013e lrc: 5/1,1 mode: PR/PR res: [0x200015bc1:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0f71 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.769617:0:78051:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06000/0xb3f83dad1b7e013e lrc: 4/1,0 mode: PR/PR res: [0x200015bc1:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0f71 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826821.769624:0:78051:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff985610e4e940 x1874669225540736/t193273528339(193273528339) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/664 e 0 to 0 dl 1787826837 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:3.0:1787826821.769636:0:78051:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff985610e4e940 x1874669225540736/t193273528339(193273528339) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/664 e 0 to 0 dl 1787826837 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826821.769657:0:78051:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06000/0xb3f83dad1b7e013e lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0f71 expref: -99 pid: 78051 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:3.0:1787826821.769673:0:78051:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06000/0xb3f83dad1b7e013e lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0f71 expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.769683:0:78051:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8590023617/10 (0 0) 00000080:00200000:3.0:1787826821.769708:0:78051:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:3.0:1787826821.769721:0:78051:0:(namei.c:1628:ll_create_it()) VFS Op:name=f33a.replay-single--9, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=open|creat 00000080:00200000:3.0:1787826821.769771:0:78051:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200015bc1:0xa:0x0](ffff98561011ba88) 00000080:00200000:3.0:1787826821.769842:0:78051:0:(file.c:6854:ll_layout_conf()) [0x200015bc1:0xa:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:3.0:1787826821.769846:0:78051:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x200015bc1:0xa:0x0](ffff98561011ba88) 00000080:00010000:3.0:1787826821.769849:0:78051:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200015bc1:0xa:0x0](ffff98561011ba88) for lock 0xb3f83dad1b7e013e 00000080:00010000:3.0:1787826821.769856:0:78051:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200015bc1:0xa:0x0](ffff98561011ba88) 00000080:00200008:3.0:1787826821.769868:0:78051:0:(file.c:1090:ll_file_open()) START file f33a.replay-single--9:[0x200015bc1:0xa:0x0](ffff98561011ba88), flags 100102 00000002:00100000:3.0:1787826821.769890:0:78051:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff985610e4e940 x1874669225540736/t193273528339(193273528339) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826837 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:3.0:1787826821.769917:0:78051:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e013e from it ffff985610c4a420 00010000:00010000:3.0:1787826821.769920:0:78051:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06000/0xb3f83dad1b7e013e lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a0f71 expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.769926:0:78051:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06000/0xb3f83dad1b7e013e lrc: 2/0,0 mode: PR/PR res: [0x200015bc1:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a0f71 expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826821.769943:0:78051:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200015bc1:0xa:0x0](ffff98561011ba88), inode mode 81a4 mask 40 00000080:00200008:3.0:1787826821.769984:0:78051:0:(file.c:479:ll_file_release()) START file f33a.replay-single--9:[0x200015bc1:0xa:0x0](ffff98561011ba88), flags 100002 00000002:00100000:3.0:1787826821.770025:0:78051:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff985610e4e940 x1874669225540736/t193273528339(193273528339) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826837 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:3.0:1787826821.770037:0:78051:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:3.0:1787826821.770041:0:78051:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610e4cb40 x1874669225540992/t0(0) o35->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:3.0:1787826821.770055:0:78051:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e4cb40 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78051:1874669225540992:192.168.203.114@tcp:35:createmany.0 00000100:00100000:3.0:1787826821.770111:0:78051:0:(client.c:2715:ptlrpc_set_wait()) set ffff985610f13480 going to sleep for 11 seconds 00000100:00100000:3.0:1787826821.771101:0:78051:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e4cb40 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78051:1874669225540992:192.168.203.114@tcp:35:createmany.0 00000002:00100000:3.0:1787826821.771122:0:78051:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff985610e4e940 x1874669225540736/t193273528339(193273528339) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826837 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:1.0:1787826821.772787:0:77345:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826821.772814:0:77345:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826821.772885:0:77345:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ac00/0xb3f83dad1b7e0145 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 77345 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.772903:0:77345:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ac00/0xb3f83dad1b7e0145 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 77345 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.772916:0:77345:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ac00/0xb3f83dad1b7e0145 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 77345 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826821.772941:0:77345:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e31680 pname:cluuid:pid:xid:nid:opc:job bash:7b4e8535-9db7-485d-abf6-bda966b63320:77345:1874669225541120:192.168.203.114@tcp:101:bash.0 00000100:00100000:1.0:1787826821.773019:0:77345:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607e06d80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826821.774345:0:77345:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e31680 pname:cluuid:pid:xid:nid:opc:job bash:7b4e8535-9db7-485d-abf6-bda966b63320:77345:1874669225541120:192.168.203.114@tcp:101:bash.0 00010000:00010000:1.0:1787826821.774361:0:77345:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ac00/0xb3f83dad1b7e0145 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0f86 expref: -99 pid: 77345 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.774371:0:77345:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ac00/0xb3f83dad1b7e0145 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0f86 expref: -99 pid: 77345 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.774384:0:77345:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ac00/0xb3f83dad1b7e0145 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0f86 expref: -99 pid: 77345 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.774396:0:77345:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ac00/0xb3f83dad1b7e0145 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0f86 expref: -99 pid: 77345 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.774406:0:77345:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ac00/0xb3f83dad1b7e0145 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0f86 expref: -99 pid: 77345 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.774414:0:77345:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ac00/0xb3f83dad1b7e0145 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0f86 expref: -99 pid: 77345 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.774422:0:77345:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ac00/0xb3f83dad1b7e0145 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0f86 expref: -99 pid: 77345 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826821.774433:0:77345:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff985610e31680 x1874669225541120/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826837 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'bash.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826821.774444:0:77345:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ac00/0xb3f83dad1b7e0145 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0f86 expref: -99 pid: 77345 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.774456:0:77345:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:1.0:1787826821.774462:0:77345:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:1.0:1787826821.774488:0:77345:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:1.0:1787826821.774494:0:77345:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e0145 00000080:00010000:1.0:1787826821.774501:0:77345:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826821.774503:0:77345:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0145 from it ffffab3ac1ca7bb0 00010000:00010000:1.0:1787826821.774505:0:77345:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ac00/0xb3f83dad1b7e0145 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0f86 expref: -99 pid: 77345 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.774512:0:77345:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ac00/0xb3f83dad1b7e0145 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0f86 expref: -99 pid: 77345 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826821.774530:0:77345:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 44 00000080:00200008:1.0:1787826821.774540:0:77345:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 304000 00000080:00200008:1.0:1787826821.774556:0:77345:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 2047, flags 0, foreign 0 00000002:00010000:1.0:1787826821.774573:0:77345:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: readdir flags 0 00010000:00010000:1.0:1787826821.774579:0:77345:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ac00/0xb3f83dad1b7e0145 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0f86 expref: -99 pid: 77345 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826821.774586:0:77345:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ac00/0xb3f83dad1b7e0145 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0f86 expref: -99 pid: 77345 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000100:00100000:1.0:1787826821.775244:0:77345:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e31680 pname:cluuid:pid:xid:nid:opc:job bash:7b4e8535-9db7-485d-abf6-bda966b63320:77345:1874669225541248:192.168.203.114@tcp:37:bash.0 00000100:00100000:1.0:1787826821.775324:0:77345:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607e06d80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826821.781500:0:77345:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e31680 pname:cluuid:pid:xid:nid:opc:job bash:7b4e8535-9db7-485d-abf6-bda966b63320:77345:1874669225541248:192.168.203.114@tcp:37:bash.0 00010000:00010000:1.0:1787826821.782287:0:77345:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ac00/0xb3f83dad1b7e0145 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0f86 expref: -99 pid: 77345 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826821.782311:0:77345:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ac00/0xb3f83dad1b7e0145 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0f86 expref: -99 pid: 77345 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826821.782345:0:77345:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826821.782370:0:77345:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826821.782386:0:77345:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ac00/0xb3f83dad1b7e0145 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0f86 expref: -99 pid: 77345 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826821.782396:0:77345:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ac00/0xb3f83dad1b7e0145 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0f86 expref: -99 pid: 77345 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826821.782401:0:77345:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826821.782407:0:77345:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826821.782410:0:77345:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e0145 00000080:00010000:1.0:1787826821.782415:0:77345:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0145 from it ffffab3ac1ca7cd8 00010000:00010000:1.0:1787826821.782417:0:77345:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ac00/0xb3f83dad1b7e0145 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0f86 expref: -99 pid: 77345 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826821.782424:0:77345:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ac00/0xb3f83dad1b7e0145 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0f86 expref: -99 pid: 77345 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787826821.782814:0:77345:0:(dir.c:332:ll_iterate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) pos/size0/20480 32bit_api 0 00000080:00200000:1.0:1787826821.782874:0:77345:0:(dir.c:332:ll_iterate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) pos/size18446744073709551614/20480 32bit_api 0 00000080:00200008:1.0:1787826821.782892:0:77345:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 304000 00000080:00200000:3.0:1787826821.788790:0:78052:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:3.0:1787826821.788819:0:78052:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826821.788837:0:78052:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ac00/0xb3f83dad1b7e0145 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0f86 expref: -99 pid: 77345 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:3.0:1787826821.788852:0:78052:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ac00/0xb3f83dad1b7e0145 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0f86 expref: -99 pid: 77345 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:3.0:1787826821.788858:0:78052:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826821.788865:0:78052:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:3.0:1787826821.788867:0:78052:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e0145 00000080:00010000:3.0:1787826821.788873:0:78052:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0145 from it ffffab3ac1da7b58 00010000:00010000:3.0:1787826821.788876:0:78052:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ac00/0xb3f83dad1b7e0145 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0f86 expref: -99 pid: 77345 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:3.0:1787826821.788882:0:78052:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ac00/0xb3f83dad1b7e0145 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0f86 expref: -99 pid: 77345 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00000080:00200000:3.0:1787826821.788895:0:78052:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:3.0:1787826821.788901:0:78052:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f33a.replay-single--0, flags=8 00000080:00200008:3.0:1787826821.788909:0:78052:0:(file.c:6313:ll_getattr_dentry()) START file f33a.replay-single--0:[0x200015bc1:0x1:0x0](ffff98561158ba88), request_mask 2047, flags 0, foreign 0 00000080:00200000:3.0:1787826821.788913:0:78052:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200015bc1:0x1:0x0](ffff98561158ba88),name=f33a.replay-single--0 00000002:00010000:3.0:1787826821.788925:0:78052:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200015bc1:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826821.788933:0:78052:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06400/0xb3f83dad1b7e00c0 lrc: 2/1,0 mode: PR/PR res: [0x200015bc1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a0db8 expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826821.788941:0:78052:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06400/0xb3f83dad1b7e00c0 lrc: 2/1,0 mode: PR/PR res: [0x200015bc1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a0db8 expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826821.788948:0:78052:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826821.788953:0:78052:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200015bc1:0x1:0x0](ffff98561158ba88) 00000080:00010000:3.0:1787826821.788955:0:78052:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200015bc1:0x1:0x0](ffff98561158ba88) for lock 0xb3f83dad1b7e00c0 00000080:00010000:3.0:1787826821.788958:0:78052:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e00c0 from it ffffab3ac1da7cb8 00010000:00010000:3.0:1787826821.788961:0:78052:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06400/0xb3f83dad1b7e00c0 lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a0db8 expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826821.788966:0:78052:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06400/0xb3f83dad1b7e00c0 lrc: 2/0,0 mode: PR/PR res: [0x200015bc1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a0db8 expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00000080:00200000:3.0:1787826821.788983:0:78052:0:(vvp_io.c:1885:vvp_io_init()) [0x200015bc1:0x1:0x0] ignore/verify layout 0/1, layout version 0 restore needed 0 00000001:00010000:3.0:1787826821.788994:0:78052:0:(glimpse.c:73:cl_glimpse_lock()) Glimpsing inode [0x200015bc1:0x1:0x0] 00000008:00010000:3.0:1787826821.789011:0:78052:0:(lustre_osc.h:166:osc_enq2ldlm_flags()) flags: a 00000008:00010000:3.0:1787826821.789015:0:78052:0:(osc_lock.c:1320:osc_lock_init()) ### lock ffff985610fb9430, osc lock ffff98560897f5f0, flags 0x400001002 00010000:00010000:3.0:1787826821.789057:0:78052:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0001-osc-ffff985602bc9800 lock: 00000000b57f056c/0xb3f83dad1b7e014c lrc: 3/1,0 mode: --/PR res: [0x2c0000401:0x5e2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 78052 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.789065:0:78052:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x400001000 ns: lustre-OST0001-osc-ffff985602bc9800 lock: 00000000b57f056c/0xb3f83dad1b7e014c lrc: 3/1,0 mode: --/PR res: [0x2c0000401:0x5e2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 78052 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000020:00200000:3.0:1787826821.789157:0:78052:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:3.0:1787826821.790944:0:78052:0:(file.c:1775:ll_merge_attr_nolock()) [0x200015bc1:0x1:0x0] updating i_size 0 i_blocks 0 00000020:00010000:3.0:1787826821.790953:0:78052:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff98560549a290 (ffff985610fe12d8/0) at cl_lock_release():245 00000020:00010000:3.0:1787826821.790957:0:78052:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff98560549a290 (ffff985610fe12d8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1787826821.790963:0:78052:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff985610fb9430 (ffff9856131a1758/1) at cl_lock_cancel():142 00000020:00010000:3.0:1787826821.790971:0:78052:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff98560549a290 (ffff985610fe12d8/0) at cl_lock_fini():72 00000020:00010000:3.0:1787826821.790974:0:78052:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff985610fb9430 (ffff985610fe12d8/1) at cl_lock_fini():72 00000080:00200000:3.0:1787826821.790987:0:78052:0:(vvp_io.c:304:vvp_io_fini()) [0x200015bc1:0x1:0x0] ignore/verify layout 0/1, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826821.791040:0:78052:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:3.0:1787826821.791056:0:78052:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826821.791066:0:78052:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ac00/0xb3f83dad1b7e0145 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0f86 expref: -99 pid: 77345 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00010000:00010000:3.0:1787826821.791077:0:78052:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ac00/0xb3f83dad1b7e0145 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0f86 expref: -99 pid: 77345 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00000080:00200000:3.0:1787826821.791084:0:78052:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826821.791089:0:78052:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:3.0:1787826821.791093:0:78052:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e0145 00000080:00010000:3.0:1787826821.791103:0:78052:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0145 from it ffffab3ac1da7c10 00010000:00010000:3.0:1787826821.791107:0:78052:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ac00/0xb3f83dad1b7e0145 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0f86 expref: -99 pid: 77345 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00010000:00010000:3.0:1787826821.791119:0:78052:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ac00/0xb3f83dad1b7e0145 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0f86 expref: -99 pid: 77345 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00000080:00200000:3.0:1787826821.791126:0:78052:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:3.0:1787826821.791132:0:78052:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f33a.replay-single--0, flags=0 00000080:00200000:3.0:1787826821.791135:0:78052:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:3.0:1787826821.791140:0:78052:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826821.791146:0:78052:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ac00/0xb3f83dad1b7e0145 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0f86 expref: -99 pid: 77345 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00010000:00010000:3.0:1787826821.791154:0:78052:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ac00/0xb3f83dad1b7e0145 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0f86 expref: -99 pid: 77345 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00000080:00200000:3.0:1787826821.791160:0:78052:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826821.791164:0:78052:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:3.0:1787826821.791167:0:78052:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e0145 00000080:00010000:3.0:1787826821.791170:0:78052:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0145 from it ffffab3ac1da7d68 00010000:00010000:3.0:1787826821.791173:0:78052:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ac00/0xb3f83dad1b7e0145 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0f86 expref: -99 pid: 77345 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00010000:00010000:3.0:1787826821.791181:0:78052:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ac00/0xb3f83dad1b7e0145 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0f86 expref: -99 pid: 77345 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00000080:00200000:3.0:1787826821.791187:0:78052:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 3 00000080:00200000:3.0:1787826821.791198:0:78052:0:(namei.c:2418:ll_unlink()) VFS Op:name=f33a.replay-single--0, dir=[0x200000007:0x1:0x0](ffff98561158e3c8) 00010000:00010000:3.0:1787826821.791236:0:78052:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ac00/0xb3f83dad1b7e0145 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x5aa6d504d91a0f86 expref: -99 pid: 77345 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00010000:00010000:3.0:1787826821.791303:0:78052:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06400/0xb3f83dad1b7e00c0 lrc: 2/0,0 mode: PR/PR res: [0x200015bc1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x5aa6d504d91a0db8 expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00000080:00200000:3.0:1787826821.791316:0:78052:0:(vvp_object.c:117:vvp_conf_set()) [0x200015bc1:0x1:0x0]: losing layout lock 00010000:00010000:3.0:1787826821.791341:0:78052:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06400/0xb3f83dad1b7e00c0 lrc: 1/0,0 mode: --/PR res: [0x200015bc1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x5aa6d504d91a0db8 expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826821.791349:0:78052:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ac00/0xb3f83dad1b7e0145 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a0f86 expref: -99 pid: 77345 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00010000:00010000:3.0:1787826821.791355:0:78052:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 2 locks packed 00010000:00010000:3.0:1787826821.791357:0:78052:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06400/0xb3f83dad1b7e00c0 lrc: 0/0,0 mode: --/PR res: [0x200015bc1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x5aa6d504d91a0db8 expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826821.791384:0:78052:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ac00/0xb3f83dad1b7e0145 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a0f86 expref: -99 pid: 77345 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00000020:00100000:3.0:1787826821.791397:0:78052:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:3.0:1787826821.791406:0:78052:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610e4f840 x1874669225541632/t0(0) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826821.791424:0:78052:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e4f840 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:78052:1874669225541632:192.168.203.114@tcp:36:rm.0 00000100:00100000:3.0:1787826821.791472:0:78052:0:(client.c:2715:ptlrpc_set_wait()) set ffff985610f13480 going to sleep for 11 seconds 00000100:00100000:1.0:1787826821.793389:0:78052:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e4f840 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:78052:1874669225541632:192.168.203.114@tcp:36:rm.0 00000080:00200000:1.0:1787826821.793461:0:78052:0:(vvp_io.c:1885:vvp_io_init()) [0x200015bc1:0x1:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826821.793477:0:78052:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826821.793510:0:78052:0:(vvp_io.c:304:vvp_io_fini()) [0x200015bc1:0x1:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826821.793519:0:78052:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200015bc1:0x1:0x0](ffff98561158ba88) 00000008:00010000:1.0:1787826821.793556:0:78052:0:(osc_object.c:202:osc_object_ast_clear()) ### update lock size 0 blocks 0 [cma]time: 0 0 0 by oinfo size 0 blocks 0 [cma]time 0 0 0 ns: lustre-OST0001-osc-ffff985602bc9800 lock: 00000000b57f056c/0xb3f83dad1b7e014c lrc: 1/0,0 mode: PR/PR res: [0x2c0000401:0x5e2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x5aa6d504d91a0f8d expref: -99 pid: 78052 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826821.793651:0:78052:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826821.793672:0:78052:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826821.793753:0:78052:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ae00/0xb3f83dad1b7e0153 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 78052 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.793761:0:78052:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ae00/0xb3f83dad1b7e0153 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78052 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.793770:0:78052:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ae00/0xb3f83dad1b7e0153 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78052 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826821.793790:0:78052:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e31680 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:78052:1874669225541760:192.168.203.114@tcp:101:rm.0 00000100:00100000:1.0:1787826821.793877:0:78052:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607e06d80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826821.795204:0:78052:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e31680 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:78052:1874669225541760:192.168.203.114@tcp:101:rm.0 00010000:00010000:1.0:1787826821.795234:0:78052:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ae00/0xb3f83dad1b7e0153 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0fb7 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.795247:0:78052:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ae00/0xb3f83dad1b7e0153 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0fb7 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.795259:0:78052:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ae00/0xb3f83dad1b7e0153 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0fb7 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.795272:0:78052:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ae00/0xb3f83dad1b7e0153 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0fb7 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.795282:0:78052:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ae00/0xb3f83dad1b7e0153 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0fb7 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.795287:0:78052:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ae00/0xb3f83dad1b7e0153 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0fb7 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.795292:0:78052:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ae00/0xb3f83dad1b7e0153 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0fb7 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826821.795300:0:78052:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff985610e31680 x1874669225541760/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826837 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826821.795310:0:78052:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ae00/0xb3f83dad1b7e0153 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0fb7 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.795317:0:78052:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:1.0:1787826821.795322:0:78052:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:1.0:1787826821.795347:0:78052:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:1.0:1787826821.795353:0:78052:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e0153 00000080:00010000:1.0:1787826821.795359:0:78052:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826821.795362:0:78052:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0153 from it ffffab3ac1da7b58 00010000:00010000:1.0:1787826821.795365:0:78052:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ae00/0xb3f83dad1b7e0153 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0fb7 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.795381:0:78052:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ae00/0xb3f83dad1b7e0153 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0fb7 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826821.795404:0:78052:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787826821.795415:0:78052:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f33a.replay-single--1, flags=8 00000080:00200008:1.0:1787826821.795424:0:78052:0:(file.c:6313:ll_getattr_dentry()) START file f33a.replay-single--1:[0x200015bc1:0x2:0x0](ffff98561158f448), request_mask 2047, flags 0, foreign 0 00000080:00200000:1.0:1787826821.795428:0:78052:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200015bc1:0x2:0x0](ffff98561158f448),name=f33a.replay-single--1 00000002:00010000:1.0:1787826821.795445:0:78052:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200015bc1:0x2:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826821.795454:0:78052:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07800/0xb3f83dad1b7e00ce lrc: 2/1,0 mode: PR/PR res: [0x200015bc1:0x2:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a0de9 expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826821.795465:0:78052:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07800/0xb3f83dad1b7e00ce lrc: 2/1,0 mode: PR/PR res: [0x200015bc1:0x2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a0de9 expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826821.795471:0:78052:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826821.795476:0:78052:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200015bc1:0x2:0x0](ffff98561158f448) 00000080:00010000:1.0:1787826821.795478:0:78052:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200015bc1:0x2:0x0](ffff98561158f448) for lock 0xb3f83dad1b7e00ce 00000080:00010000:1.0:1787826821.795483:0:78052:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e00ce from it ffffab3ac1da7cb8 00010000:00010000:1.0:1787826821.795486:0:78052:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07800/0xb3f83dad1b7e00ce lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a0de9 expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826821.795493:0:78052:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07800/0xb3f83dad1b7e00ce lrc: 2/0,0 mode: PR/PR res: [0x200015bc1:0x2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a0de9 expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826821.795515:0:78052:0:(vvp_io.c:1885:vvp_io_init()) [0x200015bc1:0x2:0x0] ignore/verify layout 0/1, layout version 0 restore needed 0 00000001:00010000:1.0:1787826821.795526:0:78052:0:(glimpse.c:73:cl_glimpse_lock()) Glimpsing inode [0x200015bc1:0x2:0x0] 00000008:00010000:1.0:1787826821.795549:0:78052:0:(lustre_osc.h:166:osc_enq2ldlm_flags()) flags: a 00000008:00010000:1.0:1787826821.795553:0:78052:0:(osc_lock.c:1320:osc_lock_init()) ### lock ffff98560560c830, osc lock ffff985602f8efd8, flags 0x400001002 00010000:00010000:1.0:1787826821.795597:0:78052:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0000-osc-ffff985602bc9800 lock: 000000009c08d72b/0xb3f83dad1b7e015a lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5e2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 78052 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.795607:0:78052:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x400001000 ns: lustre-OST0000-osc-ffff985602bc9800 lock: 000000009c08d72b/0xb3f83dad1b7e015a lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5e2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 78052 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000020:00200000:1.0:1787826821.795653:0:78052:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1787826821.797321:0:78052:0:(file.c:1775:ll_merge_attr_nolock()) [0x200015bc1:0x2:0x0] updating i_size 0 i_blocks 0 00000020:00010000:1.0:1787826821.797330:0:78052:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff9856054989e0 (ffff985610fe1638/0) at cl_lock_release():245 00000020:00010000:1.0:1787826821.797337:0:78052:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9856054989e0 (ffff985610fe1638/0) at cl_lock_cancel():142 00000020:00010000:1.0:1787826821.797341:0:78052:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff98560560c830 (ffff985610fe1e18/1) at cl_lock_cancel():142 00000020:00010000:1.0:1787826821.797345:0:78052:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9856054989e0 (ffff985610fe1638/0) at cl_lock_fini():72 00000020:00010000:1.0:1787826821.797348:0:78052:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff98560560c830 (ffff985610fe1638/1) at cl_lock_fini():72 00000080:00200000:1.0:1787826821.797361:0:78052:0:(vvp_io.c:304:vvp_io_fini()) [0x200015bc1:0x2:0x0] ignore/verify layout 0/1, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826821.797397:0:78052:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826821.797412:0:78052:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826821.797423:0:78052:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ae00/0xb3f83dad1b7e0153 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0fb7 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826821.797431:0:78052:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ae00/0xb3f83dad1b7e0153 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0fb7 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826821.797436:0:78052:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826821.797441:0:78052:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826821.797443:0:78052:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e0153 00000080:00010000:1.0:1787826821.797447:0:78052:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0153 from it ffffab3ac1da7c10 00010000:00010000:1.0:1787826821.797450:0:78052:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ae00/0xb3f83dad1b7e0153 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0fb7 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826821.797455:0:78052:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ae00/0xb3f83dad1b7e0153 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0fb7 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826821.797462:0:78052:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787826821.797469:0:78052:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f33a.replay-single--1, flags=0 00000080:00200000:1.0:1787826821.797473:0:78052:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826821.797476:0:78052:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826821.797479:0:78052:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ae00/0xb3f83dad1b7e0153 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0fb7 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826821.797484:0:78052:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ae00/0xb3f83dad1b7e0153 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0fb7 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826821.797488:0:78052:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826821.797490:0:78052:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826821.797491:0:78052:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e0153 00000080:00010000:1.0:1787826821.797493:0:78052:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0153 from it ffffab3ac1da7d68 00010000:00010000:1.0:1787826821.797494:0:78052:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ae00/0xb3f83dad1b7e0153 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0fb7 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826821.797498:0:78052:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ae00/0xb3f83dad1b7e0153 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0fb7 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787826821.797502:0:78052:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787826821.797506:0:78052:0:(namei.c:2418:ll_unlink()) VFS Op:name=f33a.replay-single--1, dir=[0x200000007:0x1:0x0](ffff98561158e3c8) 00010000:00010000:1.0:1787826821.797531:0:78052:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ae00/0xb3f83dad1b7e0153 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x5aa6d504d91a0fb7 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787826821.797559:0:78052:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07800/0xb3f83dad1b7e00ce lrc: 2/0,0 mode: PR/PR res: [0x200015bc1:0x2:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x5aa6d504d91a0de9 expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826821.797570:0:78052:0:(vvp_object.c:117:vvp_conf_set()) [0x200015bc1:0x2:0x0]: losing layout lock 00010000:00010000:1.0:1787826821.797600:0:78052:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07800/0xb3f83dad1b7e00ce lrc: 1/0,0 mode: --/PR res: [0x200015bc1:0x2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x5aa6d504d91a0de9 expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826821.797622:0:78052:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ae00/0xb3f83dad1b7e0153 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a0fb7 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787826821.797628:0:78052:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 2 locks packed 00010000:00010000:1.0:1787826821.797631:0:78052:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07800/0xb3f83dad1b7e00ce lrc: 0/0,0 mode: --/PR res: [0x200015bc1:0x2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x5aa6d504d91a0de9 expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826821.797651:0:78052:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ae00/0xb3f83dad1b7e0153 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a0fb7 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000020:00100000:1.0:1787826821.797670:0:78052:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787826821.797676:0:78052:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610e33480 x1874669225542144/t0(0) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826821.797729:0:78052:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e33480 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:78052:1874669225542144:192.168.203.114@tcp:36:rm.0 00000100:00100000:1.0:1787826821.797793:0:78052:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607e06d80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826821.799643:0:78052:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e33480 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:78052:1874669225542144:192.168.203.114@tcp:36:rm.0 00000080:00200000:1.0:1787826821.799725:0:78052:0:(vvp_io.c:1885:vvp_io_init()) [0x200015bc1:0x2:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826821.799740:0:78052:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826821.799772:0:78052:0:(vvp_io.c:304:vvp_io_fini()) [0x200015bc1:0x2:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826821.799781:0:78052:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200015bc1:0x2:0x0](ffff98561158f448) 00000008:00010000:1.0:1787826821.799815:0:78052:0:(osc_object.c:202:osc_object_ast_clear()) ### update lock size 0 blocks 0 [cma]time: 0 0 0 by oinfo size 0 blocks 0 [cma]time 0 0 0 ns: lustre-OST0000-osc-ffff985602bc9800 lock: 000000009c08d72b/0xb3f83dad1b7e015a lrc: 1/0,0 mode: PR/PR res: [0x280000401:0x5e2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x5aa6d504d91a0fbe expref: -99 pid: 78052 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826821.799901:0:78052:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826821.799918:0:78052:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826821.799984:0:78052:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e0161 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 78052 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.799993:0:78052:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e0161 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78052 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.800003:0:78052:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e0161 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78052 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826821.800022:0:78052:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e31a40 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:78052:1874669225542272:192.168.203.114@tcp:101:rm.0 00000100:00100000:1.0:1787826821.800087:0:78052:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607e06d80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826821.801088:0:78052:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e31a40 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:78052:1874669225542272:192.168.203.114@tcp:101:rm.0 00010000:00010000:1.0:1787826821.801107:0:78052:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e0161 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0fe8 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.801118:0:78052:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e0161 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0fe8 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.801143:0:78052:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e0161 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0fe8 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.801153:0:78052:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e0161 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0fe8 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.801164:0:78052:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e0161 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0fe8 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.801170:0:78052:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e0161 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0fe8 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.801175:0:78052:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e0161 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0fe8 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826821.801183:0:78052:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff985610e31a40 x1874669225542272/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826837 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826821.801195:0:78052:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e0161 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0fe8 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.801206:0:78052:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:1.0:1787826821.801212:0:78052:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:1.0:1787826821.801248:0:78052:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:1.0:1787826821.801256:0:78052:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e0161 00000080:00010000:1.0:1787826821.801261:0:78052:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826821.801264:0:78052:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0161 from it ffffab3ac1da7b58 00010000:00010000:1.0:1787826821.801266:0:78052:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e0161 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0fe8 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.801282:0:78052:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e0161 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0fe8 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826821.801310:0:78052:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787826821.801320:0:78052:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f33a.replay-single--2, flags=8 00000080:00200008:1.0:1787826821.801331:0:78052:0:(file.c:6313:ll_getattr_dentry()) START file f33a.replay-single--2:[0x200015bc1:0x3:0x0](ffff985611588908), request_mask 2047, flags 0, foreign 0 00000080:00200000:1.0:1787826821.801340:0:78052:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200015bc1:0x3:0x0](ffff985611588908),name=f33a.replay-single--2 00000002:00010000:1.0:1787826821.801356:0:78052:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200015bc1:0x3:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826821.801365:0:78052:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06c00/0xb3f83dad1b7e00dc lrc: 2/1,0 mode: PR/PR res: [0x200015bc1:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a0e1a expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826821.801375:0:78052:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06c00/0xb3f83dad1b7e00dc lrc: 2/1,0 mode: PR/PR res: [0x200015bc1:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a0e1a expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826821.801381:0:78052:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826821.801385:0:78052:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200015bc1:0x3:0x0](ffff985611588908) 00000080:00010000:1.0:1787826821.801388:0:78052:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200015bc1:0x3:0x0](ffff985611588908) for lock 0xb3f83dad1b7e00dc 00000080:00010000:1.0:1787826821.801392:0:78052:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e00dc from it ffffab3ac1da7cb8 00010000:00010000:1.0:1787826821.801395:0:78052:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06c00/0xb3f83dad1b7e00dc lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a0e1a expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826821.801411:0:78052:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06c00/0xb3f83dad1b7e00dc lrc: 2/0,0 mode: PR/PR res: [0x200015bc1:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a0e1a expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826821.801431:0:78052:0:(vvp_io.c:1885:vvp_io_init()) [0x200015bc1:0x3:0x0] ignore/verify layout 0/1, layout version 0 restore needed 0 00000001:00010000:1.0:1787826821.801440:0:78052:0:(glimpse.c:73:cl_glimpse_lock()) Glimpsing inode [0x200015bc1:0x3:0x0] 00000008:00010000:1.0:1787826821.801458:0:78052:0:(lustre_osc.h:166:osc_enq2ldlm_flags()) flags: a 00000008:00010000:1.0:1787826821.801461:0:78052:0:(osc_lock.c:1320:osc_lock_init()) ### lock ffff98560560c830, osc lock ffff985602f8efd8, flags 0x400001002 00010000:00010000:1.0:1787826821.801499:0:78052:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0001-osc-ffff985602bc9800 lock: 0000000077e2eada/0xb3f83dad1b7e0168 lrc: 3/1,0 mode: --/PR res: [0x2c0000401:0x5e3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 78052 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.801509:0:78052:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x400001000 ns: lustre-OST0001-osc-ffff985602bc9800 lock: 0000000077e2eada/0xb3f83dad1b7e0168 lrc: 3/1,0 mode: --/PR res: [0x2c0000401:0x5e3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 78052 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000020:00200000:1.0:1787826821.801549:0:78052:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000100:00100000:1.0:1787826821.801561:0:2384:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1787826821.801589:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e31a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669225542400:192.168.203.114@tcp:101:rm.0 00010000:00010000:1.0:1787826821.802913:0:2384:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0001-osc-ffff985602bc9800 lock: 0000000077e2eada/0xb3f83dad1b7e0168 lrc: 6/2,0 mode: --/PR res: [0x2c0000401:0x5e3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 78052 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.802947:0:2384:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-OST0001-osc-ffff985602bc9800 lock: 0000000077e2eada/0xb3f83dad1b7e0168 lrc: 6/2,0 mode: --/PR res: [0x2c0000401:0x5e3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0fef expref: -99 pid: 78052 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.802962:0:2384:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0001-osc-ffff985602bc9800 lock: 0000000077e2eada/0xb3f83dad1b7e0168 lrc: 6/2,0 mode: PR/PR res: [0x2c0000401:0x5e3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a0fef expref: -99 pid: 78052 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.802971:0:2384:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-OST0001-osc-ffff985602bc9800 lock: 0000000077e2eada/0xb3f83dad1b7e0168 lrc: 6/2,0 mode: PR/PR res: [0x2c0000401:0x5e3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a0fef expref: -99 pid: 78052 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787826821.802986:0:2384:0:(osc_lock.c:192:osc_lock_lvb_update()) ### acquired size 0, setting rss=0; kms=0, end=18446744073709551615 ns: lustre-OST0001-osc-ffff985602bc9800 lock: 0000000077e2eada/0xb3f83dad1b7e0168 lrc: 5/2,0 mode: PR/PR res: [0x2c0000401:0x5e3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a0fef expref: -99 pid: 78052 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000020:00200000:1.0:1787826821.802998:0:2384:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00010000:00010000:1.0:1787826821.803022:0:2384:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0001-osc-ffff985602bc9800 lock: 0000000077e2eada/0xb3f83dad1b7e0168 lrc: 6/2,0 mode: PR/PR res: [0x2c0000401:0x5e3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x5aa6d504d91a0fef expref: -99 pid: 78052 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.803037:0:2384:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-OST0001-osc-ffff985602bc9800 lock: 0000000077e2eada/0xb3f83dad1b7e0168 lrc: 5/1,0 mode: PR/PR res: [0x2c0000401:0x5e3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x5aa6d504d91a0fef expref: -99 pid: 78052 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.803044:0:2384:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0001-osc-ffff985602bc9800 lock: 0000000077e2eada/0xb3f83dad1b7e0168 lrc: 5/1,0 mode: PR/PR res: [0x2c0000401:0x5e3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x5aa6d504d91a0fef expref: -99 pid: 78052 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.803052:0:2384:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0001-osc-ffff985602bc9800 lock: 0000000077e2eada/0xb3f83dad1b7e0168 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x5e3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x5aa6d504d91a0fef expref: -99 pid: 78052 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000100:00100000:1.0:1787826821.803086:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e31a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669225542400:192.168.203.114@tcp:101:rm.0 00000080:00200000:1.0:1787826821.808478:0:78052:0:(file.c:1775:ll_merge_attr_nolock()) [0x200015bc1:0x4:0x0] updating i_size 0 i_blocks 0 00000020:00010000:1.0:1787826821.808484:0:78052:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff9856054989e0 (ffff985610fe1638/0) at cl_lock_release():245 00000020:00010000:1.0:1787826821.808487:0:78052:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9856054989e0 (ffff985610fe1638/0) at cl_lock_cancel():142 00000020:00010000:1.0:1787826821.808490:0:78052:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff985612a3cfb0 (ffff985610fe1e18/1) at cl_lock_cancel():142 00000020:00010000:1.0:1787826821.808494:0:78052:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9856054989e0 (ffff985610fe1638/0) at cl_lock_fini():72 00000020:00010000:1.0:1787826821.808497:0:78052:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff985612a3cfb0 (ffff985610fe1638/1) at cl_lock_fini():72 00000080:00200000:1.0:1787826821.808507:0:78052:0:(vvp_io.c:304:vvp_io_fini()) [0x200015bc1:0x4:0x0] ignore/verify layout 0/1, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826821.808534:0:78052:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826821.808545:0:78052:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826821.808554:0:78052:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1200/0xb3f83dad1b7e016f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1019 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826821.808564:0:78052:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1200/0xb3f83dad1b7e016f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1019 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826821.808576:0:78052:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826821.808581:0:78052:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826821.808583:0:78052:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e016f 00000080:00010000:1.0:1787826821.808588:0:78052:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e016f from it ffffab3ac1da7c10 00010000:00010000:1.0:1787826821.808591:0:78052:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1200/0xb3f83dad1b7e016f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1019 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826821.808598:0:78052:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1200/0xb3f83dad1b7e016f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1019 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826821.808607:0:78052:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787826821.808614:0:78052:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f33a.replay-single--3, flags=0 00000080:00200000:1.0:1787826821.808618:0:78052:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826821.808625:0:78052:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826821.808631:0:78052:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1200/0xb3f83dad1b7e016f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1019 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826821.808639:0:78052:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1200/0xb3f83dad1b7e016f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1019 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826821.808645:0:78052:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826821.808649:0:78052:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826821.808651:0:78052:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e016f 00000080:00010000:1.0:1787826821.808654:0:78052:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e016f from it ffffab3ac1da7d68 00010000:00010000:1.0:1787826821.808657:0:78052:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1200/0xb3f83dad1b7e016f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1019 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826821.808669:0:78052:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1200/0xb3f83dad1b7e016f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1019 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787826821.808676:0:78052:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787826821.808682:0:78052:0:(namei.c:2418:ll_unlink()) VFS Op:name=f33a.replay-single--3, dir=[0x200000007:0x1:0x0](ffff98561158e3c8) 00010000:00010000:1.0:1787826821.808771:0:78052:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1200/0xb3f83dad1b7e016f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x5aa6d504d91a1019 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787826821.808795:0:78052:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07200/0xb3f83dad1b7e00ea lrc: 2/0,0 mode: PR/PR res: [0x200015bc1:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x5aa6d504d91a0e4b expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826821.808806:0:78052:0:(vvp_object.c:117:vvp_conf_set()) [0x200015bc1:0x4:0x0]: losing layout lock 00010000:00010000:1.0:1787826821.808829:0:78052:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07200/0xb3f83dad1b7e00ea lrc: 1/0,0 mode: --/PR res: [0x200015bc1:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x5aa6d504d91a0e4b expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826821.808836:0:78052:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1200/0xb3f83dad1b7e016f lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a1019 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787826821.808842:0:78052:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 2 locks packed 00010000:00010000:1.0:1787826821.808845:0:78052:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07200/0xb3f83dad1b7e00ea lrc: 0/0,0 mode: --/PR res: [0x200015bc1:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x5aa6d504d91a0e4b expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826821.808860:0:78052:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1200/0xb3f83dad1b7e016f lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a1019 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000020:00100000:1.0:1787826821.808875:0:78052:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787826821.808882:0:78052:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610e31a40 x1874669225543168/t0(0) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826821.808901:0:78052:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e31a40 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:78052:1874669225543168:192.168.203.114@tcp:36:rm.0 00000100:00100000:1.0:1787826821.808946:0:78052:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607e06d80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826821.810545:0:78052:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e31a40 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:78052:1874669225543168:192.168.203.114@tcp:36:rm.0 00000080:00200000:1.0:1787826821.810581:0:78052:0:(vvp_io.c:1885:vvp_io_init()) [0x200015bc1:0x4:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826821.810591:0:78052:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826821.810614:0:78052:0:(vvp_io.c:304:vvp_io_fini()) [0x200015bc1:0x4:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826821.810622:0:78052:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200015bc1:0x4:0x0](ffff985611589148) 00000008:00010000:1.0:1787826821.810646:0:78052:0:(osc_object.c:202:osc_object_ast_clear()) ### update lock size 0 blocks 0 [cma]time: 0 0 0 by oinfo size 0 blocks 0 [cma]time 0 0 0 ns: lustre-OST0000-osc-ffff985602bc9800 lock: 0000000089b1b66a/0xb3f83dad1b7e0176 lrc: 1/0,0 mode: PR/PR res: [0x280000401:0x5e3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x5aa6d504d91a1020 expref: -99 pid: 78052 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826821.810727:0:78052:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826821.810739:0:78052:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826821.810777:0:78052:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ba00/0xb3f83dad1b7e017d lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 78052 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.810783:0:78052:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ba00/0xb3f83dad1b7e017d lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78052 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.810789:0:78052:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ba00/0xb3f83dad1b7e017d lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78052 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826821.810799:0:78052:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e330c0 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:78052:1874669225543296:192.168.203.114@tcp:101:rm.0 00000100:00100000:1.0:1787826821.810839:0:78052:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607e06d80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826821.811866:0:78052:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e330c0 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:78052:1874669225543296:192.168.203.114@tcp:101:rm.0 00010000:00010000:1.0:1787826821.811878:0:78052:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ba00/0xb3f83dad1b7e017d lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a104a expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.811888:0:78052:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ba00/0xb3f83dad1b7e017d lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a104a expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.811906:0:78052:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ba00/0xb3f83dad1b7e017d lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a104a expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.811917:0:78052:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ba00/0xb3f83dad1b7e017d lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a104a expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.811926:0:78052:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ba00/0xb3f83dad1b7e017d lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a104a expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.811934:0:78052:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ba00/0xb3f83dad1b7e017d lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a104a expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.811941:0:78052:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ba00/0xb3f83dad1b7e017d lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a104a expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826821.811951:0:78052:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff985610e330c0 x1874669225543296/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826837 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826821.811964:0:78052:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ba00/0xb3f83dad1b7e017d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a104a expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.811973:0:78052:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:1.0:1787826821.811978:0:78052:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:1.0:1787826821.811998:0:78052:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:1.0:1787826821.812005:0:78052:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e017d 00000080:00010000:1.0:1787826821.812009:0:78052:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826821.812012:0:78052:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e017d from it ffffab3ac1da7b58 00010000:00010000:1.0:1787826821.812015:0:78052:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ba00/0xb3f83dad1b7e017d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a104a expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.812023:0:78052:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ba00/0xb3f83dad1b7e017d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a104a expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826821.812040:0:78052:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787826821.812048:0:78052:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f33a.replay-single--4, flags=8 00000080:00200008:1.0:1787826821.812056:0:78052:0:(file.c:6313:ll_getattr_dentry()) START file f33a.replay-single--4:[0x200015bc1:0x5:0x0](ffff98561158db88), request_mask 2047, flags 0, foreign 0 00000080:00200000:1.0:1787826821.812064:0:78052:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200015bc1:0x5:0x0](ffff98561158db88),name=f33a.replay-single--4 00000002:00010000:1.0:1787826821.812076:0:78052:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200015bc1:0x5:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826821.812085:0:78052:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07400/0xb3f83dad1b7e00f8 lrc: 2/1,0 mode: PR/PR res: [0x200015bc1:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a0e7c expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826821.812104:0:78052:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07400/0xb3f83dad1b7e00f8 lrc: 2/1,0 mode: PR/PR res: [0x200015bc1:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a0e7c expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826821.812112:0:78052:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826821.812116:0:78052:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200015bc1:0x5:0x0](ffff98561158db88) 00000080:00010000:1.0:1787826821.812119:0:78052:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200015bc1:0x5:0x0](ffff98561158db88) for lock 0xb3f83dad1b7e00f8 00000080:00010000:1.0:1787826821.812124:0:78052:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e00f8 from it ffffab3ac1da7cb8 00010000:00010000:1.0:1787826821.812126:0:78052:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07400/0xb3f83dad1b7e00f8 lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a0e7c expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826821.812134:0:78052:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07400/0xb3f83dad1b7e00f8 lrc: 2/0,0 mode: PR/PR res: [0x200015bc1:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a0e7c expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826821.812151:0:78052:0:(vvp_io.c:1885:vvp_io_init()) [0x200015bc1:0x5:0x0] ignore/verify layout 0/1, layout version 0 restore needed 0 00000001:00010000:1.0:1787826821.812158:0:78052:0:(glimpse.c:73:cl_glimpse_lock()) Glimpsing inode [0x200015bc1:0x5:0x0] 00000008:00010000:1.0:1787826821.812178:0:78052:0:(lustre_osc.h:166:osc_enq2ldlm_flags()) flags: a 00000008:00010000:1.0:1787826821.812181:0:78052:0:(osc_lock.c:1320:osc_lock_init()) ### lock ffff98560560ca30, osc lock ffff985602f8eaf8, flags 0x400001002 00010000:00010000:1.0:1787826821.812210:0:78052:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0001-osc-ffff985602bc9800 lock: 00000000e6dcfdb4/0xb3f83dad1b7e0184 lrc: 3/1,0 mode: --/PR res: [0x2c0000401:0x5e4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 78052 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.812218:0:78052:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x400001000 ns: lustre-OST0001-osc-ffff985602bc9800 lock: 00000000e6dcfdb4/0xb3f83dad1b7e0184 lrc: 3/1,0 mode: --/PR res: [0x2c0000401:0x5e4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 78052 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000020:00200000:1.0:1787826821.812263:0:78052:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000100:00100000:1.0:1787826821.812273:0:2384:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1787826821.812287:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e330c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669225543424:192.168.203.114@tcp:101:rm.0 00010000:00010000:1.0:1787826821.813427:0:2384:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0001-osc-ffff985602bc9800 lock: 00000000e6dcfdb4/0xb3f83dad1b7e0184 lrc: 6/2,0 mode: --/PR res: [0x2c0000401:0x5e4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 78052 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.813439:0:2384:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-OST0001-osc-ffff985602bc9800 lock: 00000000e6dcfdb4/0xb3f83dad1b7e0184 lrc: 6/2,0 mode: --/PR res: [0x2c0000401:0x5e4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1051 expref: -99 pid: 78052 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.813448:0:2384:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0001-osc-ffff985602bc9800 lock: 00000000e6dcfdb4/0xb3f83dad1b7e0184 lrc: 6/2,0 mode: PR/PR res: [0x2c0000401:0x5e4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1051 expref: -99 pid: 78052 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.813455:0:2384:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-OST0001-osc-ffff985602bc9800 lock: 00000000e6dcfdb4/0xb3f83dad1b7e0184 lrc: 6/2,0 mode: PR/PR res: [0x2c0000401:0x5e4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1051 expref: -99 pid: 78052 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787826821.813464:0:2384:0:(osc_lock.c:192:osc_lock_lvb_update()) ### acquired size 0, setting rss=0; kms=0, end=18446744073709551615 ns: lustre-OST0001-osc-ffff985602bc9800 lock: 00000000e6dcfdb4/0xb3f83dad1b7e0184 lrc: 5/2,0 mode: PR/PR res: [0x2c0000401:0x5e4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a1051 expref: -99 pid: 78052 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000020:00200000:1.0:1787826821.813471:0:2384:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00010000:00010000:1.0:1787826821.813483:0:2384:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0001-osc-ffff985602bc9800 lock: 00000000e6dcfdb4/0xb3f83dad1b7e0184 lrc: 6/2,0 mode: PR/PR res: [0x2c0000401:0x5e4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x5aa6d504d91a1051 expref: -99 pid: 78052 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.813491:0:2384:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-OST0001-osc-ffff985602bc9800 lock: 00000000e6dcfdb4/0xb3f83dad1b7e0184 lrc: 5/1,0 mode: PR/PR res: [0x2c0000401:0x5e4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x5aa6d504d91a1051 expref: -99 pid: 78052 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.813499:0:2384:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0001-osc-ffff985602bc9800 lock: 00000000e6dcfdb4/0xb3f83dad1b7e0184 lrc: 5/1,0 mode: PR/PR res: [0x2c0000401:0x5e4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x5aa6d504d91a1051 expref: -99 pid: 78052 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.813506:0:2384:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0001-osc-ffff985602bc9800 lock: 00000000e6dcfdb4/0xb3f83dad1b7e0184 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x5e4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x5aa6d504d91a1051 expref: -99 pid: 78052 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000100:00100000:1.0:1787826821.813519:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e330c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669225543424:192.168.203.114@tcp:101:rm.0 00000100:00100000:1.0:1787826821.815929:0:78052:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e7b840 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:78052:1874669225543680:192.168.203.114@tcp:36:rm.0 00000080:00200000:1.0:1787826821.815970:0:78052:0:(vvp_io.c:1885:vvp_io_init()) [0x200015bc1:0x5:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826821.815978:0:78052:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826821.816016:0:78052:0:(vvp_io.c:304:vvp_io_fini()) [0x200015bc1:0x5:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826821.816023:0:78052:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200015bc1:0x5:0x0](ffff98561158db88) 00000008:00010000:1.0:1787826821.816052:0:78052:0:(osc_object.c:202:osc_object_ast_clear()) ### update lock size 0 blocks 0 [cma]time: 0 0 0 by oinfo size 0 blocks 0 [cma]time 0 0 0 ns: lustre-OST0001-osc-ffff985602bc9800 lock: 00000000e6dcfdb4/0xb3f83dad1b7e0184 lrc: 1/0,0 mode: PR/PR res: [0x2c0000401:0x5e4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x5aa6d504d91a1051 expref: -99 pid: 78052 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826821.816116:0:78052:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826821.816184:0:78052:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826821.816222:0:78052:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e018b lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 78052 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.816243:0:78052:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e018b lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78052 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.816252:0:78052:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e018b lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78052 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826821.816267:0:78052:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e330c0 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:78052:1874669225543808:192.168.203.114@tcp:101:rm.0 00000100:00100000:1.0:1787826821.816320:0:78052:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607e06d80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826821.817496:0:78052:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e330c0 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:78052:1874669225543808:192.168.203.114@tcp:101:rm.0 00010000:00010000:1.0:1787826821.817509:0:78052:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e018b lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a107b expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.817516:0:78052:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e018b lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a107b expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.817524:0:78052:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e018b lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a107b expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.817532:0:78052:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e018b lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a107b expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.817542:0:78052:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e018b lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a107b expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.817550:0:78052:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e018b lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a107b expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.817559:0:78052:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e018b lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a107b expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826821.817583:0:78052:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff985610e330c0 x1874669225543808/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826837 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826821.817595:0:78052:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e018b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a107b expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.817602:0:78052:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:1.0:1787826821.817606:0:78052:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:1.0:1787826821.817623:0:78052:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:1.0:1787826821.817635:0:78052:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e018b 00000080:00010000:1.0:1787826821.817639:0:78052:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826821.817642:0:78052:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e018b from it ffffab3ac1da7b58 00010000:00010000:1.0:1787826821.817645:0:78052:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e018b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a107b expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.817658:0:78052:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e018b lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a107b expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826821.817676:0:78052:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787826821.817683:0:78052:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f33a.replay-single--5, flags=8 00000080:00200008:1.0:1787826821.817719:0:78052:0:(file.c:6313:ll_getattr_dentry()) START file f33a.replay-single--5:[0x200015bc1:0x6:0x0](ffff98561158d348), request_mask 2047, flags 0, foreign 0 00000080:00200000:1.0:1787826821.817727:0:78052:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200015bc1:0x6:0x0](ffff98561158d348),name=f33a.replay-single--5 00000002:00010000:1.0:1787826821.817743:0:78052:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200015bc1:0x6:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826821.817752:0:78052:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07a00/0xb3f83dad1b7e0106 lrc: 2/1,0 mode: PR/PR res: [0x200015bc1:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a0ead expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826821.817774:0:78052:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07a00/0xb3f83dad1b7e0106 lrc: 2/1,0 mode: PR/PR res: [0x200015bc1:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a0ead expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826821.817781:0:78052:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826821.817786:0:78052:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200015bc1:0x6:0x0](ffff98561158d348) 00000080:00010000:1.0:1787826821.817789:0:78052:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200015bc1:0x6:0x0](ffff98561158d348) for lock 0xb3f83dad1b7e0106 00000080:00010000:1.0:1787826821.817793:0:78052:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0106 from it ffffab3ac1da7cb8 00010000:00010000:1.0:1787826821.817796:0:78052:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07a00/0xb3f83dad1b7e0106 lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a0ead expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826821.817804:0:78052:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07a00/0xb3f83dad1b7e0106 lrc: 2/0,0 mode: PR/PR res: [0x200015bc1:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a0ead expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826821.817819:0:78052:0:(vvp_io.c:1885:vvp_io_init()) [0x200015bc1:0x6:0x0] ignore/verify layout 0/1, layout version 0 restore needed 0 00000001:00010000:1.0:1787826821.817827:0:78052:0:(glimpse.c:73:cl_glimpse_lock()) Glimpsing inode [0x200015bc1:0x6:0x0] 00000008:00010000:1.0:1787826821.817847:0:78052:0:(lustre_osc.h:166:osc_enq2ldlm_flags()) flags: a 00000008:00010000:1.0:1787826821.817851:0:78052:0:(osc_lock.c:1320:osc_lock_init()) ### lock ffff98560560c730, osc lock ffff985602f8f110, flags 0x400001002 00010000:00010000:1.0:1787826821.817880:0:78052:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0000-osc-ffff985602bc9800 lock: 00000000c9e21032/0xb3f83dad1b7e0192 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5e4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 78052 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.817890:0:78052:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x400001000 ns: lustre-OST0000-osc-ffff985602bc9800 lock: 00000000c9e21032/0xb3f83dad1b7e0192 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5e4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 78052 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000020:00200000:1.0:1787826821.817924:0:78052:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1787826821.819359:0:78052:0:(file.c:1775:ll_merge_attr_nolock()) [0x200015bc1:0x6:0x0] updating i_size 0 i_blocks 0 00000020:00010000:1.0:1787826821.819366:0:78052:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff985605499b28 (ffff985610fe1bd8/0) at cl_lock_release():245 00000020:00010000:1.0:1787826821.819371:0:78052:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff985605499b28 (ffff985610fe1bd8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1787826821.819379:0:78052:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff98560560c730 (ffff985610fe1998/1) at cl_lock_cancel():142 00000020:00010000:1.0:1787826821.819386:0:78052:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff985605499b28 (ffff985610fe1bd8/0) at cl_lock_fini():72 00000020:00010000:1.0:1787826821.819389:0:78052:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff98560560c730 (ffff985610fe1bd8/1) at cl_lock_fini():72 00000080:00200000:1.0:1787826821.819400:0:78052:0:(vvp_io.c:304:vvp_io_fini()) [0x200015bc1:0x6:0x0] ignore/verify layout 0/1, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826821.819429:0:78052:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826821.819441:0:78052:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826821.819450:0:78052:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e018b lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a107b expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826821.819460:0:78052:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e018b lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a107b expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826821.819467:0:78052:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826821.819472:0:78052:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826821.819474:0:78052:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e018b 00000080:00010000:1.0:1787826821.819479:0:78052:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e018b from it ffffab3ac1da7c10 00010000:00010000:1.0:1787826821.819482:0:78052:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e018b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a107b expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826821.819489:0:78052:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e018b lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a107b expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826821.819498:0:78052:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787826821.819504:0:78052:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f33a.replay-single--5, flags=0 00000080:00200000:1.0:1787826821.819508:0:78052:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826821.819514:0:78052:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826821.819519:0:78052:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e018b lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a107b expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826821.819532:0:78052:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e018b lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a107b expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826821.819538:0:78052:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826821.819541:0:78052:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826821.819544:0:78052:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e018b 00000080:00010000:1.0:1787826821.819547:0:78052:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e018b from it ffffab3ac1da7d68 00010000:00010000:1.0:1787826821.819550:0:78052:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e018b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a107b expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826821.819557:0:78052:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e018b lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a107b expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787826821.819564:0:78052:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787826821.819569:0:78052:0:(namei.c:2418:ll_unlink()) VFS Op:name=f33a.replay-single--5, dir=[0x200000007:0x1:0x0](ffff98561158e3c8) 00010000:00010000:1.0:1787826821.819589:0:78052:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e018b lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x5aa6d504d91a107b expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787826821.819609:0:78052:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07a00/0xb3f83dad1b7e0106 lrc: 2/0,0 mode: PR/PR res: [0x200015bc1:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x5aa6d504d91a0ead expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826821.819619:0:78052:0:(vvp_object.c:117:vvp_conf_set()) [0x200015bc1:0x6:0x0]: losing layout lock 00010000:00010000:1.0:1787826821.819641:0:78052:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07a00/0xb3f83dad1b7e0106 lrc: 1/0,0 mode: --/PR res: [0x200015bc1:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x5aa6d504d91a0ead expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826821.819648:0:78052:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e018b lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a107b expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787826821.819659:0:78052:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 2 locks packed 00010000:00010000:1.0:1787826821.819662:0:78052:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07a00/0xb3f83dad1b7e0106 lrc: 0/0,0 mode: --/PR res: [0x200015bc1:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x5aa6d504d91a0ead expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826821.819678:0:78052:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e018b lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a107b expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000020:00100000:1.0:1787826821.819723:0:78052:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787826821.819731:0:78052:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610e30780 x1874669225544192/t0(0) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826821.819749:0:78052:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e30780 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:78052:1874669225544192:192.168.203.114@tcp:36:rm.0 00000100:00100000:1.0:1787826821.819794:0:78052:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607e06d80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826821.821970:0:78052:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e30780 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:78052:1874669225544192:192.168.203.114@tcp:36:rm.0 00000080:00200000:1.0:1787826821.822016:0:78052:0:(vvp_io.c:1885:vvp_io_init()) [0x200015bc1:0x6:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826821.822028:0:78052:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826821.822051:0:78052:0:(vvp_io.c:304:vvp_io_fini()) [0x200015bc1:0x6:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826821.822057:0:78052:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200015bc1:0x6:0x0](ffff98561158d348) 00000008:00010000:1.0:1787826821.822090:0:78052:0:(osc_object.c:202:osc_object_ast_clear()) ### update lock size 0 blocks 0 [cma]time: 0 0 0 by oinfo size 0 blocks 0 [cma]time 0 0 0 ns: lustre-OST0000-osc-ffff985602bc9800 lock: 00000000c9e21032/0xb3f83dad1b7e0192 lrc: 1/0,0 mode: PR/PR res: [0x280000401:0x5e4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x5aa6d504d91a1082 expref: -99 pid: 78052 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826821.822151:0:78052:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826821.822165:0:78052:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826821.822202:0:78052:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a000/0xb3f83dad1b7e0199 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 78052 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.822216:0:78052:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a000/0xb3f83dad1b7e0199 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78052 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.822235:0:78052:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a000/0xb3f83dad1b7e0199 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78052 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826821.822251:0:78052:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e30000 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:78052:1874669225544320:192.168.203.114@tcp:101:rm.0 00000100:00100000:1.0:1787826821.822302:0:78052:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607e06d80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826821.823757:0:78052:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e30000 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:78052:1874669225544320:192.168.203.114@tcp:101:rm.0 00010000:00010000:1.0:1787826821.823773:0:78052:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a000/0xb3f83dad1b7e0199 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a10ac expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.823784:0:78052:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a000/0xb3f83dad1b7e0199 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a10ac expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.823795:0:78052:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a000/0xb3f83dad1b7e0199 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a10ac expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.823814:0:78052:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a000/0xb3f83dad1b7e0199 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a10ac expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.823821:0:78052:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a000/0xb3f83dad1b7e0199 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a10ac expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.823826:0:78052:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a000/0xb3f83dad1b7e0199 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a10ac expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.823834:0:78052:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a000/0xb3f83dad1b7e0199 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a10ac expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826821.823841:0:78052:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff985610e30000 x1874669225544320/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826837 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826821.823850:0:78052:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a000/0xb3f83dad1b7e0199 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a10ac expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.823856:0:78052:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:1.0:1787826821.823860:0:78052:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:1.0:1787826821.823877:0:78052:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:1.0:1787826821.823881:0:78052:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e0199 00000080:00010000:1.0:1787826821.823884:0:78052:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826821.823886:0:78052:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0199 from it ffffab3ac1da7b58 00010000:00010000:1.0:1787826821.823888:0:78052:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a000/0xb3f83dad1b7e0199 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a10ac expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.823894:0:78052:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a000/0xb3f83dad1b7e0199 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a10ac expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826821.823907:0:78052:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787826821.823916:0:78052:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f33a.replay-single--6, flags=8 00000080:00200008:1.0:1787826821.823925:0:78052:0:(file.c:6313:ll_getattr_dentry()) START file f33a.replay-single--6:[0x200015bc1:0x7:0x0](ffff98561158a1c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:1.0:1787826821.824249:0:78052:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200015bc1:0x7:0x0](ffff98561158a1c8),name=f33a.replay-single--6 00000002:00010000:1.0:1787826821.824262:0:78052:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200015bc1:0x7:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826821.824271:0:78052:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e0114 lrc: 2/1,0 mode: PR/PR res: [0x200015bc1:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a0ede expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826821.824282:0:78052:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e0114 lrc: 2/1,0 mode: PR/PR res: [0x200015bc1:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a0ede expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826821.824288:0:78052:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826821.824293:0:78052:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200015bc1:0x7:0x0](ffff98561158a1c8) 00000080:00010000:1.0:1787826821.824296:0:78052:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200015bc1:0x7:0x0](ffff98561158a1c8) for lock 0xb3f83dad1b7e0114 00000080:00010000:1.0:1787826821.824300:0:78052:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0114 from it ffffab3ac1da7cb8 00010000:00010000:1.0:1787826821.824303:0:78052:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e0114 lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a0ede expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826821.824310:0:78052:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e0114 lrc: 2/0,0 mode: PR/PR res: [0x200015bc1:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a0ede expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826821.824327:0:78052:0:(vvp_io.c:1885:vvp_io_init()) [0x200015bc1:0x7:0x0] ignore/verify layout 0/1, layout version 0 restore needed 0 00000001:00010000:1.0:1787826821.824335:0:78052:0:(glimpse.c:73:cl_glimpse_lock()) Glimpsing inode [0x200015bc1:0x7:0x0] 00000008:00010000:1.0:1787826821.824351:0:78052:0:(lustre_osc.h:166:osc_enq2ldlm_flags()) flags: a 00000008:00010000:1.0:1787826821.824355:0:78052:0:(osc_lock.c:1320:osc_lock_init()) ### lock ffff98560560c730, osc lock ffff985602f8f110, flags 0x400001002 00010000:00010000:1.0:1787826821.824389:0:78052:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0001-osc-ffff985602bc9800 lock: 00000000af8864da/0xb3f83dad1b7e01a0 lrc: 3/1,0 mode: --/PR res: [0x2c0000401:0x5e5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 78052 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.824399:0:78052:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x400001000 ns: lustre-OST0001-osc-ffff985602bc9800 lock: 00000000af8864da/0xb3f83dad1b7e01a0 lrc: 3/1,0 mode: --/PR res: [0x2c0000401:0x5e5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 78052 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000020:00200000:1.0:1787826821.824428:0:78052:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000100:00100000:3.0:1787826821.824999:0:2385:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1787826821.825008:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669225544704:192.168.203.114@tcp:101:ll_sa.0 00000100:00100000:3.0:1787826821.825094:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee2d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669225544832:192.168.203.114@tcp:101:ll_sa.0 00010000:00010000:3.0:1787826821.827398:0:2385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0001-osc-ffff985602bc9800 lock: 0000000005f20dd6/0xb3f83dad1b7e01d1 lrc: 6/2,0 mode: --/PR res: [0x2c0000401:0x5e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 78053 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.827425:0:2385:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-OST0001-osc-ffff985602bc9800 lock: 0000000005f20dd6/0xb3f83dad1b7e01d1 lrc: 6/2,0 mode: --/PR res: [0x2c0000401:0x5e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a10d6 expref: -99 pid: 78053 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.827443:0:2385:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0001-osc-ffff985602bc9800 lock: 0000000005f20dd6/0xb3f83dad1b7e01d1 lrc: 6/2,0 mode: PR/PR res: [0x2c0000401:0x5e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a10d6 expref: -99 pid: 78053 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.827467:0:2385:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-OST0001-osc-ffff985602bc9800 lock: 0000000005f20dd6/0xb3f83dad1b7e01d1 lrc: 6/2,0 mode: PR/PR res: [0x2c0000401:0x5e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a10d6 expref: -99 pid: 78053 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:3.0:1787826821.827489:0:2385:0:(osc_lock.c:192:osc_lock_lvb_update()) ### acquired size 0, setting rss=0; kms=0, end=18446744073709551615 ns: lustre-OST0001-osc-ffff985602bc9800 lock: 0000000005f20dd6/0xb3f83dad1b7e01d1 lrc: 5/2,0 mode: PR/PR res: [0x2c0000401:0x5e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a10d6 expref: -99 pid: 78053 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.827502:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0001-osc-ffff985602bc9800 lock: 0000000005f20dd6/0xb3f83dad1b7e01d1 lrc: 5/2,0 mode: PR/PR res: [0x2c0000401:0x5e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a10d6 expref: -99 pid: 78053 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.827511:0:2385:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-OST0001-osc-ffff985602bc9800 lock: 0000000005f20dd6/0xb3f83dad1b7e01d1 lrc: 4/1,0 mode: PR/PR res: [0x2c0000401:0x5e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a10d6 expref: -99 pid: 78053 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.827518:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0001-osc-ffff985602bc9800 lock: 0000000005f20dd6/0xb3f83dad1b7e01d1 lrc: 4/1,0 mode: PR/PR res: [0x2c0000401:0x5e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a10d6 expref: -99 pid: 78053 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.827525:0:2385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0001-osc-ffff985602bc9800 lock: 0000000005f20dd6/0xb3f83dad1b7e01d1 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x5e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a10d6 expref: -99 pid: 78053 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000100:00100000:3.0:1787826821.827538:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee2d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669225544832:192.168.203.114@tcp:101:ll_sa.0 00000080:00200000:1.0:1787826821.827606:0:78052:0:(file.c:1775:ll_merge_attr_nolock()) [0x200015bc1:0x7:0x0] updating i_size 0 i_blocks 0 00000020:00010000:1.0:1787826821.827614:0:78052:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff985605499b28 (ffff985610fe1bd8/0) at cl_lock_release():245 00000020:00010000:1.0:1787826821.827621:0:78052:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff985605499b28 (ffff985610fe1bd8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1787826821.827629:0:78052:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff98560560c730 (ffff985610fe1998/1) at cl_lock_cancel():142 00000020:00010000:1.0:1787826821.827637:0:78052:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff985605499b28 (ffff985610fe1bd8/0) at cl_lock_fini():72 00000020:00010000:1.0:1787826821.827641:0:78052:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff98560560c730 (ffff985610fe1bd8/1) at cl_lock_fini():72 00000080:00200000:1.0:1787826821.827655:0:78052:0:(vvp_io.c:304:vvp_io_fini()) [0x200015bc1:0x7:0x0] ignore/verify layout 0/1, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826821.827727:0:78052:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826821.827746:0:78052:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826821.827759:0:78052:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a000/0xb3f83dad1b7e0199 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a10ac expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826821.827770:0:78052:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a000/0xb3f83dad1b7e0199 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a10ac expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826821.827778:0:78052:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826821.827784:0:78052:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826821.827787:0:78052:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e0199 00000080:00010000:1.0:1787826821.827793:0:78052:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0199 from it ffffab3ac1da7c10 00010000:00010000:1.0:1787826821.827796:0:78052:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a000/0xb3f83dad1b7e0199 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a10ac expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826821.827804:0:78052:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a000/0xb3f83dad1b7e0199 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a10ac expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826821.827814:0:78052:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787826821.827823:0:78052:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f33a.replay-single--6, flags=0 00000080:00200000:1.0:1787826821.827839:0:78052:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826821.827845:0:78052:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826821.827852:0:78052:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a000/0xb3f83dad1b7e0199 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a10ac expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826821.827865:0:78052:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a000/0xb3f83dad1b7e0199 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a10ac expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826821.827872:0:78052:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826821.827876:0:78052:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826821.827878:0:78052:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e0199 00000080:00010000:1.0:1787826821.827881:0:78052:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0199 from it ffffab3ac1da7d68 00010000:00010000:1.0:1787826821.827884:0:78052:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a000/0xb3f83dad1b7e0199 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a10ac expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826821.827891:0:78052:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a000/0xb3f83dad1b7e0199 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a10ac expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787826821.827899:0:78052:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787826821.827904:0:78052:0:(namei.c:2418:ll_unlink()) VFS Op:name=f33a.replay-single--6, dir=[0x200000007:0x1:0x0](ffff98561158e3c8) 00010000:00010000:1.0:1787826821.827933:0:78052:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a000/0xb3f83dad1b7e0199 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x5aa6d504d91a10ac expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787826821.827954:0:78052:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e0114 lrc: 2/0,0 mode: PR/PR res: [0x200015bc1:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x5aa6d504d91a0ede expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826821.827964:0:78052:0:(vvp_object.c:117:vvp_conf_set()) [0x200015bc1:0x7:0x0]: losing layout lock 00010000:00010000:1.0:1787826821.827990:0:78052:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e0114 lrc: 1/0,0 mode: --/PR res: [0x200015bc1:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x5aa6d504d91a0ede expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826821.827997:0:78052:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a000/0xb3f83dad1b7e0199 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a10ac expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787826821.828008:0:78052:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 2 locks packed 00010000:00010000:1.0:1787826821.828011:0:78052:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e0114 lrc: 0/0,0 mode: --/PR res: [0x200015bc1:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x5aa6d504d91a0ede expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826821.828026:0:78052:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a000/0xb3f83dad1b7e0199 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a10ac expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000020:00100000:1.0:1787826821.828043:0:78052:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787826821.828051:0:78052:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610e33840 x1874669225545216/t0(0) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826821.828069:0:78052:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e33840 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:78052:1874669225545216:192.168.203.114@tcp:36:rm.0 00000100:00100000:1.0:1787826821.828128:0:78052:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607e069c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826821.832584:0:78052:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e33840 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:78052:1874669225545216:192.168.203.114@tcp:36:rm.0 00000080:00200000:1.0:1787826821.832655:0:78052:0:(vvp_io.c:1885:vvp_io_init()) [0x200015bc1:0x7:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826821.832668:0:78052:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826821.832755:0:78052:0:(vvp_io.c:304:vvp_io_fini()) [0x200015bc1:0x7:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826821.832764:0:78052:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200015bc1:0x7:0x0](ffff98561158a1c8) 00000008:00010000:1.0:1787826821.832798:0:78052:0:(osc_object.c:202:osc_object_ast_clear()) ### update lock size 0 blocks 0 [cma]time: 0 0 0 by oinfo size 0 blocks 0 [cma]time 0 0 0 ns: lustre-OST0001-osc-ffff985602bc9800 lock: 00000000af8864da/0xb3f83dad1b7e01a0 lrc: 1/0,0 mode: PR/PR res: [0x2c0000401:0x5e5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x5aa6d504d91a10ba expref: -99 pid: 78052 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826821.832874:0:78052:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826821.832893:0:78052:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826821.832941:0:78052:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e01df lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 78052 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.832954:0:78052:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e01df lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78052 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.832962:0:78052:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e01df lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78052 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826821.832980:0:78052:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e33c00 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:78052:1874669225545344:192.168.203.114@tcp:101:rm.0 00000100:00100000:1.0:1787826821.833052:0:78052:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607e069c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826821.834262:0:78052:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e33c00 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:78052:1874669225545344:192.168.203.114@tcp:101:rm.0 00010000:00010000:1.0:1787826821.834277:0:78052:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e01df lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1107 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.834291:0:78052:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e01df lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1107 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.834303:0:78052:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e01df lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1107 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.834313:0:78052:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e01df lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1107 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.834323:0:78052:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e01df lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1107 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.834331:0:78052:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e01df lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1107 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.834343:0:78052:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e01df lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1107 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826821.834353:0:78052:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff985610e33c00 x1874669225545344/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826837 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826821.834366:0:78052:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e01df lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1107 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.834375:0:78052:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:1.0:1787826821.834380:0:78052:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:1.0:1787826821.834402:0:78052:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:1.0:1787826821.834408:0:78052:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e01df 00000080:00010000:1.0:1787826821.834412:0:78052:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826821.834415:0:78052:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e01df from it ffffab3ac1da7b58 00010000:00010000:1.0:1787826821.834418:0:78052:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e01df lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1107 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.834427:0:78052:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e01df lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1107 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826821.834445:0:78052:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787826821.834453:0:78052:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f33a.replay-single--7, flags=8 00010000:00010000:1.0:1787826821.834466:0:78052:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e0122 lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a0f0f expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00000080:00010000:1.0:1787826821.834474:0:78052:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0122 from it ffffab3ac1da7af8 00010000:00010000:1.0:1787826821.834477:0:78052:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e0122 lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a0f0f expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826821.834484:0:78052:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e0122 lrc: 2/0,0 mode: PR/PR res: [0x200015bc1:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a0f0f expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200008:1.0:1787826821.834511:0:78052:0:(file.c:6313:ll_getattr_dentry()) START file f33a.replay-single--7:[0x200015bc1:0x8:0x0](ffff98561158aa08), request_mask 2047, flags 0, foreign 0 00000080:00200000:1.0:1787826821.834516:0:78052:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200015bc1:0x8:0x0](ffff98561158aa08),name=f33a.replay-single--7 00000002:00010000:1.0:1787826821.834529:0:78052:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200015bc1:0x8:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826821.834537:0:78052:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e0122 lrc: 2/1,0 mode: PR/PR res: [0x200015bc1:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a0f0f expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787826821.834546:0:78052:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e0122 lrc: 2/1,0 mode: PR/PR res: [0x200015bc1:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a0f0f expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787826821.834553:0:78052:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826821.834557:0:78052:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200015bc1:0x8:0x0](ffff98561158aa08) 00000080:00010000:1.0:1787826821.834560:0:78052:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200015bc1:0x8:0x0](ffff98561158aa08) for lock 0xb3f83dad1b7e0122 00000080:00010000:1.0:1787826821.834563:0:78052:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0122 from it ffffab3ac1da7cb8 00010000:00010000:1.0:1787826821.834567:0:78052:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e0122 lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a0f0f expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787826821.834574:0:78052:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e0122 lrc: 2/0,0 mode: PR/PR res: [0x200015bc1:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a0f0f expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 4 lru_type: 0 00000080:00200000:1.0:1787826821.834592:0:78052:0:(vvp_io.c:1885:vvp_io_init()) [0x200015bc1:0x8:0x0] ignore/verify layout 0/1, layout version 0 restore needed 0 00000001:00010000:1.0:1787826821.834603:0:78052:0:(glimpse.c:73:cl_glimpse_lock()) Glimpsing inode [0x200015bc1:0x8:0x0] 00000008:00010000:1.0:1787826821.834621:0:78052:0:(lustre_osc.h:166:osc_enq2ldlm_flags()) flags: a 00000008:00010000:1.0:1787826821.834626:0:78052:0:(osc_lock.c:1320:osc_lock_init()) ### lock ffff98560560c730, osc lock ffff985602f8f110, flags 0x400001002 00010000:00010000:1.0:1787826821.834642:0:78052:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0000-osc-ffff985602bc9800 lock: 000000004bcc6baa/0xb3f83dad1b7e01ca lrc: 2/1,0 mode: PR/PR res: [0x280000401:0x5e5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a10cf expref: -99 pid: 78053 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826821.834654:0:78052:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 18446744073709551615) ns: lustre-OST0000-osc-ffff985602bc9800 lock: 000000004bcc6baa/0xb3f83dad1b7e01ca lrc: 2/1,0 mode: PR/PR res: [0x280000401:0x5e5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a10cf expref: -99 pid: 78053 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000008:00010000:1.0:1787826821.834672:0:78052:0:(osc_lock.c:192:osc_lock_lvb_update()) ### acquired size 0, setting rss=0; kms=0, end=18446744073709551615 ns: lustre-OST0000-osc-ffff985602bc9800 lock: 000000004bcc6baa/0xb3f83dad1b7e01ca lrc: 4/1,0 mode: PR/PR res: [0x280000401:0x5e5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a10cf expref: -99 pid: 78053 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000020:00200000:1.0:1787826821.834687:0:78052:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00010000:00010000:1.0:1787826821.834710:0:78052:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff985602bc9800 lock: 000000004bcc6baa/0xb3f83dad1b7e01ca lrc: 5/1,0 mode: PR/PR res: [0x280000401:0x5e5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x5aa6d504d91a10cf expref: -99 pid: 78053 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826821.834733:0:78052:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0000-osc-ffff985602bc9800 lock: 000000004bcc6baa/0xb3f83dad1b7e01ca lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5e5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x5aa6d504d91a10cf expref: -99 pid: 78053 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000020:00200000:1.0:1787826821.834758:0:78052:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1787826821.834769:0:78052:0:(file.c:1775:ll_merge_attr_nolock()) [0x200015bc1:0x8:0x0] updating i_size 0 i_blocks 0 00000020:00010000:1.0:1787826821.834773:0:78052:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff985605499b28 (ffff985610fe1bd8/0) at cl_lock_release():245 00000020:00010000:1.0:1787826821.834779:0:78052:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff985605499b28 (ffff985610fe1bd8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1787826821.834787:0:78052:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff98560560c730 (ffff985610fe1998/1) at cl_lock_cancel():142 00000020:00010000:1.0:1787826821.834793:0:78052:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff985605499b28 (ffff985610fe1bd8/0) at cl_lock_fini():72 00000020:00010000:1.0:1787826821.834799:0:78052:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff98560560c730 (ffff985610fe1bd8/1) at cl_lock_fini():72 00000080:00200000:1.0:1787826821.834810:0:78052:0:(vvp_io.c:304:vvp_io_fini()) [0x200015bc1:0x8:0x0] ignore/verify layout 0/1, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826821.834833:0:78052:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826821.834846:0:78052:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826821.834860:0:78052:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e01df lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 9 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1107 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826821.834871:0:78052:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e01df lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 8 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1107 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826821.834878:0:78052:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826821.834887:0:78052:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826821.834892:0:78052:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e01df 00000080:00010000:1.0:1787826821.834899:0:78052:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e01df from it ffffab3ac1da7c10 00010000:00010000:1.0:1787826821.834905:0:78052:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e01df lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 9 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1107 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826821.834913:0:78052:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e01df lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 10 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1107 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826821.834929:0:78052:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787826821.834940:0:78052:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f33a.replay-single--7, flags=0 00000080:00200000:1.0:1787826821.834946:0:78052:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826821.834954:0:78052:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826821.834975:0:78052:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e01df lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 13 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1107 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826821.834989:0:78052:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e01df lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 11 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1107 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826821.834996:0:78052:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826821.835000:0:78052:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826821.835005:0:78052:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e01df 00000080:00010000:1.0:1787826821.835010:0:78052:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e01df from it ffffab3ac1da7d68 00010000:00010000:1.0:1787826821.835013:0:78052:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e01df lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 11 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1107 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826821.835020:0:78052:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e01df lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 11 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1107 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787826821.835029:0:78052:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787826821.835035:0:78052:0:(namei.c:2418:ll_unlink()) VFS Op:name=f33a.replay-single--7, dir=[0x200000007:0x1:0x0](ffff98561158e3c8) 00010000:00010000:1.0:1787826821.835059:0:78052:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e01df lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 12 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x5aa6d504d91a1107 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787826821.835084:0:78052:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e0122 lrc: 2/0,0 mode: PR/PR res: [0x200015bc1:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x5aa6d504d91a0f0f expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 4 lru_type: 0 00000080:00200000:1.0:1787826821.835097:0:78052:0:(vvp_object.c:117:vvp_conf_set()) [0x200015bc1:0x8:0x0]: losing layout lock 00010000:00010000:1.0:1787826821.835118:0:78052:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e0122 lrc: 1/0,0 mode: --/PR res: [0x200015bc1:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x5aa6d504d91a0f0f expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 4 lru_type: 0 00010000:00010000:1.0:1787826821.835125:0:78052:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e01df lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 11 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a1107 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787826821.835131:0:78052:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 2 locks packed 00010000:00010000:1.0:1787826821.835133:0:78052:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e0122 lrc: 0/0,0 mode: --/PR res: [0x200015bc1:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x5aa6d504d91a0f0f expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 4 lru_type: 0 00010000:00010000:1.0:1787826821.835153:0:78052:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e01df lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 11 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a1107 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000020:00100000:1.0:1787826821.835170:0:78052:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787826821.835178:0:78052:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610e33c00 x1874669225545728/t0(0) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826821.835193:0:78052:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e33c00 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:78052:1874669225545728:192.168.203.114@tcp:36:rm.0 00000100:00100000:1.0:1787826821.835244:0:78052:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607e069c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826821.837405:0:78052:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e33c00 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:78052:1874669225545728:192.168.203.114@tcp:36:rm.0 00000080:00200000:1.0:1787826821.837443:0:78052:0:(vvp_io.c:1885:vvp_io_init()) [0x200015bc1:0x8:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826821.837453:0:78052:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826821.837476:0:78052:0:(vvp_io.c:304:vvp_io_fini()) [0x200015bc1:0x8:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826821.837488:0:78052:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200015bc1:0x8:0x0](ffff98561158aa08) 00000008:00010000:1.0:1787826821.837514:0:78052:0:(osc_object.c:202:osc_object_ast_clear()) ### update lock size 0 blocks 0 [cma]time: 0 0 0 by oinfo size 0 blocks 0 [cma]time 0 0 0 ns: lustre-OST0000-osc-ffff985602bc9800 lock: 000000004bcc6baa/0xb3f83dad1b7e01ca lrc: 1/0,0 mode: PR/PR res: [0x280000401:0x5e5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x5aa6d504d91a10cf expref: -99 pid: 78053 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826821.837581:0:78052:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826821.837595:0:78052:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826821.837624:0:78052:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a000/0xb3f83dad1b7e0225 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 78052 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.837633:0:78052:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a000/0xb3f83dad1b7e0225 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78052 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.837641:0:78052:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a000/0xb3f83dad1b7e0225 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78052 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826821.837654:0:78052:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e30b40 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:78052:1874669225545856:192.168.203.114@tcp:101:rm.0 00000100:00100000:1.0:1787826821.837720:0:78052:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607e069c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826821.839010:0:78052:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e30b40 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:78052:1874669225545856:192.168.203.114@tcp:101:rm.0 00010000:00010000:1.0:1787826821.839022:0:78052:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a000/0xb3f83dad1b7e0225 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1146 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.839029:0:78052:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a000/0xb3f83dad1b7e0225 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1146 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.839036:0:78052:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a000/0xb3f83dad1b7e0225 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1146 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.839042:0:78052:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a000/0xb3f83dad1b7e0225 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1146 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.839049:0:78052:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a000/0xb3f83dad1b7e0225 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1146 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.839053:0:78052:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a000/0xb3f83dad1b7e0225 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1146 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.839058:0:78052:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a000/0xb3f83dad1b7e0225 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1146 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826821.839064:0:78052:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff985610e30b40 x1874669225545856/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826837 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826821.839072:0:78052:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a000/0xb3f83dad1b7e0225 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1146 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.839082:0:78052:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:1.0:1787826821.839085:0:78052:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:1.0:1787826821.839098:0:78052:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:1.0:1787826821.839101:0:78052:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e0225 00000080:00010000:1.0:1787826821.839104:0:78052:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826821.839106:0:78052:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0225 from it ffffab3ac1da7b58 00010000:00010000:1.0:1787826821.839108:0:78052:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a000/0xb3f83dad1b7e0225 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1146 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.839114:0:78052:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a000/0xb3f83dad1b7e0225 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1146 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826821.839123:0:78052:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787826821.839128:0:78052:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f33a.replay-single--8, flags=8 00010000:00010000:1.0:1787826821.839135:0:78052:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7e0130 lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a0f40 expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00000080:00010000:1.0:1787826821.839140:0:78052:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0130 from it ffffab3ac1da7af8 00010000:00010000:1.0:1787826821.839141:0:78052:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7e0130 lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a0f40 expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826821.839145:0:78052:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7e0130 lrc: 2/0,0 mode: PR/PR res: [0x200015bc1:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a0f40 expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200008:1.0:1787826821.839163:0:78052:0:(file.c:6313:ll_getattr_dentry()) START file f33a.replay-single--8:[0x200015bc1:0x9:0x0](ffff9856115880c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:1.0:1787826821.839166:0:78052:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200015bc1:0x9:0x0](ffff9856115880c8),name=f33a.replay-single--8 00000002:00010000:1.0:1787826821.839175:0:78052:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200015bc1:0x9:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826821.839179:0:78052:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7e0130 lrc: 2/1,0 mode: PR/PR res: [0x200015bc1:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a0f40 expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787826821.839185:0:78052:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7e0130 lrc: 2/1,0 mode: PR/PR res: [0x200015bc1:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a0f40 expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787826821.839190:0:78052:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826821.839199:0:78052:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200015bc1:0x9:0x0](ffff9856115880c8) 00000080:00010000:1.0:1787826821.839202:0:78052:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200015bc1:0x9:0x0](ffff9856115880c8) for lock 0xb3f83dad1b7e0130 00000080:00010000:1.0:1787826821.839208:0:78052:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0130 from it ffffab3ac1da7cb8 00010000:00010000:1.0:1787826821.839212:0:78052:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7e0130 lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a0f40 expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787826821.839221:0:78052:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7e0130 lrc: 2/0,0 mode: PR/PR res: [0x200015bc1:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a0f40 expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 4 lru_type: 0 00000080:00200000:1.0:1787826821.839256:0:78052:0:(vvp_io.c:1885:vvp_io_init()) [0x200015bc1:0x9:0x0] ignore/verify layout 0/1, layout version 0 restore needed 0 00000001:00010000:1.0:1787826821.839264:0:78052:0:(glimpse.c:73:cl_glimpse_lock()) Glimpsing inode [0x200015bc1:0x9:0x0] 00000008:00010000:1.0:1787826821.839278:0:78052:0:(lustre_osc.h:166:osc_enq2ldlm_flags()) flags: a 00000008:00010000:1.0:1787826821.839282:0:78052:0:(osc_lock.c:1320:osc_lock_init()) ### lock ffff98560560c8b0, osc lock ffff985602f8f110, flags 0x400001002 00010000:00010000:1.0:1787826821.839300:0:78052:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0001-osc-ffff985602bc9800 lock: 0000000005f20dd6/0xb3f83dad1b7e01d1 lrc: 2/1,0 mode: PR/PR res: [0x2c0000401:0x5e6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a10d6 expref: -99 pid: 78053 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826821.839313:0:78052:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 18446744073709551615) ns: lustre-OST0001-osc-ffff985602bc9800 lock: 0000000005f20dd6/0xb3f83dad1b7e01d1 lrc: 2/1,0 mode: PR/PR res: [0x2c0000401:0x5e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a10d6 expref: -99 pid: 78053 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000008:00010000:1.0:1787826821.839334:0:78052:0:(osc_lock.c:192:osc_lock_lvb_update()) ### acquired size 0, setting rss=0; kms=0, end=18446744073709551615 ns: lustre-OST0001-osc-ffff985602bc9800 lock: 0000000005f20dd6/0xb3f83dad1b7e01d1 lrc: 4/1,0 mode: PR/PR res: [0x2c0000401:0x5e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a10d6 expref: -99 pid: 78053 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000020:00200000:1.0:1787826821.839351:0:78052:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00010000:00010000:1.0:1787826821.839362:0:78052:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0001-osc-ffff985602bc9800 lock: 0000000005f20dd6/0xb3f83dad1b7e01d1 lrc: 5/1,0 mode: PR/PR res: [0x2c0000401:0x5e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x5aa6d504d91a10d6 expref: -99 pid: 78053 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826821.839371:0:78052:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0001-osc-ffff985602bc9800 lock: 0000000005f20dd6/0xb3f83dad1b7e01d1 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x5e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x5aa6d504d91a10d6 expref: -99 pid: 78053 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000020:00200000:1.0:1787826821.839379:0:78052:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1787826821.839389:0:78052:0:(file.c:1775:ll_merge_attr_nolock()) [0x200015bc1:0x9:0x0] updating i_size 0 i_blocks 0 00000020:00010000:1.0:1787826821.839396:0:78052:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff985605499b28 (ffff985610fe1bd8/0) at cl_lock_release():245 00000020:00010000:1.0:1787826821.839401:0:78052:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff985605499b28 (ffff985610fe1bd8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1787826821.839405:0:78052:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff98560560c8b0 (ffff985610fe1998/1) at cl_lock_cancel():142 00000020:00010000:1.0:1787826821.839410:0:78052:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff985605499b28 (ffff985610fe1bd8/0) at cl_lock_fini():72 00000020:00010000:1.0:1787826821.839416:0:78052:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff98560560c8b0 (ffff985610fe1bd8/1) at cl_lock_fini():72 00000080:00200000:1.0:1787826821.839427:0:78052:0:(vvp_io.c:304:vvp_io_fini()) [0x200015bc1:0x9:0x0] ignore/verify layout 0/1, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826821.839452:0:78052:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826821.839464:0:78052:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826821.839473:0:78052:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a000/0xb3f83dad1b7e0225 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 10 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1146 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826821.839484:0:78052:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a000/0xb3f83dad1b7e0225 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 10 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1146 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826821.839501:0:78052:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826821.839510:0:78052:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826821.839515:0:78052:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e0225 00000080:00010000:1.0:1787826821.839520:0:78052:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0225 from it ffffab3ac1da7c10 00010000:00010000:1.0:1787826821.839532:0:78052:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a000/0xb3f83dad1b7e0225 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 12 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1146 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826821.839548:0:78052:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a000/0xb3f83dad1b7e0225 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 11 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1146 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826821.839558:0:78052:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787826821.839567:0:78052:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f33a.replay-single--8, flags=0 00000080:00200000:1.0:1787826821.839573:0:78052:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826821.839583:0:78052:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826821.839596:0:78052:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a000/0xb3f83dad1b7e0225 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 15 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1146 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826821.839610:0:78052:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a000/0xb3f83dad1b7e0225 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 13 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1146 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826821.839620:0:78052:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826821.839626:0:78052:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826821.839633:0:78052:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e0225 00000080:00010000:1.0:1787826821.839639:0:78052:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0225 from it ffffab3ac1da7d68 00010000:00010000:1.0:1787826821.839642:0:78052:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a000/0xb3f83dad1b7e0225 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 14 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1146 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826821.839651:0:78052:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a000/0xb3f83dad1b7e0225 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 15 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1146 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787826821.839677:0:78052:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787826821.839684:0:78052:0:(namei.c:2418:ll_unlink()) VFS Op:name=f33a.replay-single--8, dir=[0x200000007:0x1:0x0](ffff98561158e3c8) 00010000:00010000:1.0:1787826821.839725:0:78052:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a000/0xb3f83dad1b7e0225 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 18 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x5aa6d504d91a1146 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787826821.839765:0:78052:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7e0130 lrc: 2/0,0 mode: PR/PR res: [0x200015bc1:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x5aa6d504d91a0f40 expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 4 lru_type: 0 00000080:00200000:1.0:1787826821.839787:0:78052:0:(vvp_object.c:117:vvp_conf_set()) [0x200015bc1:0x9:0x0]: losing layout lock 00010000:00010000:1.0:1787826821.839814:0:78052:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7e0130 lrc: 1/0,0 mode: --/PR res: [0x200015bc1:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x5aa6d504d91a0f40 expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 4 lru_type: 0 00010000:00010000:1.0:1787826821.839827:0:78052:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a000/0xb3f83dad1b7e0225 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 19 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a1146 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787826821.839833:0:78052:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 2 locks packed 00010000:00010000:1.0:1787826821.839836:0:78052:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7e0130 lrc: 0/0,0 mode: --/PR res: [0x200015bc1:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x5aa6d504d91a0f40 expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 4 lru_type: 0 00010000:00010000:1.0:1787826821.839851:0:78052:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a000/0xb3f83dad1b7e0225 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 19 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a1146 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000020:00100000:1.0:1787826821.839876:0:78052:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:3.0:1787826821.839882:0:2385:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1787826821.839884:0:78052:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610e30b40 x1874669225546240/t0(0) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826821.839895:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669225546112:192.168.203.114@tcp:63:ll_sa.0 00000100:00100000:1.0:1787826821.839905:0:78052:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e30b40 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:78052:1874669225546240:192.168.203.114@tcp:36:rm.0 00000100:00100000:1.0:1787826821.839954:0:78052:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607e069c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826821.841808:0:78052:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e30b40 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:78052:1874669225546240:192.168.203.114@tcp:36:rm.0 00000080:00200000:1.0:1787826821.841838:0:78052:0:(vvp_io.c:1885:vvp_io_init()) [0x200015bc1:0x9:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826821.841846:0:78052:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826821.841865:0:78052:0:(vvp_io.c:304:vvp_io_fini()) [0x200015bc1:0x9:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826821.841871:0:78052:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200015bc1:0x9:0x0](ffff9856115880c8) 00000008:00010000:1.0:1787826821.841889:0:78052:0:(osc_object.c:202:osc_object_ast_clear()) ### update lock size 0 blocks 0 [cma]time: 0 0 0 by oinfo size 0 blocks 0 [cma]time 0 0 0 ns: lustre-OST0001-osc-ffff985602bc9800 lock: 0000000005f20dd6/0xb3f83dad1b7e01d1 lrc: 1/0,0 mode: PR/PR res: [0x2c0000401:0x5e6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x5aa6d504d91a10d6 expref: -99 pid: 78053 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826821.841931:0:78052:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826821.841940:0:78052:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826821.841944:0:78052:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 30 res 8589934599/1 (0 0) 00010000:00010000:1.0:1787826821.841963:0:78052:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a200/0xb3f83dad1b7e02a3 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 19 type: IBT flags: 0x10000000000000 pid: 78052 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.841968:0:78052:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a200/0xb3f83dad1b7e02a3 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 19 type: IBT flags: 0x0 pid: 78052 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.841972:0:78052:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a200/0xb3f83dad1b7e02a3 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 19 type: IBT flags: 0x0 pid: 78052 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826821.841980:0:78052:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e30000 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:78052:1874669225546368:192.168.203.114@tcp:101:rm.0 00000100:00100000:1.0:1787826821.842009:0:78052:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607e069c0 going to sleep for 11 seconds 00010000:00010000:3.0:1787826821.842539:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ede00/0xb3f83dad1b7e022c lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 19 type: IBT flags: 0x0 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.842550:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ede00/0xb3f83dad1b7e022c lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 19 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.842559:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ede00/0xb3f83dad1b7e022c lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 19 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.842568:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ede00/0xb3f83dad1b7e022c lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 19 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.842584:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ede00/0xb3f83dad1b7e022c lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 19 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.842627:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9856129ede00 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826821.842637:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ede00/0xb3f83dad1b7e022c lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 19 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.842644:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ede00/0xb3f83dad1b7e022c lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 19 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.842651:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ede00/0xb3f83dad1b7e022c lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 19 type: IBT flags: 0x20e400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.842662:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826821.842667:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ede00/0xb3f83dad1b7e022c lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 19 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.842674:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ede00/0xb3f83dad1b7e022c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 19 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.842707:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ec600/0xb3f83dad1b7e0233 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 18 type: IBT flags: 0x0 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.842714:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ec600/0xb3f83dad1b7e0233 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 18 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.842721:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ec600/0xb3f83dad1b7e0233 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 18 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.842727:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ec600/0xb3f83dad1b7e0233 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 18 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.842734:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ec600/0xb3f83dad1b7e0233 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 18 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.842739:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9856129ec600 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826821.842742:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ec600/0xb3f83dad1b7e0233 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 18 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.842748:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ec600/0xb3f83dad1b7e0233 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 18 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.842754:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ec600/0xb3f83dad1b7e0233 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 18 type: IBT flags: 0x20e400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.842760:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826821.842762:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ec600/0xb3f83dad1b7e0233 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 18 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.842768:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ec600/0xb3f83dad1b7e0233 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 18 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.842781:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ed200/0xb3f83dad1b7e023a lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 17 type: IBT flags: 0x0 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.842787:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ed200/0xb3f83dad1b7e023a lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 17 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.842793:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ed200/0xb3f83dad1b7e023a lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 17 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.842812:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ed200/0xb3f83dad1b7e023a lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 17 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.842819:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ed200/0xb3f83dad1b7e023a lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 17 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.842824:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9856129ed200 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826821.842832:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ed200/0xb3f83dad1b7e023a lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 17 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.842837:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ed200/0xb3f83dad1b7e023a lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 17 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.842843:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ed200/0xb3f83dad1b7e023a lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 17 type: IBT flags: 0x20e400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.842849:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826821.842858:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ed200/0xb3f83dad1b7e023a lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 17 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.842864:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ed200/0xb3f83dad1b7e023a lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 17 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.842877:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129edc00/0xb3f83dad1b7e0241 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 16 type: IBT flags: 0x0 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.842884:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129edc00/0xb3f83dad1b7e0241 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 16 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.842890:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129edc00/0xb3f83dad1b7e0241 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 16 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.842896:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129edc00/0xb3f83dad1b7e0241 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 16 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.842907:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129edc00/0xb3f83dad1b7e0241 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 16 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.842912:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9856129edc00 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826821.842915:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129edc00/0xb3f83dad1b7e0241 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 16 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.842921:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129edc00/0xb3f83dad1b7e0241 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 16 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.842927:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129edc00/0xb3f83dad1b7e0241 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 16 type: IBT flags: 0x20e400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.842933:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826821.842936:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129edc00/0xb3f83dad1b7e0241 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 16 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.842943:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129edc00/0xb3f83dad1b7e0241 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 16 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.842953:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da5a00/0xb3f83dad1b7e0248 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 15 type: IBT flags: 0x0 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.842960:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da5a00/0xb3f83dad1b7e0248 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 15 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.842965:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da5a00/0xb3f83dad1b7e0248 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 15 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.842972:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da5a00/0xb3f83dad1b7e0248 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 15 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.842978:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da5a00/0xb3f83dad1b7e0248 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 15 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.842983:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985610da5a00 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826821.842986:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da5a00/0xb3f83dad1b7e0248 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 15 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.842992:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da5a00/0xb3f83dad1b7e0248 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 15 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.843132:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da5a00/0xb3f83dad1b7e0248 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 15 type: IBT flags: 0x20e400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.843139:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826821.843144:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da5a00/0xb3f83dad1b7e0248 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 15 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.843151:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da5a00/0xb3f83dad1b7e0248 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 15 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.843164:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da4800/0xb3f83dad1b7e024f lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 14 type: IBT flags: 0x0 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.843171:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da4800/0xb3f83dad1b7e024f lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 14 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.843177:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da4800/0xb3f83dad1b7e024f lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 14 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.843183:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da4800/0xb3f83dad1b7e024f lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 14 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.843190:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da4800/0xb3f83dad1b7e024f lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 14 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.843196:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985610da4800 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826821.843200:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da4800/0xb3f83dad1b7e024f lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 14 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.843206:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da4800/0xb3f83dad1b7e024f lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 14 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826821.843212:0:78052:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e30000 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:78052:1874669225546368:192.168.203.114@tcp:101:rm.0 00010000:00010000:3.0:1787826821.843215:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da4800/0xb3f83dad1b7e024f lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 14 type: IBT flags: 0x20e400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.843234:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787826821.843236:0:78052:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a200/0xb3f83dad1b7e02a3 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 14 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1185 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.843239:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da4800/0xb3f83dad1b7e024f lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 15 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.843257:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da4800/0xb3f83dad1b7e024f lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 15 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.843269:0:78052:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a200/0xb3f83dad1b7e02a3 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 15 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1185 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.843276:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da4e00/0xb3f83dad1b7e0256 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 14 type: IBT flags: 0x0 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.843281:0:78052:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a200/0xb3f83dad1b7e02a3 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 14 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1185 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.843293:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da4e00/0xb3f83dad1b7e0256 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 13 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.843295:0:78052:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a200/0xb3f83dad1b7e02a3 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 14 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1185 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.843310:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da4e00/0xb3f83dad1b7e0256 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 14 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.843318:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da4e00/0xb3f83dad1b7e0256 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 13 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.843325:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da4e00/0xb3f83dad1b7e0256 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 13 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.843327:0:78052:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a200/0xb3f83dad1b7e02a3 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 14 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1185 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.843341:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985610da4e00 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:1.0:1787826821.843342:0:78052:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a200/0xb3f83dad1b7e02a3 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 13 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1185 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.843352:0:78052:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a200/0xb3f83dad1b7e02a3 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 13 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1185 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.843360:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da4e00/0xb3f83dad1b7e0256 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 13 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826821.843363:0:78052:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff985610e30000 x1874669225546368/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826837 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00010000:00010000:3.0:1787826821.843374:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da4e00/0xb3f83dad1b7e0256 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 13 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00010000:1.0:1787826821.843383:0:78052:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a200/0xb3f83dad1b7e02a3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 14 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1185 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.843385:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da4e00/0xb3f83dad1b7e0256 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 14 type: IBT flags: 0x20e400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.843402:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787826821.843403:0:78052:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8589934599/1 (0 0) 00010000:00010000:3.0:1787826821.843407:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da4e00/0xb3f83dad1b7e0256 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 13 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00000080:00200000:1.0:1787826821.843410:0:78052:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00010000:00010000:3.0:1787826821.843419:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da4e00/0xb3f83dad1b7e0256 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 13 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00000080:00200000:1.0:1787826821.843432:0:78052:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00010000:00010000:3.0:1787826821.843432:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da4000/0xb3f83dad1b7e025d lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 12 type: IBT flags: 0x0 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00000080:00010000:1.0:1787826821.843442:0:78052:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e02a3 00010000:00010000:3.0:1787826821.843443:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da4000/0xb3f83dad1b7e025d lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 12 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00000080:00010000:1.0:1787826821.843451:0:78052:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00010000:00010000:3.0:1787826821.843453:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da4000/0xb3f83dad1b7e025d lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 12 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00000080:00010000:1.0:1787826821.843456:0:78052:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e02a3 from it ffffab3ac1da7b58 00010000:00010000:3.0:1787826821.843464:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da4000/0xb3f83dad1b7e025d lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 12 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.843470:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da4000/0xb3f83dad1b7e025d lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 12 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.843472:0:78052:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a200/0xb3f83dad1b7e02a3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 13 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1185 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.843485:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985610da4000 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:1.0:1787826821.843486:0:78052:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a200/0xb3f83dad1b7e02a3 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 12 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1185 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826821.843491:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da4000/0xb3f83dad1b7e025d lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 13 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.843504:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da4000/0xb3f83dad1b7e025d lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 12 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00000080:00200000:1.0:1787826821.843512:0:78052:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00010000:00010000:3.0:1787826821.843512:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da4000/0xb3f83dad1b7e025d lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 12 type: IBT flags: 0x20e400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00000080:00200000:1.0:1787826821.843522:0:78052:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f33a.replay-single--9, flags=8 00010000:00010000:3.0:1787826821.843523:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826821.843529:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da4000/0xb3f83dad1b7e025d lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 12 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.843540:0:78052:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06000/0xb3f83dad1b7e013e lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a0f71 expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826821.843540:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da4000/0xb3f83dad1b7e025d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 12 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00000080:00010000:1.0:1787826821.843561:0:78052:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e013e from it ffffab3ac1da7af8 00010000:00010000:3.0:1787826821.843563:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da4c00/0xb3f83dad1b7e0264 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 11 type: IBT flags: 0x0 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.843565:0:78052:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06000/0xb3f83dad1b7e013e lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a0f71 expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826821.843579:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da4c00/0xb3f83dad1b7e0264 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 11 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.843582:0:78052:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06000/0xb3f83dad1b7e013e lrc: 2/0,0 mode: PR/PR res: [0x200015bc1:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a0f71 expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:3.0:1787826821.843591:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da4c00/0xb3f83dad1b7e0264 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 11 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.843601:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da4c00/0xb3f83dad1b7e0264 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 11 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.843607:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da4c00/0xb3f83dad1b7e0264 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 11 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.843612:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985610da4c00 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826821.843616:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da4c00/0xb3f83dad1b7e0264 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 11 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00000080:00200008:1.0:1787826821.843620:0:78052:0:(file.c:6313:ll_getattr_dentry()) START file f33a.replay-single--9:[0x200015bc1:0xa:0x0](ffff98561011ba88), request_mask 2047, flags 0, foreign 0 00010000:00010000:3.0:1787826821.843622:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da4c00/0xb3f83dad1b7e0264 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 11 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00000080:00200000:1.0:1787826821.843629:0:78052:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200015bc1:0xa:0x0](ffff98561011ba88),name=f33a.replay-single--9 00010000:00010000:3.0:1787826821.843633:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da4c00/0xb3f83dad1b7e0264 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 11 type: IBT flags: 0x20e400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.843641:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00000002:00010000:1.0:1787826821.843644:0:78052:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200015bc1:0xa:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826821.843646:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da4c00/0xb3f83dad1b7e0264 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 11 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.843657:0:78052:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06000/0xb3f83dad1b7e013e lrc: 2/1,0 mode: PR/PR res: [0x200015bc1:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a0f71 expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:3.0:1787826821.843657:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da4c00/0xb3f83dad1b7e0264 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 11 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.843679:0:78052:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06000/0xb3f83dad1b7e013e lrc: 2/1,0 mode: PR/PR res: [0x200015bc1:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a0f71 expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:3.0:1787826821.843680:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da4a00/0xb3f83dad1b7e026b lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 12 type: IBT flags: 0x0 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.843720:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da4a00/0xb3f83dad1b7e026b lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 12 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00000080:00200000:1.0:1787826821.843724:0:78052:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826821.843735:0:78052:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200015bc1:0xa:0x0](ffff98561011ba88) 00010000:00010000:3.0:1787826821.843735:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da4a00/0xb3f83dad1b7e026b lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 11 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00000080:00010000:1.0:1787826821.843742:0:78052:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200015bc1:0xa:0x0](ffff98561011ba88) for lock 0xb3f83dad1b7e013e 00010000:00010000:3.0:1787826821.843747:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da4a00/0xb3f83dad1b7e026b lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 12 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00000080:00010000:1.0:1787826821.843749:0:78052:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e013e from it ffffab3ac1da7cb8 00010000:00010000:1.0:1787826821.843756:0:78052:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06000/0xb3f83dad1b7e013e lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a0f71 expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:3.0:1787826821.843758:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da4a00/0xb3f83dad1b7e026b lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 12 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.843779:0:78052:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06000/0xb3f83dad1b7e013e lrc: 2/0,0 mode: PR/PR res: [0x200015bc1:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a0f71 expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 4 lru_type: 0 00010000:00010000:3.0:1787826821.843781:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985610da4a00 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826821.843789:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da4a00/0xb3f83dad1b7e026b lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 13 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.843810:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da4a00/0xb3f83dad1b7e026b lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 12 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00000080:00200000:1.0:1787826821.843812:0:78052:0:(vvp_io.c:1885:vvp_io_init()) [0x200015bc1:0xa:0x0] ignore/verify layout 0/1, layout version 0 restore needed 0 00000001:00010000:1.0:1787826821.843825:0:78052:0:(glimpse.c:73:cl_glimpse_lock()) Glimpsing inode [0x200015bc1:0xa:0x0] 00010000:00010000:3.0:1787826821.843828:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da4a00/0xb3f83dad1b7e026b lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 14 type: IBT flags: 0x20e400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787826821.843840:0:78052:0:(lustre_osc.h:166:osc_enq2ldlm_flags()) flags: a 00000008:00010000:1.0:1787826821.843846:0:78052:0:(osc_lock.c:1320:osc_lock_init()) ### lock ffff98560560c8b0, osc lock ffff985602f8f110, flags 0x400001002 00010000:00010000:3.0:1787826821.843846:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826821.843852:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da4a00/0xb3f83dad1b7e026b lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 13 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.843860:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da4a00/0xb3f83dad1b7e026b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 14 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.843864:0:78052:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0000-osc-ffff985602bc9800 lock: 000000007f725c53/0xb3f83dad1b7e01d8 lrc: 2/1,0 mode: PR/PR res: [0x280000401:0x5e6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a10dd expref: -99 pid: 78053 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826821.843873:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da5400/0xb3f83dad1b7e0272 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 13 type: IBT flags: 0x0 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.843877:0:78052:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 18446744073709551615) ns: lustre-OST0000-osc-ffff985602bc9800 lock: 000000007f725c53/0xb3f83dad1b7e01d8 lrc: 2/1,0 mode: PR/PR res: [0x280000401:0x5e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a10dd expref: -99 pid: 78053 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826821.843897:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da5400/0xb3f83dad1b7e0272 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 14 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787826821.843902:0:78052:0:(osc_lock.c:192:osc_lock_lvb_update()) ### acquired size 0, setting rss=0; kms=0, end=18446744073709551615 ns: lustre-OST0000-osc-ffff985602bc9800 lock: 000000007f725c53/0xb3f83dad1b7e01d8 lrc: 4/1,0 mode: PR/PR res: [0x280000401:0x5e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a10dd expref: -99 pid: 78053 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826821.843918:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da5400/0xb3f83dad1b7e0272 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 14 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00200000:1.0:1787826821.843923:0:78052:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00010000:00010000:3.0:1787826821.843929:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da5400/0xb3f83dad1b7e0272 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 13 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.843930:0:78052:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff985602bc9800 lock: 000000007f725c53/0xb3f83dad1b7e01d8 lrc: 5/1,0 mode: PR/PR res: [0x280000401:0x5e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x5aa6d504d91a10dd expref: -99 pid: 78053 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826821.843945:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da5400/0xb3f83dad1b7e0272 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 14 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.843977:0:78052:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0000-osc-ffff985602bc9800 lock: 000000007f725c53/0xb3f83dad1b7e01d8 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x5aa6d504d91a10dd expref: -99 pid: 78053 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000020:00200000:1.0:1787826821.843988:0:78052:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1787826821.843997:0:78052:0:(file.c:1775:ll_merge_attr_nolock()) [0x200015bc1:0xa:0x0] updating i_size 0 i_blocks 0 00000020:00010000:1.0:1787826821.844001:0:78052:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff985605499b28 (ffff985610fe1bd8/0) at cl_lock_release():245 00000020:00010000:1.0:1787826821.844003:0:78052:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff985605499b28 (ffff985610fe1bd8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1787826821.844006:0:78052:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff98560560c8b0 (ffff985610fe1998/1) at cl_lock_cancel():142 00000020:00010000:1.0:1787826821.844010:0:78052:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff985605499b28 (ffff985610fe1bd8/0) at cl_lock_fini():72 00000020:00010000:1.0:1787826821.844013:0:78052:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff98560560c8b0 (ffff985610fe1bd8/1) at cl_lock_fini():72 00000080:00200000:1.0:1787826821.844020:0:78052:0:(vvp_io.c:304:vvp_io_fini()) [0x200015bc1:0xa:0x0] ignore/verify layout 0/1, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826821.844039:0:78052:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826821.844047:0:78052:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826821.844126:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985610da5400 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:1.0:1787826821.844130:0:78052:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a200/0xb3f83dad1b7e02a3 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 16 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1185 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826821.844147:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da5400/0xb3f83dad1b7e0272 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 14 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.844148:0:78052:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a200/0xb3f83dad1b7e02a3 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 15 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1185 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826821.844166:0:78052:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826821.844174:0:78052:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826821.844178:0:78052:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e02a3 00010000:00010000:3.0:1787826821.844178:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da5400/0xb3f83dad1b7e0272 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 16 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00000080:00010000:1.0:1787826821.844188:0:78052:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e02a3 from it ffffab3ac1da7c10 00010000:00010000:3.0:1787826821.844191:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da5400/0xb3f83dad1b7e0272 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 16 type: IBT flags: 0x20e400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.844193:0:78052:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a200/0xb3f83dad1b7e02a3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 17 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1185 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826821.844217:0:78052:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a200/0xb3f83dad1b7e02a3 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 15 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1185 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826821.844217:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00000080:00200000:1.0:1787826821.844247:0:78052:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00010000:00010000:3.0:1787826821.844249:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da5400/0xb3f83dad1b7e0272 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 17 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00000080:00200000:1.0:1787826821.844262:0:78052:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f33a.replay-single--9, flags=0 00010000:00010000:3.0:1787826821.844265:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da5400/0xb3f83dad1b7e0272 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 16 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00000080:00200000:1.0:1787826821.844268:0:78052:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826821.844278:0:78052:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826821.844280:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da5600/0xb3f83dad1b7e0279 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 17 type: IBT flags: 0x0 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.844303:0:78052:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a200/0xb3f83dad1b7e02a3 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 18 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1185 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826821.844317:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da5600/0xb3f83dad1b7e0279 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 16 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.844319:0:78052:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a200/0xb3f83dad1b7e02a3 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 17 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1185 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826821.844325:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da5600/0xb3f83dad1b7e0279 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 17 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00000080:00200000:1.0:1787826821.844337:0:78052:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00010000:00010000:3.0:1787826821.844337:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da5600/0xb3f83dad1b7e0279 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 17 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00000080:00010000:1.0:1787826821.844349:0:78052:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00010000:00010000:3.0:1787826821.844349:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da5600/0xb3f83dad1b7e0279 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 17 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00000080:00010000:1.0:1787826821.844356:0:78052:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e02a3 00010000:00010000:3.0:1787826821.844369:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985610da5600 already unused, calling callback (ffffffffc1d08990) 00000080:00010000:1.0:1787826821.844371:0:78052:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e02a3 from it ffffab3ac1da7d68 00010000:00010000:3.0:1787826821.844375:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da5600/0xb3f83dad1b7e0279 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 18 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.844377:0:78052:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a200/0xb3f83dad1b7e02a3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 19 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1185 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826821.844395:0:78052:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a200/0xb3f83dad1b7e02a3 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 18 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1185 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:3.0:1787826821.844396:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da5600/0xb3f83dad1b7e0279 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 19 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00000080:00200000:1.0:1787826821.844414:0:78052:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 3 00010000:00010000:3.0:1787826821.844414:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da5600/0xb3f83dad1b7e0279 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 18 type: IBT flags: 0x20e400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00000080:00200000:1.0:1787826821.844425:0:78052:0:(namei.c:2418:ll_unlink()) VFS Op:name=f33a.replay-single--9, dir=[0x200000007:0x1:0x0](ffff98561158e3c8) 00010000:00010000:3.0:1787826821.844425:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826821.844428:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da5600/0xb3f83dad1b7e0279 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 18 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.844440:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da5600/0xb3f83dad1b7e0279 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 17 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.844448:0:78052:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a200/0xb3f83dad1b7e02a3 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 17 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x5aa6d504d91a1185 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:3.0:1787826821.844450:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da5800/0xb3f83dad1b7e0280 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 18 type: IBT flags: 0x0 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.844475:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da5800/0xb3f83dad1b7e0280 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 19 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.844488:0:78052:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06000/0xb3f83dad1b7e013e lrc: 2/0,0 mode: PR/PR res: [0x200015bc1:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x5aa6d504d91a0f71 expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 4 lru_type: 0 00010000:00010000:3.0:1787826821.844494:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da5800/0xb3f83dad1b7e0280 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 18 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.844506:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da5800/0xb3f83dad1b7e0280 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 17 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.844511:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da5800/0xb3f83dad1b7e0280 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 18 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00000080:00200000:1.0:1787826821.844512:0:78052:0:(vvp_object.c:117:vvp_conf_set()) [0x200015bc1:0xa:0x0]: losing layout lock 00010000:00010000:3.0:1787826821.844524:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985610da5800 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826821.844528:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da5800/0xb3f83dad1b7e0280 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 19 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.844544:0:78052:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06000/0xb3f83dad1b7e013e lrc: 1/0,0 mode: --/PR res: [0x200015bc1:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x5aa6d504d91a0f71 expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 4 lru_type: 0 00010000:00010000:3.0:1787826821.844544:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da5800/0xb3f83dad1b7e0280 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 18 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.844572:0:78052:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a200/0xb3f83dad1b7e02a3 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 21 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a1185 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:3.0:1787826821.844572:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da5800/0xb3f83dad1b7e0280 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 20 type: IBT flags: 0x20e400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.844592:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787826821.844593:0:78052:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 2 locks packed 00010000:00010000:1.0:1787826821.844598:0:78052:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06000/0xb3f83dad1b7e013e lrc: 0/0,0 mode: --/PR res: [0x200015bc1:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x5aa6d504d91a0f71 expref: -99 pid: 78051 timeout: 0 lvb_type: 3 lru_score: 4 lru_type: 0 00010000:00010000:3.0:1787826821.844599:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da5800/0xb3f83dad1b7e0280 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 19 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.844615:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da5800/0xb3f83dad1b7e0280 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 19 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826821.844628:0:78052:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a200/0xb3f83dad1b7e02a3 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 18 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a1185 expref: -99 pid: 78052 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:3.0:1787826821.844633:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da5e00/0xb3f83dad1b7e0287 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 19 type: IBT flags: 0x0 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.844643:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da5e00/0xb3f83dad1b7e0287 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 18 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:1.0:1787826821.844646:0:78052:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787826821.844656:0:78052:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610e30000 x1874669225546624/t0(0) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:3.0:1787826821.844661:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da5e00/0xb3f83dad1b7e0287 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 19 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826821.844677:0:78052:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e30000 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:78052:1874669225546624:192.168.203.114@tcp:36:rm.0 00010000:00010000:3.0:1787826821.844682:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da5e00/0xb3f83dad1b7e0287 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 18 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.844710:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da5e00/0xb3f83dad1b7e0287 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 18 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.844717:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985610da5e00 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826821.844720:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da5e00/0xb3f83dad1b7e0287 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 18 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.844727:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da5e00/0xb3f83dad1b7e0287 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 18 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.844733:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da5e00/0xb3f83dad1b7e0287 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 18 type: IBT flags: 0x20e400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.844740:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826821.844744:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da5e00/0xb3f83dad1b7e0287 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 18 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.844750:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da5e00/0xb3f83dad1b7e0287 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 18 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826821.844755:0:78052:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607e069c0 going to sleep for 11 seconds 00010000:00010000:3.0:1787826821.844765:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da4600/0xb3f83dad1b7e028e lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 17 type: IBT flags: 0x0 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.844772:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da4600/0xb3f83dad1b7e028e lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 17 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.844778:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da4600/0xb3f83dad1b7e028e lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 17 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.844784:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da4600/0xb3f83dad1b7e028e lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 17 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.844790:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da4600/0xb3f83dad1b7e028e lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 17 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.844795:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985610da4600 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826821.844798:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da4600/0xb3f83dad1b7e028e lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 17 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.844803:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da4600/0xb3f83dad1b7e028e lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 17 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.844809:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da4600/0xb3f83dad1b7e028e lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 17 type: IBT flags: 0x20e400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.844816:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826821.844819:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da4600/0xb3f83dad1b7e028e lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 17 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.844824:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da4600/0xb3f83dad1b7e028e lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 17 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.844836:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da5200/0xb3f83dad1b7e0295 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 16 type: IBT flags: 0x0 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.844843:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da5200/0xb3f83dad1b7e0295 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 16 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.844848:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da5200/0xb3f83dad1b7e0295 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 16 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.844855:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da5200/0xb3f83dad1b7e0295 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 16 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.844862:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da5200/0xb3f83dad1b7e0295 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 16 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.844868:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985610da5200 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826821.844872:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da5200/0xb3f83dad1b7e0295 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 17 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.844879:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da5200/0xb3f83dad1b7e0295 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 17 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.844885:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da5200/0xb3f83dad1b7e0295 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 17 type: IBT flags: 0x20e400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.844892:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826821.844895:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da5200/0xb3f83dad1b7e0295 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 17 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.844902:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da5200/0xb3f83dad1b7e0295 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 17 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.844914:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da5000/0xb3f83dad1b7e029c lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 16 type: IBT flags: 0x0 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.844921:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da5000/0xb3f83dad1b7e029c lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 16 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.844928:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da5000/0xb3f83dad1b7e029c lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 16 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.844935:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da5000/0xb3f83dad1b7e029c lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 16 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.844942:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da5000/0xb3f83dad1b7e029c lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 16 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.844956:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985610da5000 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826821.844962:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da5000/0xb3f83dad1b7e029c lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 17 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.844973:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da5000/0xb3f83dad1b7e029c lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 16 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.844980:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da5000/0xb3f83dad1b7e029c lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 16 type: IBT flags: 0x20e400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.844994:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826821.845000:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da5000/0xb3f83dad1b7e029c lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 18 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.845008:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da5000/0xb3f83dad1b7e029c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 17 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826821.845071:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669225546112:192.168.203.114@tcp:63:ll_sa.0 00000100:00100000:3.0:1787826821.845615:0:2386:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1787826821.845634:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee2d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669225546752:192.168.203.114@tcp:63:ll_sa.0 00000100:00100000:1.0:1787826821.846932:0:78052:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e30000 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:78052:1874669225546624:192.168.203.114@tcp:36:rm.0 00000080:00200000:1.0:1787826821.846969:0:78052:0:(vvp_io.c:1885:vvp_io_init()) [0x200015bc1:0xa:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826821.846979:0:78052:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826821.847001:0:78052:0:(vvp_io.c:304:vvp_io_fini()) [0x200015bc1:0xa:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826821.847009:0:78052:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200015bc1:0xa:0x0](ffff98561011ba88) 00000008:00010000:1.0:1787826821.847032:0:78052:0:(osc_object.c:202:osc_object_ast_clear()) ### update lock size 0 blocks 0 [cma]time: 0 0 0 by oinfo size 0 blocks 0 [cma]time 0 0 0 ns: lustre-OST0000-osc-ffff985602bc9800 lock: 000000007f725c53/0xb3f83dad1b7e01d8 lrc: 1/0,0 mode: PR/PR res: [0x280000401:0x5e6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x5aa6d504d91a10dd expref: -99 pid: 78053 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826821.848396:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da5c00/0xb3f83dad1b7e02aa lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 34 type: IBT flags: 0x0 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.848412:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da5c00/0xb3f83dad1b7e02aa lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 34 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.848421:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da5c00/0xb3f83dad1b7e02aa lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 34 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.848430:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da5c00/0xb3f83dad1b7e02aa lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 34 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.848441:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da5c00/0xb3f83dad1b7e02aa lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 34 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.848449:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985610da5c00 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826821.848454:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da5c00/0xb3f83dad1b7e02aa lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 34 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.848466:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da5c00/0xb3f83dad1b7e02aa lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 34 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.848473:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da5c00/0xb3f83dad1b7e02aa lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 34 type: IBT flags: 0x20e400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.848481:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826821.848489:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da5c00/0xb3f83dad1b7e02aa lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 34 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.848497:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da5c00/0xb3f83dad1b7e02aa lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 34 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.848522:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da4400/0xb3f83dad1b7e02b1 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 33 type: IBT flags: 0x0 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.848528:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da4400/0xb3f83dad1b7e02b1 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 33 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.848531:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da4400/0xb3f83dad1b7e02b1 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 33 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.848535:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da4400/0xb3f83dad1b7e02b1 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 33 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.848538:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da4400/0xb3f83dad1b7e02b1 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 33 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.848541:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985610da4400 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826821.848543:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da4400/0xb3f83dad1b7e02b1 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 33 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.848550:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da4400/0xb3f83dad1b7e02b1 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 33 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.848554:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da4400/0xb3f83dad1b7e02b1 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 33 type: IBT flags: 0x20e400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.848557:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826821.848560:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da4400/0xb3f83dad1b7e02b1 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 33 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.848573:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da4400/0xb3f83dad1b7e02b1 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 33 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.848587:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da4200/0xb3f83dad1b7e02b8 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 32 type: IBT flags: 0x0 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.848593:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da4200/0xb3f83dad1b7e02b8 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 32 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.848598:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da4200/0xb3f83dad1b7e02b8 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 32 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.848605:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da4200/0xb3f83dad1b7e02b8 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 32 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.848612:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da4200/0xb3f83dad1b7e02b8 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 32 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.848618:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985610da4200 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826821.848621:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da4200/0xb3f83dad1b7e02b8 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 32 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.848628:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da4200/0xb3f83dad1b7e02b8 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 32 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.848634:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da4200/0xb3f83dad1b7e02b8 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 32 type: IBT flags: 0x20e400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.848640:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826821.848643:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da4200/0xb3f83dad1b7e02b8 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 32 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.848649:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610da4200/0xb3f83dad1b7e02b8 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 32 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.848663:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562ca00/0xb3f83dad1b7e02bf lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 31 type: IBT flags: 0x0 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.848670:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562ca00/0xb3f83dad1b7e02bf lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 31 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.848676:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562ca00/0xb3f83dad1b7e02bf lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 31 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.848683:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562ca00/0xb3f83dad1b7e02bf lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 31 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.848688:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562ca00/0xb3f83dad1b7e02bf lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 31 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.848731:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98560562ca00 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826821.848733:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562ca00/0xb3f83dad1b7e02bf lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 31 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.848741:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562ca00/0xb3f83dad1b7e02bf lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 31 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.848753:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562ca00/0xb3f83dad1b7e02bf lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 31 type: IBT flags: 0x20e400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.848759:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826821.848763:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562ca00/0xb3f83dad1b7e02bf lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 31 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.848770:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562ca00/0xb3f83dad1b7e02bf lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 31 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.848785:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562d800/0xb3f83dad1b7e02c6 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 30 type: IBT flags: 0x0 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.848792:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562d800/0xb3f83dad1b7e02c6 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 30 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.848798:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562d800/0xb3f83dad1b7e02c6 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 30 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.848804:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562d800/0xb3f83dad1b7e02c6 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 30 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.848810:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562d800/0xb3f83dad1b7e02c6 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 30 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.848817:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98560562d800 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826821.848820:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562d800/0xb3f83dad1b7e02c6 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 30 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.848830:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562d800/0xb3f83dad1b7e02c6 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 30 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.848836:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562d800/0xb3f83dad1b7e02c6 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 30 type: IBT flags: 0x20e400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.848841:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826821.848846:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562d800/0xb3f83dad1b7e02c6 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 30 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.848853:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562d800/0xb3f83dad1b7e02c6 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 30 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.848868:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562da00/0xb3f83dad1b7e02cd lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 29 type: IBT flags: 0x0 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.848876:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562da00/0xb3f83dad1b7e02cd lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 29 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.848883:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562da00/0xb3f83dad1b7e02cd lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 29 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.848888:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562da00/0xb3f83dad1b7e02cd lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 29 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.848895:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562da00/0xb3f83dad1b7e02cd lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 29 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.848900:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98560562da00 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826821.848903:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562da00/0xb3f83dad1b7e02cd lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 29 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.848913:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562da00/0xb3f83dad1b7e02cd lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 29 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.848921:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562da00/0xb3f83dad1b7e02cd lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 29 type: IBT flags: 0x20e400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.848928:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826821.848933:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562da00/0xb3f83dad1b7e02cd lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 29 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.848940:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562da00/0xb3f83dad1b7e02cd lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 29 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.848954:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562d200/0xb3f83dad1b7e02d4 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 28 type: IBT flags: 0x0 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.848961:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562d200/0xb3f83dad1b7e02d4 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 28 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.848968:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562d200/0xb3f83dad1b7e02d4 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 28 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.848975:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562d200/0xb3f83dad1b7e02d4 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 28 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.848981:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562d200/0xb3f83dad1b7e02d4 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 28 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.848986:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98560562d200 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826821.848989:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562d200/0xb3f83dad1b7e02d4 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 28 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.848995:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562d200/0xb3f83dad1b7e02d4 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 28 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.848999:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562d200/0xb3f83dad1b7e02d4 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 28 type: IBT flags: 0x20e400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849003:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826821.849006:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562d200/0xb3f83dad1b7e02d4 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 28 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849009:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562d200/0xb3f83dad1b7e02d4 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 28 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849020:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562c800/0xb3f83dad1b7e02db lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 27 type: IBT flags: 0x0 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849026:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562c800/0xb3f83dad1b7e02db lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 27 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849035:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562c800/0xb3f83dad1b7e02db lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 27 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849041:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562c800/0xb3f83dad1b7e02db lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 27 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849050:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562c800/0xb3f83dad1b7e02db lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 27 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849055:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98560562c800 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826821.849059:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562c800/0xb3f83dad1b7e02db lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 27 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849069:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562c800/0xb3f83dad1b7e02db lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 27 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849075:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562c800/0xb3f83dad1b7e02db lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 27 type: IBT flags: 0x20e400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849081:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826821.849084:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562c800/0xb3f83dad1b7e02db lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 27 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849088:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562c800/0xb3f83dad1b7e02db lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 27 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849099:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562c400/0xb3f83dad1b7e02e2 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 26 type: IBT flags: 0x0 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849103:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562c400/0xb3f83dad1b7e02e2 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 26 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849107:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562c400/0xb3f83dad1b7e02e2 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 26 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849111:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562c400/0xb3f83dad1b7e02e2 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 26 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849115:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562c400/0xb3f83dad1b7e02e2 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 26 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849118:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98560562c400 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826821.849119:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562c400/0xb3f83dad1b7e02e2 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 26 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849126:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562c400/0xb3f83dad1b7e02e2 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 26 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849130:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562c400/0xb3f83dad1b7e02e2 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 26 type: IBT flags: 0x20e400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849134:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826821.849136:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562c400/0xb3f83dad1b7e02e2 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 26 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849139:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562c400/0xb3f83dad1b7e02e2 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 26 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849149:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562c000/0xb3f83dad1b7e02e9 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 25 type: IBT flags: 0x0 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849155:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562c000/0xb3f83dad1b7e02e9 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 25 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849159:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562c000/0xb3f83dad1b7e02e9 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 25 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849164:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562c000/0xb3f83dad1b7e02e9 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 25 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849170:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562c000/0xb3f83dad1b7e02e9 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 25 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849175:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98560562c000 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826821.849178:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562c000/0xb3f83dad1b7e02e9 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 25 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849189:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562c000/0xb3f83dad1b7e02e9 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 25 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849195:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562c000/0xb3f83dad1b7e02e9 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 25 type: IBT flags: 0x20e400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849201:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826821.849204:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562c000/0xb3f83dad1b7e02e9 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 25 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849209:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562c000/0xb3f83dad1b7e02e9 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 25 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849220:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562c200/0xb3f83dad1b7e02f0 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 24 type: IBT flags: 0x0 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849237:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562c200/0xb3f83dad1b7e02f0 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 24 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849241:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562c200/0xb3f83dad1b7e02f0 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 24 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849245:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562c200/0xb3f83dad1b7e02f0 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 24 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849248:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562c200/0xb3f83dad1b7e02f0 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 24 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849251:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98560562c200 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826821.849253:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562c200/0xb3f83dad1b7e02f0 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 24 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849259:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562c200/0xb3f83dad1b7e02f0 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 24 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849264:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562c200/0xb3f83dad1b7e02f0 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 24 type: IBT flags: 0x20e400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849267:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826821.849271:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562c200/0xb3f83dad1b7e02f0 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 24 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849274:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562c200/0xb3f83dad1b7e02f0 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 24 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849287:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562d400/0xb3f83dad1b7e02f7 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 23 type: IBT flags: 0x0 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849291:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562d400/0xb3f83dad1b7e02f7 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 23 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849295:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562d400/0xb3f83dad1b7e02f7 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 23 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849298:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562d400/0xb3f83dad1b7e02f7 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 23 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849302:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562d400/0xb3f83dad1b7e02f7 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 23 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849304:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98560562d400 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826821.849306:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562d400/0xb3f83dad1b7e02f7 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 23 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849313:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562d400/0xb3f83dad1b7e02f7 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 23 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849317:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562d400/0xb3f83dad1b7e02f7 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 23 type: IBT flags: 0x20e400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849320:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826821.849322:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562d400/0xb3f83dad1b7e02f7 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 23 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849326:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562d400/0xb3f83dad1b7e02f7 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 23 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849333:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562ce00/0xb3f83dad1b7e02fe lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 22 type: IBT flags: 0x0 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849337:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562ce00/0xb3f83dad1b7e02fe lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 22 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849340:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562ce00/0xb3f83dad1b7e02fe lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 22 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849344:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562ce00/0xb3f83dad1b7e02fe lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 22 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849347:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562ce00/0xb3f83dad1b7e02fe lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 22 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849352:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98560562ce00 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826821.849355:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562ce00/0xb3f83dad1b7e02fe lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 22 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849365:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562ce00/0xb3f83dad1b7e02fe lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 22 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849371:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562ce00/0xb3f83dad1b7e02fe lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 22 type: IBT flags: 0x20e400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849377:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826821.849381:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562ce00/0xb3f83dad1b7e02fe lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 22 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849388:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562ce00/0xb3f83dad1b7e02fe lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 22 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849401:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562d600/0xb3f83dad1b7e0305 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 21 type: IBT flags: 0x0 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849407:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562d600/0xb3f83dad1b7e0305 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 21 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849412:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562d600/0xb3f83dad1b7e0305 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 21 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849415:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562d600/0xb3f83dad1b7e0305 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 21 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849421:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562d600/0xb3f83dad1b7e0305 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 21 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849427:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98560562d600 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826821.849430:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562d600/0xb3f83dad1b7e0305 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 21 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849440:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562d600/0xb3f83dad1b7e0305 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 21 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849446:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562d600/0xb3f83dad1b7e0305 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 21 type: IBT flags: 0x20e400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849452:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826821.849456:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562d600/0xb3f83dad1b7e0305 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 21 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849463:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562d600/0xb3f83dad1b7e0305 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 21 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849477:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562c600/0xb3f83dad1b7e030c lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 20 type: IBT flags: 0x0 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849484:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562c600/0xb3f83dad1b7e030c lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 20 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849490:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562c600/0xb3f83dad1b7e030c lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 20 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849496:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562c600/0xb3f83dad1b7e030c lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 20 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849503:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562c600/0xb3f83dad1b7e030c lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 20 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849509:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98560562c600 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826821.849512:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562c600/0xb3f83dad1b7e030c lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 20 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849522:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562c600/0xb3f83dad1b7e030c lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 20 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849528:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562c600/0xb3f83dad1b7e030c lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 20 type: IBT flags: 0x20e400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849533:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826821.849537:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562c600/0xb3f83dad1b7e030c lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 20 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849542:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562c600/0xb3f83dad1b7e030c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 20 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849560:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562d000/0xb3f83dad1b7e0313 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 19 type: IBT flags: 0x0 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849567:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562d000/0xb3f83dad1b7e0313 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 19 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849574:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562d000/0xb3f83dad1b7e0313 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 19 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849580:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562d000/0xb3f83dad1b7e0313 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 19 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849587:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562d000/0xb3f83dad1b7e0313 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 19 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849593:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98560562d000 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826821.849596:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562d000/0xb3f83dad1b7e0313 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 19 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849606:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562d000/0xb3f83dad1b7e0313 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 19 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849612:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562d000/0xb3f83dad1b7e0313 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 19 type: IBT flags: 0x20e400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849617:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826821.849620:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562d000/0xb3f83dad1b7e0313 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 19 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849624:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562d000/0xb3f83dad1b7e0313 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 19 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849638:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562cc00/0xb3f83dad1b7e031a lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 18 type: IBT flags: 0x0 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849644:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562cc00/0xb3f83dad1b7e031a lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 18 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849650:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562cc00/0xb3f83dad1b7e031a lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 18 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849656:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562cc00/0xb3f83dad1b7e031a lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 18 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849663:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562cc00/0xb3f83dad1b7e031a lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 18 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849669:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98560562cc00 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826821.849672:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562cc00/0xb3f83dad1b7e031a lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 18 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849683:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562cc00/0xb3f83dad1b7e031a lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 18 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849689:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562cc00/0xb3f83dad1b7e031a lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 18 type: IBT flags: 0x20e400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849713:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826821.849717:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562cc00/0xb3f83dad1b7e031a lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 18 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849721:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562cc00/0xb3f83dad1b7e031a lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 18 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849733:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562dc00/0xb3f83dad1b7e0321 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 17 type: IBT flags: 0x0 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849737:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562dc00/0xb3f83dad1b7e0321 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 17 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849740:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562dc00/0xb3f83dad1b7e0321 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 17 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849745:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562dc00/0xb3f83dad1b7e0321 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 17 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849750:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562dc00/0xb3f83dad1b7e0321 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 17 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849755:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff98560562dc00 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826821.849758:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562dc00/0xb3f83dad1b7e0321 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 17 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849767:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562dc00/0xb3f83dad1b7e0321 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 17 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849773:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562dc00/0xb3f83dad1b7e0321 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 17 type: IBT flags: 0x20e400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849780:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826821.849785:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562dc00/0xb3f83dad1b7e0321 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 17 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849792:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560562dc00/0xb3f83dad1b7e0321 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 17 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849807:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0600/0xb3f83dad1b7e0328 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 16 type: IBT flags: 0x0 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849812:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0600/0xb3f83dad1b7e0328 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 16 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849816:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0600/0xb3f83dad1b7e0328 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 16 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849819:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0600/0xb3f83dad1b7e0328 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 16 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849822:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0600/0xb3f83dad1b7e0328 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 16 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849825:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985610dc0600 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826821.849827:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0600/0xb3f83dad1b7e0328 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 16 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849834:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0600/0xb3f83dad1b7e0328 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 16 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849840:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0600/0xb3f83dad1b7e0328 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 16 type: IBT flags: 0x20e400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849846:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826821.849851:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0600/0xb3f83dad1b7e0328 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 16 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849858:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0600/0xb3f83dad1b7e0328 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 16 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849873:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1000/0xb3f83dad1b7e032f lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 15 type: IBT flags: 0x0 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849880:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1000/0xb3f83dad1b7e032f lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 15 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849885:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1000/0xb3f83dad1b7e032f lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 15 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849891:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1000/0xb3f83dad1b7e032f lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 15 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849897:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1000/0xb3f83dad1b7e032f lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 15 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849902:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985610dc1000 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826821.849906:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1000/0xb3f83dad1b7e032f lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 15 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849915:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1000/0xb3f83dad1b7e032f lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 15 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849922:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1000/0xb3f83dad1b7e032f lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 15 type: IBT flags: 0x20e400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849928:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826821.849931:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1000/0xb3f83dad1b7e032f lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 15 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849937:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1000/0xb3f83dad1b7e032f lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 15 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849950:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0c00/0xb3f83dad1b7e0336 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 14 type: IBT flags: 0x0 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849957:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0c00/0xb3f83dad1b7e0336 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 14 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849962:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0c00/0xb3f83dad1b7e0336 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 14 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849969:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0c00/0xb3f83dad1b7e0336 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 14 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849975:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0c00/0xb3f83dad1b7e0336 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 14 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849980:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985610dc0c00 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826821.849983:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0c00/0xb3f83dad1b7e0336 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 14 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849993:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0c00/0xb3f83dad1b7e0336 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 14 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.849999:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0c00/0xb3f83dad1b7e0336 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 14 type: IBT flags: 0x20e400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850004:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826821.850008:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0c00/0xb3f83dad1b7e0336 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 14 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850014:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0c00/0xb3f83dad1b7e0336 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 14 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850025:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1c00/0xb3f83dad1b7e033d lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 13 type: IBT flags: 0x0 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850031:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1c00/0xb3f83dad1b7e033d lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 13 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850037:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1c00/0xb3f83dad1b7e033d lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 13 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850043:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1c00/0xb3f83dad1b7e033d lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 13 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850051:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1c00/0xb3f83dad1b7e033d lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 13 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850056:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985610dc1c00 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826821.850059:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1c00/0xb3f83dad1b7e033d lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 13 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850069:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1c00/0xb3f83dad1b7e033d lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 13 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850076:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1c00/0xb3f83dad1b7e033d lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 13 type: IBT flags: 0x20e400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850082:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826821.850084:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1c00/0xb3f83dad1b7e033d lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 13 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850089:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1c00/0xb3f83dad1b7e033d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 13 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850098:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0a00/0xb3f83dad1b7e0344 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 12 type: IBT flags: 0x0 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850102:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0a00/0xb3f83dad1b7e0344 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 12 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850106:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0a00/0xb3f83dad1b7e0344 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 12 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850109:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0a00/0xb3f83dad1b7e0344 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 12 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850113:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0a00/0xb3f83dad1b7e0344 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 12 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850117:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985610dc0a00 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826821.850119:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0a00/0xb3f83dad1b7e0344 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 12 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850125:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0a00/0xb3f83dad1b7e0344 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 12 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850131:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0a00/0xb3f83dad1b7e0344 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 12 type: IBT flags: 0x20e400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850134:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826821.850137:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0a00/0xb3f83dad1b7e0344 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 12 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850140:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0a00/0xb3f83dad1b7e0344 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 12 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850150:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1800/0xb3f83dad1b7e034b lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 11 type: IBT flags: 0x0 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850155:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1800/0xb3f83dad1b7e034b lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 11 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850160:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1800/0xb3f83dad1b7e034b lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 11 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850165:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1800/0xb3f83dad1b7e034b lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 11 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850172:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1800/0xb3f83dad1b7e034b lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 11 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850178:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985610dc1800 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826821.850181:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1800/0xb3f83dad1b7e034b lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 11 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850192:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1800/0xb3f83dad1b7e034b lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 11 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850198:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1800/0xb3f83dad1b7e034b lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 11 type: IBT flags: 0x20e400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850202:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826821.850206:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1800/0xb3f83dad1b7e034b lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 11 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850210:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1800/0xb3f83dad1b7e034b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 11 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850220:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1a00/0xb3f83dad1b7e0352 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 10 type: IBT flags: 0x0 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850238:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1a00/0xb3f83dad1b7e0352 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 10 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850242:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1a00/0xb3f83dad1b7e0352 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 10 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850246:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1a00/0xb3f83dad1b7e0352 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 10 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850250:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1a00/0xb3f83dad1b7e0352 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 10 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850253:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985610dc1a00 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826821.850256:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1a00/0xb3f83dad1b7e0352 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 10 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850263:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1a00/0xb3f83dad1b7e0352 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 10 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850267:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1a00/0xb3f83dad1b7e0352 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 10 type: IBT flags: 0x20e400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850270:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826821.850272:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1a00/0xb3f83dad1b7e0352 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 10 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850275:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1a00/0xb3f83dad1b7e0352 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 10 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850282:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1200/0xb3f83dad1b7e0359 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 9 type: IBT flags: 0x0 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850286:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1200/0xb3f83dad1b7e0359 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 9 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850290:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1200/0xb3f83dad1b7e0359 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 9 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850293:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1200/0xb3f83dad1b7e0359 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 9 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850296:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1200/0xb3f83dad1b7e0359 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 9 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850299:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985610dc1200 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826821.850300:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1200/0xb3f83dad1b7e0359 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 9 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850306:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1200/0xb3f83dad1b7e0359 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 9 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850310:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1200/0xb3f83dad1b7e0359 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 9 type: IBT flags: 0x20e400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850312:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826821.850314:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1200/0xb3f83dad1b7e0359 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 9 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850317:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1200/0xb3f83dad1b7e0359 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 9 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850326:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0800/0xb3f83dad1b7e0360 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 8 type: IBT flags: 0x0 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850329:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0800/0xb3f83dad1b7e0360 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 8 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850333:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0800/0xb3f83dad1b7e0360 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 8 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850336:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0800/0xb3f83dad1b7e0360 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 8 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850341:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0800/0xb3f83dad1b7e0360 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 8 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850346:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985610dc0800 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826821.850349:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0800/0xb3f83dad1b7e0360 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 8 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850358:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0800/0xb3f83dad1b7e0360 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 8 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850364:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0800/0xb3f83dad1b7e0360 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 8 type: IBT flags: 0x20e400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850371:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826821.850374:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0800/0xb3f83dad1b7e0360 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 8 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850380:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0800/0xb3f83dad1b7e0360 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 8 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850394:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0400/0xb3f83dad1b7e0367 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 7 type: IBT flags: 0x0 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850400:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0400/0xb3f83dad1b7e0367 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 7 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850407:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0400/0xb3f83dad1b7e0367 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 7 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850413:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0400/0xb3f83dad1b7e0367 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 7 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850419:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0400/0xb3f83dad1b7e0367 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 7 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850425:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985610dc0400 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826821.850429:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0400/0xb3f83dad1b7e0367 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 7 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850438:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0400/0xb3f83dad1b7e0367 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 7 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850444:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0400/0xb3f83dad1b7e0367 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 7 type: IBT flags: 0x20e400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850451:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826821.850455:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0400/0xb3f83dad1b7e0367 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 7 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850461:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0400/0xb3f83dad1b7e0367 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 7 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850477:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0000/0xb3f83dad1b7e036e lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 6 type: IBT flags: 0x0 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850484:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0000/0xb3f83dad1b7e036e lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 6 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850491:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0000/0xb3f83dad1b7e036e lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 6 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850497:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0000/0xb3f83dad1b7e036e lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 6 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850504:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0000/0xb3f83dad1b7e036e lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 6 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850510:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985610dc0000 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826821.850512:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0000/0xb3f83dad1b7e036e lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 6 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850521:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0000/0xb3f83dad1b7e036e lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 6 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850527:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0000/0xb3f83dad1b7e036e lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 6 type: IBT flags: 0x20e400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850534:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826821.850538:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0000/0xb3f83dad1b7e036e lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 6 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850544:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0000/0xb3f83dad1b7e036e lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 6 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850557:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0200/0xb3f83dad1b7e0375 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x0 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850564:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0200/0xb3f83dad1b7e0375 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850570:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0200/0xb3f83dad1b7e0375 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850576:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0200/0xb3f83dad1b7e0375 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850583:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0200/0xb3f83dad1b7e0375 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850589:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985610dc0200 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826821.850593:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0200/0xb3f83dad1b7e0375 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 5 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850603:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0200/0xb3f83dad1b7e0375 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 5 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850608:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0200/0xb3f83dad1b7e0375 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 5 type: IBT flags: 0x20e400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850613:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826821.850616:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0200/0xb3f83dad1b7e0375 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 5 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850619:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0200/0xb3f83dad1b7e0375 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 5 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850632:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1400/0xb3f83dad1b7e037c lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x0 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850640:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1400/0xb3f83dad1b7e037c lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850646:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1400/0xb3f83dad1b7e037c lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850652:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1400/0xb3f83dad1b7e037c lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850658:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1400/0xb3f83dad1b7e037c lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850663:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985610dc1400 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826821.850667:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1400/0xb3f83dad1b7e037c lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 4 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850676:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1400/0xb3f83dad1b7e037c lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 4 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850683:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1400/0xb3f83dad1b7e037c lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 4 type: IBT flags: 0x20e400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850689:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826821.850727:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1400/0xb3f83dad1b7e037c lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850734:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1400/0xb3f83dad1b7e037c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850748:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0e00/0xb3f83dad1b7e0383 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x0 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850754:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0e00/0xb3f83dad1b7e0383 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850762:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0e00/0xb3f83dad1b7e0383 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850768:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0e00/0xb3f83dad1b7e0383 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850774:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0e00/0xb3f83dad1b7e0383 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850779:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985610dc0e00 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826821.850782:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0e00/0xb3f83dad1b7e0383 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 3 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850791:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0e00/0xb3f83dad1b7e0383 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 3 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850796:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0e00/0xb3f83dad1b7e0383 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 3 type: IBT flags: 0x20e400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850801:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826821.850806:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0e00/0xb3f83dad1b7e0383 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850812:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0e00/0xb3f83dad1b7e0383 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850825:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1e00/0xb3f83dad1b7e038a lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850831:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1e00/0xb3f83dad1b7e038a lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850837:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1e00/0xb3f83dad1b7e038a lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850843:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1e00/0xb3f83dad1b7e038a lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850849:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1e00/0xb3f83dad1b7e038a lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850854:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985610dc1e00 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826821.850858:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1e00/0xb3f83dad1b7e038a lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 2 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850867:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1e00/0xb3f83dad1b7e038a lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 2 type: IBT flags: 0x206400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850873:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1e00/0xb3f83dad1b7e038a lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 2 type: IBT flags: 0x20e400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850880:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826821.850884:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1e00/0xb3f83dad1b7e038a lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826821.850890:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1e00/0xb3f83dad1b7e038a lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826821.850929:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee2d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669225546752:192.168.203.114@tcp:63:ll_sa.0 00010000:00010000:1.0:1787826823.989805:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826824.368025:0:78638:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee0f00 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78638:1874669225546880:192.168.203.114@tcp:101:createmany.0 00010000:00010000:1.0:1787826824.368046:0:78638:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0400/0xb3f83dad1b7e0391 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a11c4 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.368057:0:78638:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0400/0xb3f83dad1b7e0391 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a11c4 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.368069:0:78638:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0400/0xb3f83dad1b7e0391 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a11c4 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.368079:0:78638:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0400/0xb3f83dad1b7e0391 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a11c4 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.368090:0:78638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0400/0xb3f83dad1b7e0391 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a11c4 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.368098:0:78638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0400/0xb3f83dad1b7e0391 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a11c4 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.368106:0:78638:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0400/0xb3f83dad1b7e0391 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a11c4 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826824.368122:0:78638:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff985602ee0f00 x1874669225546880/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826840 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826824.368135:0:78638:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0400/0xb3f83dad1b7e0391 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a11c4 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.368146:0:78638:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:1.0:1787826824.368151:0:78638:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:1.0:1787826824.368174:0:78638:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:1.0:1787826824.368180:0:78638:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e0391 00000080:00010000:1.0:1787826824.368185:0:78638:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826824.368188:0:78638:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0391 from it ffffab3ac167bbf8 00010000:00010000:1.0:1787826824.368191:0:78638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0400/0xb3f83dad1b7e0391 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a11c4 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.368200:0:78638:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0400/0xb3f83dad1b7e0391 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a11c4 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826824.368233:0:78638:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787826824.368249:0:78638:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826824.368261:0:78638:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826824.368269:0:78638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0400/0xb3f83dad1b7e0391 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a11c4 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826824.368278:0:78638:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0400/0xb3f83dad1b7e0391 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a11c4 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826824.368299:0:78638:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826824.368303:0:78638:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826824.368306:0:78638:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e0391 00000080:00010000:1.0:1787826824.368310:0:78638:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0391 from it ffffab3ac167bb38 00010000:00010000:1.0:1787826824.368313:0:78638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0400/0xb3f83dad1b7e0391 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a11c4 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826824.368320:0:78638:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0400/0xb3f83dad1b7e0391 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a11c4 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826824.368329:0:78638:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787826824.368334:0:78638:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f33b.replay-single-0, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), file ffff985611f22900, open_flags 8042, mode 81a4 opened 0 00000080:00200000:1.0:1787826824.368343:0:78638:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f33b.replay-single-0, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=open|creat 00000002:00010000:1.0:1787826824.368362:0:78638:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f33b.replay-single-0,[0x200015bc1:0xb:0x0]) in obj [0x200000007:0x1:0x0], intent: open|creat flags 0100103 00010000:00010000:1.0:1787826824.368378:0:78638:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0400/0xb3f83dad1b7e0391 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x5aa6d504d91a11c4 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826824.368415:0:78638:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0400/0xb3f83dad1b7e0391 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a11c4 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826824.368422:0:78638:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826824.368425:0:78638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0400/0xb3f83dad1b7e0391 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a11c4 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826824.368451:0:78638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a200/0xb3f83dad1b7e0398 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 78638 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.368462:0:78638:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a200/0xb3f83dad1b7e0398 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78638 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:1.0:1787826824.368471:0:78638:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:1.0:1787826824.368477:0:78638:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610e31e00 x1874669225547136/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:1.0:1787826824.368486:0:78638:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a200/0xb3f83dad1b7e0398 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78638 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826824.368501:0:78638:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e31e00 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78638:1874669225547136:192.168.203.114@tcp:101:createmany.0 00000100:00100000:1.0:1787826824.368600:0:78638:0:(client.c:2715:ptlrpc_set_wait()) set ffff9856108f2a80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826824.370654:0:78638:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e31e00 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78638:1874669225547136:192.168.203.114@tcp:101:createmany.0 00010000:00010000:1.0:1787826824.370673:0:78638:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a200/0xb3f83dad1b7e0398 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a11e0 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.370713:0:78638:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a200/0xb3f83dad1b7e0398 lrc: 4/0,1 mode: --/PR res: [0x200015bc1:0xb:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a11e0 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.370722:0:78638:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a200/0xb3f83dad1b7e0398 lrc: 4/0,1 mode: --/PR res: [0x200015bc1:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a11e0 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.370731:0:78638:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a200/0xb3f83dad1b7e0398 lrc: 4/0,1 mode: PR/PR res: [0x200015bc1:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a11e0 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.370740:0:78638:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a200/0xb3f83dad1b7e0398 lrc: 4/0,1 mode: PR/PR res: [0x200015bc1:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a11e0 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.370757:0:78638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a200/0xb3f83dad1b7e0398 lrc: 5/1,1 mode: PR/PR res: [0x200015bc1:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a11e0 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.370765:0:78638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a200/0xb3f83dad1b7e0398 lrc: 5/1,1 mode: PR/PR res: [0x200015bc1:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a11e0 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.370772:0:78638:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a200/0xb3f83dad1b7e0398 lrc: 4/1,0 mode: PR/PR res: [0x200015bc1:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a11e0 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826824.370782:0:78638:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff985610e31e00 x1874669225547136/t193273528351(193273528351) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/664 e 0 to 0 dl 1787826840 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:1.0:1787826824.370794:0:78638:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff985610e31e00 x1874669225547136/t193273528351(193273528351) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/664 e 0 to 0 dl 1787826840 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826824.370805:0:78638:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a200/0xb3f83dad1b7e0398 lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a11e0 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:1.0:1787826824.370815:0:78638:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a200/0xb3f83dad1b7e0398 lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a11e0 expref: -99 pid: 78638 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.370823:0:78638:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8590023617/11 (0 0) 00000080:00200000:1.0:1787826824.370828:0:78638:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:1.0:1787826824.370837:0:78638:0:(namei.c:1628:ll_create_it()) VFS Op:name=f33b.replay-single-0, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=open|creat 00000080:00200000:1.0:1787826824.370858:0:78638:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200015bc1:0xb:0x0](ffff985611674b08) 00000080:00200000:1.0:1787826824.370915:0:78638:0:(file.c:6854:ll_layout_conf()) [0x200015bc1:0xb:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:1.0:1787826824.370919:0:78638:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x200015bc1:0xb:0x0](ffff985611674b08) 00000080:00010000:1.0:1787826824.370923:0:78638:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200015bc1:0xb:0x0](ffff985611674b08) for lock 0xb3f83dad1b7e0398 00000080:00010000:1.0:1787826824.370929:0:78638:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200015bc1:0xb:0x0](ffff985611674b08) 00000080:00200008:1.0:1787826824.370939:0:78638:0:(file.c:1090:ll_file_open()) START file f33b.replay-single-0:[0x200015bc1:0xb:0x0](ffff985611674b08), flags 100102 00000002:00100000:1.0:1787826824.370954:0:78638:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff985610e31e00 x1874669225547136/t193273528351(193273528351) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826840 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:1.0:1787826824.370982:0:78638:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0398 from it ffff985610e3fd20 00010000:00010000:1.0:1787826824.370986:0:78638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a200/0xb3f83dad1b7e0398 lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a11e0 expref: -99 pid: 78638 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.370994:0:78638:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a200/0xb3f83dad1b7e0398 lrc: 2/0,0 mode: PR/PR res: [0x200015bc1:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a11e0 expref: -99 pid: 78638 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826824.371008:0:78638:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200015bc1:0xb:0x0](ffff985611674b08), inode mode 81a4 mask 40 00000080:00200008:1.0:1787826824.371029:0:78638:0:(file.c:479:ll_file_release()) START file f33b.replay-single-0:[0x200015bc1:0xb:0x0](ffff985611674b08), flags 100002 00000002:00100000:1.0:1787826824.371055:0:78638:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff985610e31e00 x1874669225547136/t193273528351(193273528351) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826840 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:1.0:1787826824.371064:0:78638:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:1.0:1787826824.371069:0:78638:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610e30f00 x1874669225547392/t0(0) o35->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:1.0:1787826824.371084:0:78638:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e30f00 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78638:1874669225547392:192.168.203.114@tcp:35:createmany.0 00000100:00100000:1.0:1787826824.371130:0:78638:0:(client.c:2715:ptlrpc_set_wait()) set ffff9856108f2540 going to sleep for 11 seconds 00000100:00100000:1.0:1787826824.372063:0:78638:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e30f00 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78638:1874669225547392:192.168.203.114@tcp:35:createmany.0 00000002:00100000:1.0:1787826824.372079:0:78638:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff985610e31e00 x1874669225547136/t193273528351(193273528351) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826840 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:1.0:1787826824.372125:0:78638:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826824.372143:0:78638:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826824.372170:0:78638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e039f lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 78638 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.372178:0:78638:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e039f lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78638 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.372190:0:78638:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e039f lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78638 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826824.372202:0:78638:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e321c0 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78638:1874669225547520:192.168.203.114@tcp:101:createmany.0 00000100:00100000:1.0:1787826824.372253:0:78638:0:(client.c:2715:ptlrpc_set_wait()) set ffff9856108f2540 going to sleep for 11 seconds 00000100:00100000:1.0:1787826824.373151:0:78638:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e321c0 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78638:1874669225547520:192.168.203.114@tcp:101:createmany.0 00010000:00010000:1.0:1787826824.373163:0:78638:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e039f lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a11f5 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.373173:0:78638:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e039f lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a11f5 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.373181:0:78638:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e039f lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a11f5 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.373199:0:78638:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e039f lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a11f5 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.373220:0:78638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e039f lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a11f5 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.373228:0:78638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e039f lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a11f5 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.373234:0:78638:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e039f lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a11f5 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826824.373243:0:78638:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff985610e321c0 x1874669225547520/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826840 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826824.373258:0:78638:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e039f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a11f5 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.373265:0:78638:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:1.0:1787826824.373269:0:78638:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:1.0:1787826824.373284:0:78638:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:1.0:1787826824.373289:0:78638:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e039f 00000080:00010000:1.0:1787826824.373299:0:78638:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826824.373302:0:78638:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e039f from it ffffab3ac167bbf8 00010000:00010000:1.0:1787826824.373305:0:78638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e039f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a11f5 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.373312:0:78638:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e039f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a11f5 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826824.373328:0:78638:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787826824.373339:0:78638:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826824.373359:0:78638:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826824.373366:0:78638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e039f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a11f5 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826824.373379:0:78638:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e039f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a11f5 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826824.373385:0:78638:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826824.373389:0:78638:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826824.373391:0:78638:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e039f 00000080:00010000:1.0:1787826824.373395:0:78638:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e039f from it ffffab3ac167bb38 00010000:00010000:1.0:1787826824.373398:0:78638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e039f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a11f5 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826824.373404:0:78638:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e039f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a11f5 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826824.373411:0:78638:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787826824.373415:0:78638:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f33b.replay-single-1, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), file ffff985611f19500, open_flags 8042, mode 81a4 opened 0 00000080:00200000:1.0:1787826824.373420:0:78638:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f33b.replay-single-1, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=open|creat 00000002:00010000:1.0:1787826824.373434:0:78638:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f33b.replay-single-1,[0x200015bc1:0xc:0x0]) in obj [0x200000007:0x1:0x0], intent: open|creat flags 0100103 00010000:00010000:1.0:1787826824.373442:0:78638:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e039f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x5aa6d504d91a11f5 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826824.373469:0:78638:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e039f lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a11f5 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826824.373476:0:78638:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826824.373479:0:78638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e039f lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a11f5 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826824.373506:0:78638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a000/0xb3f83dad1b7e03a6 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 78638 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.373512:0:78638:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a000/0xb3f83dad1b7e03a6 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78638 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:1.0:1787826824.373519:0:78638:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:1.0:1787826824.373525:0:78638:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610e321c0 x1874669225547776/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:1.0:1787826824.373533:0:78638:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a000/0xb3f83dad1b7e03a6 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78638 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826824.373544:0:78638:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e321c0 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78638:1874669225547776:192.168.203.114@tcp:101:createmany.0 00000100:00100000:1.0:1787826824.373631:0:78638:0:(client.c:2715:ptlrpc_set_wait()) set ffff9856108f2540 going to sleep for 11 seconds 00000100:00100000:1.0:1787826824.375445:0:78638:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e321c0 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78638:1874669225547776:192.168.203.114@tcp:101:createmany.0 00010000:00010000:1.0:1787826824.375462:0:78638:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a000/0xb3f83dad1b7e03a6 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1211 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.375482:0:78638:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a000/0xb3f83dad1b7e03a6 lrc: 4/0,1 mode: --/PR res: [0x200015bc1:0xc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1211 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.375491:0:78638:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a000/0xb3f83dad1b7e03a6 lrc: 4/0,1 mode: --/PR res: [0x200015bc1:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1211 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.375499:0:78638:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a000/0xb3f83dad1b7e03a6 lrc: 4/0,1 mode: PR/PR res: [0x200015bc1:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1211 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.375522:0:78638:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a000/0xb3f83dad1b7e03a6 lrc: 4/0,1 mode: PR/PR res: [0x200015bc1:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1211 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.375533:0:78638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a000/0xb3f83dad1b7e03a6 lrc: 5/1,1 mode: PR/PR res: [0x200015bc1:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1211 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.375541:0:78638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a000/0xb3f83dad1b7e03a6 lrc: 5/1,1 mode: PR/PR res: [0x200015bc1:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1211 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.375548:0:78638:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a000/0xb3f83dad1b7e03a6 lrc: 4/1,0 mode: PR/PR res: [0x200015bc1:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1211 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826824.375567:0:78638:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff985610e321c0 x1874669225547776/t193273528353(193273528353) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/664 e 0 to 0 dl 1787826840 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:1.0:1787826824.375578:0:78638:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff985610e321c0 x1874669225547776/t193273528353(193273528353) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/664 e 0 to 0 dl 1787826840 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826824.375589:0:78638:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a000/0xb3f83dad1b7e03a6 lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1211 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:1.0:1787826824.375600:0:78638:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a000/0xb3f83dad1b7e03a6 lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1211 expref: -99 pid: 78638 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.375609:0:78638:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8590023617/12 (0 0) 00000080:00200000:1.0:1787826824.375613:0:78638:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:1.0:1787826824.375623:0:78638:0:(namei.c:1628:ll_create_it()) VFS Op:name=f33b.replay-single-1, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=open|creat 00000080:00200000:1.0:1787826824.375642:0:78638:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200015bc1:0xc:0x0](ffff9856116742c8) 00000080:00200000:1.0:1787826824.375726:0:78638:0:(file.c:6854:ll_layout_conf()) [0x200015bc1:0xc:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:1.0:1787826824.375731:0:78638:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x200015bc1:0xc:0x0](ffff9856116742c8) 00000080:00010000:1.0:1787826824.375735:0:78638:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200015bc1:0xc:0x0](ffff9856116742c8) for lock 0xb3f83dad1b7e03a6 00000080:00010000:1.0:1787826824.375741:0:78638:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200015bc1:0xc:0x0](ffff9856116742c8) 00000080:00200008:1.0:1787826824.375751:0:78638:0:(file.c:1090:ll_file_open()) START file f33b.replay-single-1:[0x200015bc1:0xc:0x0](ffff9856116742c8), flags 100102 00000002:00100000:1.0:1787826824.375766:0:78638:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff985610e321c0 x1874669225547776/t193273528353(193273528353) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826840 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:1.0:1787826824.375793:0:78638:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e03a6 from it ffff985610e3fd20 00010000:00010000:1.0:1787826824.375797:0:78638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a000/0xb3f83dad1b7e03a6 lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a1211 expref: -99 pid: 78638 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.375806:0:78638:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a000/0xb3f83dad1b7e03a6 lrc: 2/0,0 mode: PR/PR res: [0x200015bc1:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a1211 expref: -99 pid: 78638 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826824.375820:0:78638:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200015bc1:0xc:0x0](ffff9856116742c8), inode mode 81a4 mask 40 00000080:00200008:1.0:1787826824.375842:0:78638:0:(file.c:479:ll_file_release()) START file f33b.replay-single-1:[0x200015bc1:0xc:0x0](ffff9856116742c8), flags 100002 00000002:00100000:1.0:1787826824.375869:0:78638:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff985610e321c0 x1874669225547776/t193273528353(193273528353) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826840 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:1.0:1787826824.375878:0:78638:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:1.0:1787826824.375883:0:78638:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610e32580 x1874669225548032/t0(0) o35->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:1.0:1787826824.375899:0:78638:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e32580 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78638:1874669225548032:192.168.203.114@tcp:35:createmany.0 00000100:00100000:1.0:1787826824.376403:0:78638:0:(client.c:2715:ptlrpc_set_wait()) set ffff9856108f2300 going to sleep for 11 seconds 00000100:00100000:1.0:1787826824.377441:0:78638:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e32580 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78638:1874669225548032:192.168.203.114@tcp:35:createmany.0 00000002:00100000:1.0:1787826824.377465:0:78638:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff985610e321c0 x1874669225547776/t193273528353(193273528353) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826840 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:1.0:1787826824.377532:0:78638:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826824.377545:0:78638:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826824.377579:0:78638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ba00/0xb3f83dad1b7e03ad lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 78638 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.377588:0:78638:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ba00/0xb3f83dad1b7e03ad lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78638 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.377598:0:78638:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ba00/0xb3f83dad1b7e03ad lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78638 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826824.377622:0:78638:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e32940 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78638:1874669225548160:192.168.203.114@tcp:101:createmany.0 00000100:00100000:1.0:1787826824.377666:0:78638:0:(client.c:2715:ptlrpc_set_wait()) set ffff9856108f2300 going to sleep for 11 seconds 00000100:00100000:1.0:1787826824.379302:0:78638:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e32940 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78638:1874669225548160:192.168.203.114@tcp:101:createmany.0 00010000:00010000:1.0:1787826824.379318:0:78638:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ba00/0xb3f83dad1b7e03ad lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1226 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.379328:0:78638:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ba00/0xb3f83dad1b7e03ad lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1226 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.379337:0:78638:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ba00/0xb3f83dad1b7e03ad lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1226 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.379355:0:78638:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ba00/0xb3f83dad1b7e03ad lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1226 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.379370:0:78638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ba00/0xb3f83dad1b7e03ad lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1226 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.379376:0:78638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ba00/0xb3f83dad1b7e03ad lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1226 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.379388:0:78638:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ba00/0xb3f83dad1b7e03ad lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1226 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826824.379397:0:78638:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff985610e32940 x1874669225548160/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826840 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826824.379408:0:78638:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ba00/0xb3f83dad1b7e03ad lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1226 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.379417:0:78638:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:1.0:1787826824.379423:0:78638:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:1.0:1787826824.379442:0:78638:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:1.0:1787826824.379447:0:78638:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e03ad 00000080:00010000:1.0:1787826824.379452:0:78638:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826824.379455:0:78638:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e03ad from it ffffab3ac167bbf8 00010000:00010000:1.0:1787826824.379458:0:78638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ba00/0xb3f83dad1b7e03ad lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1226 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.379465:0:78638:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ba00/0xb3f83dad1b7e03ad lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1226 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826824.379487:0:78638:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787826824.379501:0:78638:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826824.379511:0:78638:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826824.379517:0:78638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ba00/0xb3f83dad1b7e03ad lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1226 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826824.379526:0:78638:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ba00/0xb3f83dad1b7e03ad lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1226 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826824.379533:0:78638:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826824.379537:0:78638:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826824.379540:0:78638:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e03ad 00000080:00010000:1.0:1787826824.379544:0:78638:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e03ad from it ffffab3ac167bb38 00010000:00010000:1.0:1787826824.379547:0:78638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ba00/0xb3f83dad1b7e03ad lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1226 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826824.379554:0:78638:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ba00/0xb3f83dad1b7e03ad lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1226 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826824.379561:0:78638:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787826824.379566:0:78638:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f33b.replay-single-2, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), file ffff985611f19c80, open_flags 8042, mode 81a4 opened 0 00000080:00200000:1.0:1787826824.379573:0:78638:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f33b.replay-single-2, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=open|creat 00000002:00010000:1.0:1787826824.379590:0:78638:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f33b.replay-single-2,[0x200015bc1:0xd:0x0]) in obj [0x200000007:0x1:0x0], intent: open|creat flags 0100103 00010000:00010000:1.0:1787826824.379599:0:78638:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ba00/0xb3f83dad1b7e03ad lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x5aa6d504d91a1226 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826824.379637:0:78638:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ba00/0xb3f83dad1b7e03ad lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a1226 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826824.379643:0:78638:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826824.379646:0:78638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9ba00/0xb3f83dad1b7e03ad lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a1226 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826824.379670:0:78638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e03b4 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 78638 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.379677:0:78638:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e03b4 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78638 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:1.0:1787826824.379686:0:78638:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:1.0:1787826824.379711:0:78638:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610e32940 x1874669225548416/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:1.0:1787826824.379720:0:78638:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e03b4 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78638 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826824.379735:0:78638:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e32940 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78638:1874669225548416:192.168.203.114@tcp:101:createmany.0 00000100:00100000:1.0:1787826824.379831:0:78638:0:(client.c:2715:ptlrpc_set_wait()) set ffff9856108f2300 going to sleep for 11 seconds 00000100:00100000:1.0:1787826824.381511:0:78638:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e32940 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78638:1874669225548416:192.168.203.114@tcp:101:createmany.0 00010000:00010000:1.0:1787826824.381525:0:78638:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e03b4 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1242 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.381540:0:78638:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e03b4 lrc: 4/0,1 mode: --/PR res: [0x200015bc1:0xd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1242 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.381550:0:78638:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e03b4 lrc: 4/0,1 mode: --/PR res: [0x200015bc1:0xd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1242 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.381555:0:78638:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e03b4 lrc: 4/0,1 mode: PR/PR res: [0x200015bc1:0xd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1242 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.381561:0:78638:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e03b4 lrc: 4/0,1 mode: PR/PR res: [0x200015bc1:0xd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1242 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.381568:0:78638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e03b4 lrc: 5/1,1 mode: PR/PR res: [0x200015bc1:0xd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1242 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.381572:0:78638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e03b4 lrc: 5/1,1 mode: PR/PR res: [0x200015bc1:0xd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1242 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.381576:0:78638:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e03b4 lrc: 4/1,0 mode: PR/PR res: [0x200015bc1:0xd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1242 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826824.381582:0:78638:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff985610e32940 x1874669225548416/t193273528355(193273528355) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/664 e 0 to 0 dl 1787826840 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:1.0:1787826824.381590:0:78638:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff985610e32940 x1874669225548416/t193273528355(193273528355) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/664 e 0 to 0 dl 1787826840 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826824.381598:0:78638:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e03b4 lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0xd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1242 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:1.0:1787826824.381604:0:78638:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e03b4 lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0xd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1242 expref: -99 pid: 78638 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.381618:0:78638:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8590023617/13 (0 0) 00000080:00200000:1.0:1787826824.381622:0:78638:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:1.0:1787826824.381631:0:78638:0:(namei.c:1628:ll_create_it()) VFS Op:name=f33b.replay-single-2, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=open|creat 00000080:00200000:1.0:1787826824.381654:0:78638:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200015bc1:0xd:0x0](ffff9856116763c8) 00000080:00200000:1.0:1787826824.381758:0:78638:0:(file.c:6854:ll_layout_conf()) [0x200015bc1:0xd:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:1.0:1787826824.381763:0:78638:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x200015bc1:0xd:0x0](ffff9856116763c8) 00000080:00010000:1.0:1787826824.381766:0:78638:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200015bc1:0xd:0x0](ffff9856116763c8) for lock 0xb3f83dad1b7e03b4 00000080:00010000:1.0:1787826824.381774:0:78638:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200015bc1:0xd:0x0](ffff9856116763c8) 00000080:00200008:1.0:1787826824.381793:0:78638:0:(file.c:1090:ll_file_open()) START file f33b.replay-single-2:[0x200015bc1:0xd:0x0](ffff9856116763c8), flags 100102 00000002:00100000:1.0:1787826824.381812:0:78638:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff985610e32940 x1874669225548416/t193273528355(193273528355) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826840 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:1.0:1787826824.381849:0:78638:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e03b4 from it ffff985610e3fd20 00010000:00010000:1.0:1787826824.381854:0:78638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e03b4 lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0xd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a1242 expref: -99 pid: 78638 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.381863:0:78638:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9aa00/0xb3f83dad1b7e03b4 lrc: 2/0,0 mode: PR/PR res: [0x200015bc1:0xd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a1242 expref: -99 pid: 78638 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826824.381879:0:78638:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200015bc1:0xd:0x0](ffff9856116763c8), inode mode 81a4 mask 40 00000080:00200008:1.0:1787826824.381910:0:78638:0:(file.c:479:ll_file_release()) START file f33b.replay-single-2:[0x200015bc1:0xd:0x0](ffff9856116763c8), flags 100002 00000002:00100000:1.0:1787826824.381949:0:78638:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff985610e32940 x1874669225548416/t193273528355(193273528355) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826840 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:1.0:1787826824.381956:0:78638:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:1.0:1787826824.381959:0:78638:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610e312c0 x1874669225548672/t0(0) o35->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:1.0:1787826824.381974:0:78638:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e312c0 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78638:1874669225548672:192.168.203.114@tcp:35:createmany.0 00000100:00100000:1.0:1787826824.382026:0:78638:0:(client.c:2715:ptlrpc_set_wait()) set ffff9856108f20c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826824.383124:0:78638:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e312c0 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78638:1874669225548672:192.168.203.114@tcp:35:createmany.0 00000002:00100000:1.0:1787826824.383137:0:78638:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff985610e32940 x1874669225548416/t193273528355(193273528355) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826840 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:1.0:1787826824.383178:0:78638:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826824.383189:0:78638:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826824.383231:0:78638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e03bb lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 78638 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.383239:0:78638:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e03bb lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78638 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.383247:0:78638:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e03bb lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78638 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826824.383256:0:78638:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e303c0 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78638:1874669225548800:192.168.203.114@tcp:101:createmany.0 00000100:00100000:1.0:1787826824.383293:0:78638:0:(client.c:2715:ptlrpc_set_wait()) set ffff9856108f20c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826824.384270:0:78638:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e303c0 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78638:1874669225548800:192.168.203.114@tcp:101:createmany.0 00010000:00010000:1.0:1787826824.384280:0:78638:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e03bb lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1257 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.384286:0:78638:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e03bb lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1257 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.384295:0:78638:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e03bb lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1257 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.384301:0:78638:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e03bb lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1257 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.384306:0:78638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e03bb lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1257 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.384311:0:78638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e03bb lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1257 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.384315:0:78638:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e03bb lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1257 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826824.384320:0:78638:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff985610e303c0 x1874669225548800/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826840 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826824.384327:0:78638:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e03bb lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1257 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.384333:0:78638:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:1.0:1787826824.384336:0:78638:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:1.0:1787826824.384349:0:78638:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:1.0:1787826824.384354:0:78638:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e03bb 00000080:00010000:1.0:1787826824.384357:0:78638:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826824.384360:0:78638:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e03bb from it ffffab3ac167bbf8 00010000:00010000:1.0:1787826824.384369:0:78638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e03bb lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1257 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.384377:0:78638:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e03bb lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1257 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826824.384394:0:78638:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787826824.384409:0:78638:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826824.384417:0:78638:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826824.384425:0:78638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e03bb lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1257 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826824.384434:0:78638:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e03bb lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1257 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826824.384441:0:78638:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826824.384445:0:78638:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826824.384448:0:78638:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e03bb 00000080:00010000:1.0:1787826824.384451:0:78638:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e03bb from it ffffab3ac167bb38 00010000:00010000:1.0:1787826824.384455:0:78638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e03bb lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1257 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826824.384462:0:78638:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e03bb lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1257 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826824.384471:0:78638:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787826824.384477:0:78638:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f33b.replay-single-3, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), file ffff985611f18900, open_flags 8042, mode 81a4 opened 0 00000080:00200000:1.0:1787826824.384484:0:78638:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f33b.replay-single-3, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=open|creat 00000002:00010000:1.0:1787826824.384500:0:78638:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f33b.replay-single-3,[0x200015bc1:0xe:0x0]) in obj [0x200000007:0x1:0x0], intent: open|creat flags 0100103 00010000:00010000:1.0:1787826824.384515:0:78638:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e03bb lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x5aa6d504d91a1257 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826824.384545:0:78638:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e03bb lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a1257 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826824.384552:0:78638:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826824.384555:0:78638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e9a800/0xb3f83dad1b7e03bb lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a1257 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826824.384580:0:78638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050400/0xb3f83dad1b7e03c2 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 78638 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.384587:0:78638:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050400/0xb3f83dad1b7e03c2 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78638 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:1.0:1787826824.384596:0:78638:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:1.0:1787826824.384602:0:78638:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610e303c0 x1874669225549056/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:1.0:1787826824.384619:0:78638:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050400/0xb3f83dad1b7e03c2 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78638 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826824.384631:0:78638:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e303c0 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78638:1874669225549056:192.168.203.114@tcp:101:createmany.0 00000100:00100000:1.0:1787826824.384733:0:78638:0:(client.c:2715:ptlrpc_set_wait()) set ffff9856108f20c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826824.387062:0:78638:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e303c0 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78638:1874669225549056:192.168.203.114@tcp:101:createmany.0 00010000:00010000:1.0:1787826824.387078:0:78638:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050400/0xb3f83dad1b7e03c2 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1273 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.387115:0:78638:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050400/0xb3f83dad1b7e03c2 lrc: 4/0,1 mode: --/PR res: [0x200015bc1:0xe:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1273 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.387123:0:78638:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050400/0xb3f83dad1b7e03c2 lrc: 4/0,1 mode: --/PR res: [0x200015bc1:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1273 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.387132:0:78638:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050400/0xb3f83dad1b7e03c2 lrc: 4/0,1 mode: PR/PR res: [0x200015bc1:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1273 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.387142:0:78638:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050400/0xb3f83dad1b7e03c2 lrc: 4/0,1 mode: PR/PR res: [0x200015bc1:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1273 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.387153:0:78638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050400/0xb3f83dad1b7e03c2 lrc: 5/1,1 mode: PR/PR res: [0x200015bc1:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1273 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.387161:0:78638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050400/0xb3f83dad1b7e03c2 lrc: 5/1,1 mode: PR/PR res: [0x200015bc1:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1273 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.387168:0:78638:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050400/0xb3f83dad1b7e03c2 lrc: 4/1,0 mode: PR/PR res: [0x200015bc1:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1273 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826824.387179:0:78638:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff985610e303c0 x1874669225549056/t193273528357(193273528357) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/664 e 0 to 0 dl 1787826840 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:1.0:1787826824.387191:0:78638:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff985610e303c0 x1874669225549056/t193273528357(193273528357) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/664 e 0 to 0 dl 1787826840 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826824.387203:0:78638:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050400/0xb3f83dad1b7e03c2 lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1273 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:1.0:1787826824.387228:0:78638:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050400/0xb3f83dad1b7e03c2 lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1273 expref: -99 pid: 78638 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.387238:0:78638:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8590023617/14 (0 0) 00000080:00200000:1.0:1787826824.387242:0:78638:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:1.0:1787826824.387251:0:78638:0:(namei.c:1628:ll_create_it()) VFS Op:name=f33b.replay-single-3, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=open|creat 00000080:00200000:1.0:1787826824.387270:0:78638:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200015bc1:0xe:0x0](ffff985611675b88) 00000080:00200000:1.0:1787826824.387345:0:78638:0:(file.c:6854:ll_layout_conf()) [0x200015bc1:0xe:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:1.0:1787826824.387350:0:78638:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x200015bc1:0xe:0x0](ffff985611675b88) 00000080:00010000:1.0:1787826824.387353:0:78638:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200015bc1:0xe:0x0](ffff985611675b88) for lock 0xb3f83dad1b7e03c2 00000080:00010000:1.0:1787826824.387361:0:78638:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200015bc1:0xe:0x0](ffff985611675b88) 00000080:00200008:1.0:1787826824.387371:0:78638:0:(file.c:1090:ll_file_open()) START file f33b.replay-single-3:[0x200015bc1:0xe:0x0](ffff985611675b88), flags 100102 00000002:00100000:1.0:1787826824.387386:0:78638:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff985610e303c0 x1874669225549056/t193273528357(193273528357) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826840 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:1.0:1787826824.387418:0:78638:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e03c2 from it ffff985610e3fd20 00010000:00010000:1.0:1787826824.387423:0:78638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050400/0xb3f83dad1b7e03c2 lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a1273 expref: -99 pid: 78638 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.387431:0:78638:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050400/0xb3f83dad1b7e03c2 lrc: 2/0,0 mode: PR/PR res: [0x200015bc1:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a1273 expref: -99 pid: 78638 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826824.387445:0:78638:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200015bc1:0xe:0x0](ffff985611675b88), inode mode 81a4 mask 40 00000080:00200008:1.0:1787826824.387463:0:78638:0:(file.c:479:ll_file_release()) START file f33b.replay-single-3:[0x200015bc1:0xe:0x0](ffff985611675b88), flags 100002 00000002:00100000:1.0:1787826824.387490:0:78638:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff985610e303c0 x1874669225549056/t193273528357(193273528357) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826840 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:1.0:1787826824.387505:0:78638:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:1.0:1787826824.387510:0:78638:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610e31680 x1874669225549312/t0(0) o35->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:1.0:1787826824.387525:0:78638:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e31680 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78638:1874669225549312:192.168.203.114@tcp:35:createmany.0 00000100:00100000:1.0:1787826824.387586:0:78638:0:(client.c:2715:ptlrpc_set_wait()) set ffff9856108f2b40 going to sleep for 11 seconds 00000100:00100000:1.0:1787826824.388578:0:78638:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e31680 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78638:1874669225549312:192.168.203.114@tcp:35:createmany.0 00000002:00100000:1.0:1787826824.388597:0:78638:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff985610e303c0 x1874669225549056/t193273528357(193273528357) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826840 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:1.0:1787826824.388660:0:78638:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826824.388672:0:78638:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826824.388725:0:78638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051000/0xb3f83dad1b7e03c9 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 78638 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.388734:0:78638:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051000/0xb3f83dad1b7e03c9 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78638 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.388742:0:78638:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051000/0xb3f83dad1b7e03c9 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78638 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826824.388755:0:78638:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e32d00 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78638:1874669225549440:192.168.203.114@tcp:101:createmany.0 00000100:00100000:1.0:1787826824.388788:0:78638:0:(client.c:2715:ptlrpc_set_wait()) set ffff9856108f2b40 going to sleep for 11 seconds 00000100:00100000:1.0:1787826824.389845:0:78638:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e32d00 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78638:1874669225549440:192.168.203.114@tcp:101:createmany.0 00010000:00010000:1.0:1787826824.389859:0:78638:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051000/0xb3f83dad1b7e03c9 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1288 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.389878:0:78638:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051000/0xb3f83dad1b7e03c9 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1288 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.389887:0:78638:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051000/0xb3f83dad1b7e03c9 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1288 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.389897:0:78638:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051000/0xb3f83dad1b7e03c9 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1288 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.389906:0:78638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051000/0xb3f83dad1b7e03c9 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1288 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.389913:0:78638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051000/0xb3f83dad1b7e03c9 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1288 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.389921:0:78638:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051000/0xb3f83dad1b7e03c9 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1288 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826824.389931:0:78638:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff985610e32d00 x1874669225549440/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826840 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826824.389940:0:78638:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051000/0xb3f83dad1b7e03c9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1288 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.389948:0:78638:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:1.0:1787826824.389952:0:78638:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:1.0:1787826824.389965:0:78638:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:1.0:1787826824.389968:0:78638:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e03c9 00000080:00010000:1.0:1787826824.389971:0:78638:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826824.389973:0:78638:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e03c9 from it ffffab3ac167bbf8 00010000:00010000:1.0:1787826824.389976:0:78638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051000/0xb3f83dad1b7e03c9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1288 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.389981:0:78638:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051000/0xb3f83dad1b7e03c9 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1288 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826824.389995:0:78638:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787826824.390008:0:78638:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826824.390017:0:78638:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826824.390024:0:78638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051000/0xb3f83dad1b7e03c9 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1288 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826824.390033:0:78638:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051000/0xb3f83dad1b7e03c9 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1288 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826824.390040:0:78638:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826824.390044:0:78638:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826824.390047:0:78638:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e03c9 00000080:00010000:1.0:1787826824.390050:0:78638:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e03c9 from it ffffab3ac167bb38 00010000:00010000:1.0:1787826824.390053:0:78638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051000/0xb3f83dad1b7e03c9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1288 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826824.390060:0:78638:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051000/0xb3f83dad1b7e03c9 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1288 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826824.390074:0:78638:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787826824.390078:0:78638:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f33b.replay-single-4, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), file ffff985611f19500, open_flags 8042, mode 81a4 opened 0 00000080:00200000:1.0:1787826824.390086:0:78638:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f33b.replay-single-4, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=open|creat 00000002:00010000:1.0:1787826824.390101:0:78638:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f33b.replay-single-4,[0x200015bc1:0xf:0x0]) in obj [0x200000007:0x1:0x0], intent: open|creat flags 0100103 00010000:00010000:1.0:1787826824.390111:0:78638:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051000/0xb3f83dad1b7e03c9 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x5aa6d504d91a1288 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826824.390144:0:78638:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051000/0xb3f83dad1b7e03c9 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a1288 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826824.390151:0:78638:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826824.390154:0:78638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051000/0xb3f83dad1b7e03c9 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a1288 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826824.390175:0:78638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050600/0xb3f83dad1b7e03d0 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 78638 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.390180:0:78638:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050600/0xb3f83dad1b7e03d0 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78638 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:1.0:1787826824.390186:0:78638:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:1.0:1787826824.390190:0:78638:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610e32d00 x1874669225549696/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:1.0:1787826824.390196:0:78638:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050600/0xb3f83dad1b7e03d0 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78638 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826824.390220:0:78638:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e32d00 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78638:1874669225549696:192.168.203.114@tcp:101:createmany.0 00000100:00100000:1.0:1787826824.390321:0:78638:0:(client.c:2715:ptlrpc_set_wait()) set ffff9856108f2b40 going to sleep for 11 seconds 00000100:00100000:1.0:1787826824.392104:0:78638:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e32d00 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78638:1874669225549696:192.168.203.114@tcp:101:createmany.0 00010000:00010000:1.0:1787826824.392119:0:78638:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050600/0xb3f83dad1b7e03d0 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a12a4 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.392140:0:78638:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050600/0xb3f83dad1b7e03d0 lrc: 4/0,1 mode: --/PR res: [0x200015bc1:0xf:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a12a4 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.392148:0:78638:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050600/0xb3f83dad1b7e03d0 lrc: 4/0,1 mode: --/PR res: [0x200015bc1:0xf:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a12a4 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.392158:0:78638:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050600/0xb3f83dad1b7e03d0 lrc: 4/0,1 mode: PR/PR res: [0x200015bc1:0xf:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a12a4 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.392167:0:78638:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050600/0xb3f83dad1b7e03d0 lrc: 4/0,1 mode: PR/PR res: [0x200015bc1:0xf:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a12a4 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.392177:0:78638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050600/0xb3f83dad1b7e03d0 lrc: 5/1,1 mode: PR/PR res: [0x200015bc1:0xf:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a12a4 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.392185:0:78638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050600/0xb3f83dad1b7e03d0 lrc: 5/1,1 mode: PR/PR res: [0x200015bc1:0xf:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a12a4 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.392192:0:78638:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050600/0xb3f83dad1b7e03d0 lrc: 4/1,0 mode: PR/PR res: [0x200015bc1:0xf:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a12a4 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826824.392202:0:78638:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff985610e32d00 x1874669225549696/t193273528359(193273528359) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/664 e 0 to 0 dl 1787826840 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:1.0:1787826824.392229:0:78638:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff985610e32d00 x1874669225549696/t193273528359(193273528359) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/664 e 0 to 0 dl 1787826840 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826824.392241:0:78638:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050600/0xb3f83dad1b7e03d0 lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0xf:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a12a4 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:1.0:1787826824.392252:0:78638:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050600/0xb3f83dad1b7e03d0 lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0xf:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a12a4 expref: -99 pid: 78638 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.392260:0:78638:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8590023617/15 (0 0) 00000080:00200000:1.0:1787826824.392265:0:78638:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:1.0:1787826824.392274:0:78638:0:(namei.c:1628:ll_create_it()) VFS Op:name=f33b.replay-single-4, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=open|creat 00000080:00200000:1.0:1787826824.392295:0:78638:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200015bc1:0xf:0x0](ffff9856115880c8) 00000080:00200000:1.0:1787826824.392364:0:78638:0:(file.c:6854:ll_layout_conf()) [0x200015bc1:0xf:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:1.0:1787826824.392369:0:78638:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x200015bc1:0xf:0x0](ffff9856115880c8) 00000080:00010000:1.0:1787826824.392372:0:78638:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200015bc1:0xf:0x0](ffff9856115880c8) for lock 0xb3f83dad1b7e03d0 00000080:00010000:1.0:1787826824.392379:0:78638:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200015bc1:0xf:0x0](ffff9856115880c8) 00000080:00200008:1.0:1787826824.392388:0:78638:0:(file.c:1090:ll_file_open()) START file f33b.replay-single-4:[0x200015bc1:0xf:0x0](ffff9856115880c8), flags 100102 00000002:00100000:1.0:1787826824.392405:0:78638:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff985610e32d00 x1874669225549696/t193273528359(193273528359) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826840 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:1.0:1787826824.392432:0:78638:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e03d0 from it ffff985610e3fd20 00010000:00010000:1.0:1787826824.392436:0:78638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050600/0xb3f83dad1b7e03d0 lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0xf:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a12a4 expref: -99 pid: 78638 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.392445:0:78638:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050600/0xb3f83dad1b7e03d0 lrc: 2/0,0 mode: PR/PR res: [0x200015bc1:0xf:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a12a4 expref: -99 pid: 78638 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826824.392464:0:78638:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200015bc1:0xf:0x0](ffff9856115880c8), inode mode 81a4 mask 40 00000080:00200008:1.0:1787826824.392485:0:78638:0:(file.c:479:ll_file_release()) START file f33b.replay-single-4:[0x200015bc1:0xf:0x0](ffff9856115880c8), flags 100002 00000002:00100000:1.0:1787826824.392514:0:78638:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff985610e32d00 x1874669225549696/t193273528359(193273528359) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826840 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:1.0:1787826824.392523:0:78638:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:1.0:1787826824.392528:0:78638:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610e330c0 x1874669225549952/t0(0) o35->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:1.0:1787826824.392542:0:78638:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e330c0 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78638:1874669225549952:192.168.203.114@tcp:35:createmany.0 00000100:00100000:1.0:1787826824.392594:0:78638:0:(client.c:2715:ptlrpc_set_wait()) set ffff9856108f2600 going to sleep for 11 seconds 00000100:00100000:1.0:1787826824.393581:0:78638:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e330c0 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78638:1874669225549952:192.168.203.114@tcp:35:createmany.0 00000002:00100000:1.0:1787826824.393601:0:78638:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff985610e32d00 x1874669225549696/t193273528359(193273528359) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826840 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:1.0:1787826824.393654:0:78638:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826824.393666:0:78638:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826824.393739:0:78638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e03d7 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 78638 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.393748:0:78638:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e03d7 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78638 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826824.393758:0:78638:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e03d7 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78638 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826824.393785:0:78638:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610d3c000 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78638:1874669225550080:192.168.203.114@tcp:101:createmany.0 00000100:00100000:1.0:1787826824.393829:0:78638:0:(client.c:2715:ptlrpc_set_wait()) set ffff9856108f2600 going to sleep for 11 seconds 00000100:00100000:3.0:1787826824.394847:0:78638:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610d3c000 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78638:1874669225550080:192.168.203.114@tcp:101:createmany.0 00010000:00010000:3.0:1787826824.394870:0:78638:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e03d7 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a12b9 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.394883:0:78638:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e03d7 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a12b9 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.394896:0:78638:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e03d7 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a12b9 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.394908:0:78638:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e03d7 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a12b9 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.394920:0:78638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e03d7 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a12b9 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.394930:0:78638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e03d7 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a12b9 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.394937:0:78638:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e03d7 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a12b9 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826824.394959:0:78638:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff985610d3c000 x1874669225550080/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826840 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826824.394974:0:78638:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e03d7 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a12b9 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.394986:0:78638:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:3.0:1787826824.394993:0:78638:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:3.0:1787826824.395019:0:78638:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:3.0:1787826824.395026:0:78638:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e03d7 00000080:00010000:3.0:1787826824.395031:0:78638:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:3.0:1787826824.395034:0:78638:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e03d7 from it ffffab3ac167bbf8 00010000:00010000:3.0:1787826824.395038:0:78638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e03d7 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a12b9 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.395047:0:78638:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e03d7 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a12b9 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826824.395071:0:78638:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:3.0:1787826824.395095:0:78638:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:3.0:1787826824.395110:0:78638:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826824.395121:0:78638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e03d7 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a12b9 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826824.395132:0:78638:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e03d7 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a12b9 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826824.395139:0:78638:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826824.395143:0:78638:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:3.0:1787826824.395146:0:78638:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e03d7 00000080:00010000:3.0:1787826824.395150:0:78638:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e03d7 from it ffffab3ac167bb38 00010000:00010000:3.0:1787826824.395154:0:78638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e03d7 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a12b9 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826824.395162:0:78638:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e03d7 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a12b9 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:3.0:1787826824.395170:0:78638:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 3 00000080:00200000:3.0:1787826824.395176:0:78638:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f33b.replay-single-5, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), file ffff985611f18a80, open_flags 8042, mode 81a4 opened 0 00000080:00200000:3.0:1787826824.395185:0:78638:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f33b.replay-single-5, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=open|creat 00000002:00010000:3.0:1787826824.395227:0:78638:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f33b.replay-single-5,[0x200015bc1:0x10:0x0]) in obj [0x200000007:0x1:0x0], intent: open|creat flags 0100103 00010000:00010000:3.0:1787826824.395239:0:78638:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e03d7 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x5aa6d504d91a12b9 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826824.395286:0:78638:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e03d7 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a12b9 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826824.395293:0:78638:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826824.395296:0:78638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e03d7 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a12b9 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826824.395330:0:78638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06600/0xb3f83dad1b7e03de lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 78638 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.395338:0:78638:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06600/0xb3f83dad1b7e03de lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78638 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:3.0:1787826824.395360:0:78638:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:3.0:1787826824.395367:0:78638:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610e4f480 x1874669225550336/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:3.0:1787826824.395376:0:78638:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06600/0xb3f83dad1b7e03de lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78638 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826824.395396:0:78638:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e4f480 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78638:1874669225550336:192.168.203.114@tcp:101:createmany.0 00000100:00100000:3.0:1787826824.395544:0:78638:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607e06900 going to sleep for 11 seconds 00000100:00100000:3.0:1787826824.397342:0:78638:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e4f480 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78638:1874669225550336:192.168.203.114@tcp:101:createmany.0 00010000:00010000:3.0:1787826824.397370:0:78638:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06600/0xb3f83dad1b7e03de lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a12d5 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.397393:0:78638:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06600/0xb3f83dad1b7e03de lrc: 4/0,1 mode: --/PR res: [0x200015bc1:0x10:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a12d5 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.397401:0:78638:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06600/0xb3f83dad1b7e03de lrc: 4/0,1 mode: --/PR res: [0x200015bc1:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a12d5 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.397410:0:78638:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06600/0xb3f83dad1b7e03de lrc: 4/0,1 mode: PR/PR res: [0x200015bc1:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a12d5 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.397425:0:78638:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06600/0xb3f83dad1b7e03de lrc: 4/0,1 mode: PR/PR res: [0x200015bc1:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a12d5 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.397433:0:78638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06600/0xb3f83dad1b7e03de lrc: 5/1,1 mode: PR/PR res: [0x200015bc1:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a12d5 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.397438:0:78638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06600/0xb3f83dad1b7e03de lrc: 5/1,1 mode: PR/PR res: [0x200015bc1:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a12d5 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.397443:0:78638:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06600/0xb3f83dad1b7e03de lrc: 4/1,0 mode: PR/PR res: [0x200015bc1:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a12d5 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826824.397449:0:78638:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff985610e4f480 x1874669225550336/t193273528361(193273528361) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/664 e 0 to 0 dl 1787826840 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:3.0:1787826824.397457:0:78638:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff985610e4f480 x1874669225550336/t193273528361(193273528361) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/664 e 0 to 0 dl 1787826840 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826824.397465:0:78638:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06600/0xb3f83dad1b7e03de lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a12d5 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:3.0:1787826824.397472:0:78638:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06600/0xb3f83dad1b7e03de lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a12d5 expref: -99 pid: 78638 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.397479:0:78638:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8590023617/16 (0 0) 00000080:00200000:3.0:1787826824.397483:0:78638:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:3.0:1787826824.397492:0:78638:0:(namei.c:1628:ll_create_it()) VFS Op:name=f33b.replay-single-5, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=open|creat 00000080:00200000:3.0:1787826824.397514:0:78638:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200015bc1:0x10:0x0](ffff98561011db88) 00000080:00200000:3.0:1787826824.397596:0:78638:0:(file.c:6854:ll_layout_conf()) [0x200015bc1:0x10:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:3.0:1787826824.397601:0:78638:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x200015bc1:0x10:0x0](ffff98561011db88) 00000080:00010000:3.0:1787826824.397604:0:78638:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200015bc1:0x10:0x0](ffff98561011db88) for lock 0xb3f83dad1b7e03de 00000080:00010000:3.0:1787826824.397613:0:78638:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200015bc1:0x10:0x0](ffff98561011db88) 00000080:00200008:3.0:1787826824.397636:0:78638:0:(file.c:1090:ll_file_open()) START file f33b.replay-single-5:[0x200015bc1:0x10:0x0](ffff98561011db88), flags 100102 00000002:00100000:3.0:1787826824.397653:0:78638:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff985610e4f480 x1874669225550336/t193273528361(193273528361) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826840 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:3.0:1787826824.397675:0:78638:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e03de from it ffff985610c4a4e0 00010000:00010000:3.0:1787826824.397679:0:78638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06600/0xb3f83dad1b7e03de lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a12d5 expref: -99 pid: 78638 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.397685:0:78638:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06600/0xb3f83dad1b7e03de lrc: 2/0,0 mode: PR/PR res: [0x200015bc1:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a12d5 expref: -99 pid: 78638 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826824.397724:0:78638:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200015bc1:0x10:0x0](ffff98561011db88), inode mode 81a4 mask 40 00000080:00200008:3.0:1787826824.397758:0:78638:0:(file.c:479:ll_file_release()) START file f33b.replay-single-5:[0x200015bc1:0x10:0x0](ffff98561011db88), flags 100002 00000002:00100000:3.0:1787826824.397789:0:78638:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff985610e4f480 x1874669225550336/t193273528361(193273528361) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826840 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:3.0:1787826824.397796:0:78638:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:3.0:1787826824.397799:0:78638:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610e4ed00 x1874669225550592/t0(0) o35->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:3.0:1787826824.397813:0:78638:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e4ed00 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78638:1874669225550592:192.168.203.114@tcp:35:createmany.0 00000100:00100000:3.0:1787826824.397872:0:78638:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607e060c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787826824.398838:0:78638:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e4ed00 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78638:1874669225550592:192.168.203.114@tcp:35:createmany.0 00000002:00100000:3.0:1787826824.398858:0:78638:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff985610e4f480 x1874669225550336/t193273528361(193273528361) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826840 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:3.0:1787826824.398920:0:78638:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:3.0:1787826824.398932:0:78638:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826824.398964:0:78638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06a00/0xb3f83dad1b7e03e5 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 78638 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.398971:0:78638:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06a00/0xb3f83dad1b7e03e5 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78638 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.398980:0:78638:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06a00/0xb3f83dad1b7e03e5 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78638 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826824.398992:0:78638:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e4c3c0 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78638:1874669225550720:192.168.203.114@tcp:101:createmany.0 00000100:00100000:3.0:1787826824.399030:0:78638:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607e060c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787826824.400387:0:78638:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e4c3c0 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78638:1874669225550720:192.168.203.114@tcp:101:createmany.0 00010000:00010000:3.0:1787826824.400401:0:78638:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06a00/0xb3f83dad1b7e03e5 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a12ea expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.400412:0:78638:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06a00/0xb3f83dad1b7e03e5 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a12ea expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.400424:0:78638:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06a00/0xb3f83dad1b7e03e5 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a12ea expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.400434:0:78638:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06a00/0xb3f83dad1b7e03e5 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a12ea expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.400449:0:78638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06a00/0xb3f83dad1b7e03e5 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a12ea expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.400458:0:78638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06a00/0xb3f83dad1b7e03e5 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a12ea expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.400466:0:78638:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06a00/0xb3f83dad1b7e03e5 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a12ea expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826824.400476:0:78638:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff985610e4c3c0 x1874669225550720/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826840 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826824.400487:0:78638:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06a00/0xb3f83dad1b7e03e5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a12ea expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.400496:0:78638:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:3.0:1787826824.400501:0:78638:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:3.0:1787826824.400519:0:78638:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:3.0:1787826824.400524:0:78638:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e03e5 00000080:00010000:3.0:1787826824.400529:0:78638:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:3.0:1787826824.400532:0:78638:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e03e5 from it ffffab3ac167bbf8 00010000:00010000:3.0:1787826824.400536:0:78638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06a00/0xb3f83dad1b7e03e5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a12ea expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.400545:0:78638:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06a00/0xb3f83dad1b7e03e5 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a12ea expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826824.400570:0:78638:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:3.0:1787826824.400586:0:78638:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:3.0:1787826824.400595:0:78638:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826824.400603:0:78638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06a00/0xb3f83dad1b7e03e5 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a12ea expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826824.400613:0:78638:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06a00/0xb3f83dad1b7e03e5 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a12ea expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826824.400621:0:78638:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826824.400625:0:78638:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:3.0:1787826824.400627:0:78638:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e03e5 00000080:00010000:3.0:1787826824.400631:0:78638:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e03e5 from it ffffab3ac167bb38 00010000:00010000:3.0:1787826824.400634:0:78638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06a00/0xb3f83dad1b7e03e5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a12ea expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826824.400642:0:78638:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06a00/0xb3f83dad1b7e03e5 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a12ea expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:3.0:1787826824.400651:0:78638:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 3 00000080:00200000:3.0:1787826824.400656:0:78638:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f33b.replay-single-6, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), file ffff98561436ea80, open_flags 8042, mode 81a4 opened 0 00000080:00200000:3.0:1787826824.400663:0:78638:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f33b.replay-single-6, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=open|creat 00000002:00010000:3.0:1787826824.400679:0:78638:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f33b.replay-single-6,[0x200015bc1:0x11:0x0]) in obj [0x200000007:0x1:0x0], intent: open|creat flags 0100103 00010000:00010000:3.0:1787826824.400689:0:78638:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06a00/0xb3f83dad1b7e03e5 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x5aa6d504d91a12ea expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826824.400756:0:78638:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06a00/0xb3f83dad1b7e03e5 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a12ea expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826824.400763:0:78638:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826824.400766:0:78638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06a00/0xb3f83dad1b7e03e5 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a12ea expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826824.400806:0:78638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06400/0xb3f83dad1b7e03ec lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 78638 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.400813:0:78638:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06400/0xb3f83dad1b7e03ec lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78638 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:3.0:1787826824.400822:0:78638:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:3.0:1787826824.400829:0:78638:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610e4c3c0 x1874669225550976/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:3.0:1787826824.400838:0:78638:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06400/0xb3f83dad1b7e03ec lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78638 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826824.400851:0:78638:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e4c3c0 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78638:1874669225550976:192.168.203.114@tcp:101:createmany.0 00000100:00100000:3.0:1787826824.400951:0:78638:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607e060c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787826824.402662:0:78638:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e4c3c0 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78638:1874669225550976:192.168.203.114@tcp:101:createmany.0 00010000:00010000:3.0:1787826824.402679:0:78638:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06400/0xb3f83dad1b7e03ec lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1306 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.402733:0:78638:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06400/0xb3f83dad1b7e03ec lrc: 4/0,1 mode: --/PR res: [0x200015bc1:0x11:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1306 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.402748:0:78638:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06400/0xb3f83dad1b7e03ec lrc: 4/0,1 mode: --/PR res: [0x200015bc1:0x11:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1306 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.402758:0:78638:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06400/0xb3f83dad1b7e03ec lrc: 4/0,1 mode: PR/PR res: [0x200015bc1:0x11:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1306 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.402768:0:78638:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06400/0xb3f83dad1b7e03ec lrc: 4/0,1 mode: PR/PR res: [0x200015bc1:0x11:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1306 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.402784:0:78638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06400/0xb3f83dad1b7e03ec lrc: 5/1,1 mode: PR/PR res: [0x200015bc1:0x11:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1306 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.402794:0:78638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06400/0xb3f83dad1b7e03ec lrc: 5/1,1 mode: PR/PR res: [0x200015bc1:0x11:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1306 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.402801:0:78638:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06400/0xb3f83dad1b7e03ec lrc: 4/1,0 mode: PR/PR res: [0x200015bc1:0x11:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1306 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826824.402811:0:78638:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff985610e4c3c0 x1874669225550976/t193273528363(193273528363) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/664 e 0 to 0 dl 1787826840 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:3.0:1787826824.402823:0:78638:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff985610e4c3c0 x1874669225550976/t193273528363(193273528363) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/664 e 0 to 0 dl 1787826840 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826824.402835:0:78638:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06400/0xb3f83dad1b7e03ec lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x11:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1306 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:3.0:1787826824.402846:0:78638:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06400/0xb3f83dad1b7e03ec lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x11:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1306 expref: -99 pid: 78638 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.402856:0:78638:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8590023617/17 (0 0) 00000080:00200000:3.0:1787826824.402861:0:78638:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:3.0:1787826824.402872:0:78638:0:(namei.c:1628:ll_create_it()) VFS Op:name=f33b.replay-single-6, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=open|creat 00000080:00200000:3.0:1787826824.402896:0:78638:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200015bc1:0x11:0x0](ffff985610119148) 00000080:00200000:3.0:1787826824.402965:0:78638:0:(file.c:6854:ll_layout_conf()) [0x200015bc1:0x11:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:3.0:1787826824.402970:0:78638:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x200015bc1:0x11:0x0](ffff985610119148) 00000080:00010000:3.0:1787826824.402973:0:78638:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200015bc1:0x11:0x0](ffff985610119148) for lock 0xb3f83dad1b7e03ec 00000080:00010000:3.0:1787826824.402982:0:78638:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200015bc1:0x11:0x0](ffff985610119148) 00000080:00200008:3.0:1787826824.402992:0:78638:0:(file.c:1090:ll_file_open()) START file f33b.replay-single-6:[0x200015bc1:0x11:0x0](ffff985610119148), flags 100102 00000002:00100000:3.0:1787826824.403008:0:78638:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff985610e4c3c0 x1874669225550976/t193273528363(193273528363) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826840 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:3.0:1787826824.403050:0:78638:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e03ec from it ffff985610c4a4e0 00010000:00010000:3.0:1787826824.403055:0:78638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06400/0xb3f83dad1b7e03ec lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x11:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a1306 expref: -99 pid: 78638 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.403063:0:78638:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06400/0xb3f83dad1b7e03ec lrc: 2/0,0 mode: PR/PR res: [0x200015bc1:0x11:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a1306 expref: -99 pid: 78638 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826824.403077:0:78638:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200015bc1:0x11:0x0](ffff985610119148), inode mode 81a4 mask 40 00000080:00200008:3.0:1787826824.403104:0:78638:0:(file.c:479:ll_file_release()) START file f33b.replay-single-6:[0x200015bc1:0x11:0x0](ffff985610119148), flags 100002 00000002:00100000:3.0:1787826824.403136:0:78638:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff985610e4c3c0 x1874669225550976/t193273528363(193273528363) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826840 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:3.0:1787826824.403144:0:78638:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:3.0:1787826824.403149:0:78638:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610e4d680 x1874669225551232/t0(0) o35->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:3.0:1787826824.403168:0:78638:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e4d680 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78638:1874669225551232:192.168.203.114@tcp:35:createmany.0 00000100:00100000:3.0:1787826824.403227:0:78638:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607e06240 going to sleep for 11 seconds 00000100:00100000:3.0:1787826824.404492:0:78638:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e4d680 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78638:1874669225551232:192.168.203.114@tcp:35:createmany.0 00000002:00100000:3.0:1787826824.404510:0:78638:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff985610e4c3c0 x1874669225550976/t193273528363(193273528363) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826840 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:3.0:1787826824.404567:0:78638:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:3.0:1787826824.404579:0:78638:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826824.404627:0:78638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06800/0xb3f83dad1b7e03f3 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 78638 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.404636:0:78638:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06800/0xb3f83dad1b7e03f3 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78638 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.404646:0:78638:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06800/0xb3f83dad1b7e03f3 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78638 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826824.404660:0:78638:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e4cf00 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78638:1874669225551360:192.168.203.114@tcp:101:createmany.0 00000100:00100000:3.0:1787826824.404774:0:78638:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607e06240 going to sleep for 11 seconds 00000100:00100000:3.0:1787826824.406870:0:78638:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e4cf00 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78638:1874669225551360:192.168.203.114@tcp:101:createmany.0 00010000:00010000:3.0:1787826824.406895:0:78638:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06800/0xb3f83dad1b7e03f3 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a131b expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.406907:0:78638:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06800/0xb3f83dad1b7e03f3 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a131b expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.406923:0:78638:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06800/0xb3f83dad1b7e03f3 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a131b expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.406934:0:78638:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06800/0xb3f83dad1b7e03f3 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a131b expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.406946:0:78638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06800/0xb3f83dad1b7e03f3 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a131b expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.406953:0:78638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06800/0xb3f83dad1b7e03f3 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a131b expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.406958:0:78638:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06800/0xb3f83dad1b7e03f3 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a131b expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826824.406965:0:78638:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff985610e4cf00 x1874669225551360/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826840 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826824.406975:0:78638:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06800/0xb3f83dad1b7e03f3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a131b expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.406987:0:78638:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:3.0:1787826824.407003:0:78638:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:3.0:1787826824.407027:0:78638:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:3.0:1787826824.407033:0:78638:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e03f3 00000080:00010000:3.0:1787826824.407036:0:78638:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:3.0:1787826824.407039:0:78638:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e03f3 from it ffffab3ac167bbf8 00010000:00010000:3.0:1787826824.407041:0:78638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06800/0xb3f83dad1b7e03f3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a131b expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.407047:0:78638:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06800/0xb3f83dad1b7e03f3 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a131b expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826824.407064:0:78638:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:3.0:1787826824.407079:0:78638:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:3.0:1787826824.407090:0:78638:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826824.407097:0:78638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06800/0xb3f83dad1b7e03f3 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a131b expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826824.407103:0:78638:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06800/0xb3f83dad1b7e03f3 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a131b expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826824.407107:0:78638:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826824.407110:0:78638:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:3.0:1787826824.407111:0:78638:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e03f3 00000080:00010000:3.0:1787826824.407113:0:78638:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e03f3 from it ffffab3ac167bb38 00010000:00010000:3.0:1787826824.407115:0:78638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06800/0xb3f83dad1b7e03f3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a131b expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826824.407120:0:78638:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06800/0xb3f83dad1b7e03f3 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a131b expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:3.0:1787826824.407124:0:78638:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 3 00000080:00200000:3.0:1787826824.407127:0:78638:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f33b.replay-single-7, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), file ffff98561436e480, open_flags 8042, mode 81a4 opened 0 00000080:00200000:3.0:1787826824.407133:0:78638:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f33b.replay-single-7, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=open|creat 00000002:00010000:3.0:1787826824.407147:0:78638:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f33b.replay-single-7,[0x200015bc1:0x12:0x0]) in obj [0x200000007:0x1:0x0], intent: open|creat flags 0100103 00010000:00010000:3.0:1787826824.407154:0:78638:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06800/0xb3f83dad1b7e03f3 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x5aa6d504d91a131b expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826824.407185:0:78638:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06800/0xb3f83dad1b7e03f3 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a131b expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826824.407189:0:78638:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826824.407191:0:78638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06800/0xb3f83dad1b7e03f3 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a131b expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826824.407230:0:78638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7e03fa lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 78638 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.407235:0:78638:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7e03fa lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78638 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:3.0:1787826824.407241:0:78638:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:3.0:1787826824.407246:0:78638:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610e4cf00 x1874669225551616/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:3.0:1787826824.407252:0:78638:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7e03fa lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78638 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826824.407266:0:78638:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e4cf00 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78638:1874669225551616:192.168.203.114@tcp:101:createmany.0 00000100:00100000:3.0:1787826824.407374:0:78638:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607e06240 going to sleep for 11 seconds 00000100:00100000:3.0:1787826824.408891:0:78638:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e4cf00 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78638:1874669225551616:192.168.203.114@tcp:101:createmany.0 00010000:00010000:3.0:1787826824.408902:0:78638:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7e03fa lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1337 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.408921:0:78638:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7e03fa lrc: 4/0,1 mode: --/PR res: [0x200015bc1:0x12:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1337 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.408926:0:78638:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7e03fa lrc: 4/0,1 mode: --/PR res: [0x200015bc1:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1337 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.408932:0:78638:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7e03fa lrc: 4/0,1 mode: PR/PR res: [0x200015bc1:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1337 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.408937:0:78638:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7e03fa lrc: 4/0,1 mode: PR/PR res: [0x200015bc1:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1337 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.408943:0:78638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7e03fa lrc: 5/1,1 mode: PR/PR res: [0x200015bc1:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1337 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.408947:0:78638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7e03fa lrc: 5/1,1 mode: PR/PR res: [0x200015bc1:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1337 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.408952:0:78638:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7e03fa lrc: 4/1,0 mode: PR/PR res: [0x200015bc1:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1337 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826824.408957:0:78638:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff985610e4cf00 x1874669225551616/t193273528365(193273528365) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/664 e 0 to 0 dl 1787826840 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:3.0:1787826824.408965:0:78638:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff985610e4cf00 x1874669225551616/t193273528365(193273528365) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/664 e 0 to 0 dl 1787826840 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826824.408973:0:78638:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7e03fa lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1337 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:3.0:1787826824.408980:0:78638:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7e03fa lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1337 expref: -99 pid: 78638 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.408987:0:78638:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8590023617/18 (0 0) 00000080:00200000:3.0:1787826824.408992:0:78638:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:3.0:1787826824.409002:0:78638:0:(namei.c:1628:ll_create_it()) VFS Op:name=f33b.replay-single-7, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=open|creat 00000080:00200000:3.0:1787826824.409024:0:78638:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200015bc1:0x12:0x0](ffff98561011d348) 00000080:00200000:3.0:1787826824.409080:0:78638:0:(file.c:6854:ll_layout_conf()) [0x200015bc1:0x12:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:3.0:1787826824.409083:0:78638:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x200015bc1:0x12:0x0](ffff98561011d348) 00000080:00010000:3.0:1787826824.409085:0:78638:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200015bc1:0x12:0x0](ffff98561011d348) for lock 0xb3f83dad1b7e03fa 00000080:00010000:3.0:1787826824.409091:0:78638:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200015bc1:0x12:0x0](ffff98561011d348) 00000080:00200008:3.0:1787826824.409100:0:78638:0:(file.c:1090:ll_file_open()) START file f33b.replay-single-7:[0x200015bc1:0x12:0x0](ffff98561011d348), flags 100102 00000002:00100000:3.0:1787826824.409113:0:78638:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff985610e4cf00 x1874669225551616/t193273528365(193273528365) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826840 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:3.0:1787826824.409136:0:78638:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e03fa from it ffff985610c4a4e0 00010000:00010000:3.0:1787826824.409139:0:78638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7e03fa lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a1337 expref: -99 pid: 78638 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.409144:0:78638:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7e03fa lrc: 2/0,0 mode: PR/PR res: [0x200015bc1:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a1337 expref: -99 pid: 78638 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826824.409155:0:78638:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200015bc1:0x12:0x0](ffff98561011d348), inode mode 81a4 mask 40 00000080:00200008:3.0:1787826824.409176:0:78638:0:(file.c:479:ll_file_release()) START file f33b.replay-single-7:[0x200015bc1:0x12:0x0](ffff98561011d348), flags 100002 00000002:00100000:3.0:1787826824.409216:0:78638:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff985610e4cf00 x1874669225551616/t193273528365(193273528365) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826840 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:3.0:1787826824.409231:0:78638:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:3.0:1787826824.409235:0:78638:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610e4d2c0 x1874669225551872/t0(0) o35->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:3.0:1787826824.409246:0:78638:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e4d2c0 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78638:1874669225551872:192.168.203.114@tcp:35:createmany.0 00000100:00100000:3.0:1787826824.409278:0:78638:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607e06300 going to sleep for 11 seconds 00000100:00100000:3.0:1787826824.410065:0:78638:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e4d2c0 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78638:1874669225551872:192.168.203.114@tcp:35:createmany.0 00000002:00100000:3.0:1787826824.410076:0:78638:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff985610e4cf00 x1874669225551616/t193273528365(193273528365) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826840 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:3.0:1787826824.410114:0:78638:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:3.0:1787826824.410121:0:78638:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826824.410140:0:78638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06000/0xb3f83dad1b7e0401 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 78638 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.410145:0:78638:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06000/0xb3f83dad1b7e0401 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78638 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.410150:0:78638:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06000/0xb3f83dad1b7e0401 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78638 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826824.410156:0:78638:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e4de00 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78638:1874669225552000:192.168.203.114@tcp:101:createmany.0 00000100:00100000:3.0:1787826824.410185:0:78638:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607e06300 going to sleep for 11 seconds 00000100:00100000:3.0:1787826824.410951:0:78638:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e4de00 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78638:1874669225552000:192.168.203.114@tcp:101:createmany.0 00010000:00010000:3.0:1787826824.410961:0:78638:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06000/0xb3f83dad1b7e0401 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a134c expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.410975:0:78638:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06000/0xb3f83dad1b7e0401 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a134c expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.410984:0:78638:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06000/0xb3f83dad1b7e0401 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a134c expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.410993:0:78638:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06000/0xb3f83dad1b7e0401 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a134c expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.411002:0:78638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06000/0xb3f83dad1b7e0401 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a134c expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.411010:0:78638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06000/0xb3f83dad1b7e0401 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a134c expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.411015:0:78638:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06000/0xb3f83dad1b7e0401 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a134c expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826824.411021:0:78638:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff985610e4de00 x1874669225552000/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826840 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826824.411028:0:78638:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06000/0xb3f83dad1b7e0401 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a134c expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.411034:0:78638:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:3.0:1787826824.411039:0:78638:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:3.0:1787826824.411053:0:78638:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:3.0:1787826824.411058:0:78638:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e0401 00000080:00010000:3.0:1787826824.411062:0:78638:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:3.0:1787826824.411065:0:78638:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0401 from it ffffab3ac167bbf8 00010000:00010000:3.0:1787826824.411068:0:78638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06000/0xb3f83dad1b7e0401 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a134c expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.411076:0:78638:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06000/0xb3f83dad1b7e0401 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a134c expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826824.411091:0:78638:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:3.0:1787826824.411102:0:78638:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:3.0:1787826824.411111:0:78638:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826824.411118:0:78638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06000/0xb3f83dad1b7e0401 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a134c expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826824.411126:0:78638:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06000/0xb3f83dad1b7e0401 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a134c expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826824.411133:0:78638:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826824.411137:0:78638:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:3.0:1787826824.411140:0:78638:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e0401 00000080:00010000:3.0:1787826824.411144:0:78638:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0401 from it ffffab3ac167bb38 00010000:00010000:3.0:1787826824.411148:0:78638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06000/0xb3f83dad1b7e0401 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a134c expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826824.411156:0:78638:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06000/0xb3f83dad1b7e0401 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a134c expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:3.0:1787826824.411169:0:78638:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 3 00000080:00200000:3.0:1787826824.411174:0:78638:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f33b.replay-single-8, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), file ffff98561436ed80, open_flags 8042, mode 81a4 opened 0 00000080:00200000:3.0:1787826824.411179:0:78638:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f33b.replay-single-8, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=open|creat 00000002:00010000:3.0:1787826824.411191:0:78638:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f33b.replay-single-8,[0x200015bc1:0x13:0x0]) in obj [0x200000007:0x1:0x0], intent: open|creat flags 0100103 00010000:00010000:3.0:1787826824.411197:0:78638:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06000/0xb3f83dad1b7e0401 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x5aa6d504d91a134c expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826824.411233:0:78638:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06000/0xb3f83dad1b7e0401 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a134c expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826824.411238:0:78638:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826824.411239:0:78638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06000/0xb3f83dad1b7e0401 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a134c expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826824.411255:0:78638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7e0408 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 78638 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.411259:0:78638:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7e0408 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78638 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:3.0:1787826824.411265:0:78638:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:3.0:1787826824.411272:0:78638:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610e4de00 x1874669225552256/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:3.0:1787826824.411280:0:78638:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7e0408 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78638 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826824.411291:0:78638:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e4de00 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78638:1874669225552256:192.168.203.114@tcp:101:createmany.0 00000100:00100000:3.0:1787826824.411369:0:78638:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607e06300 going to sleep for 11 seconds 00000100:00100000:3.0:1787826824.412618:0:78638:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e4de00 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78638:1874669225552256:192.168.203.114@tcp:101:createmany.0 00010000:00010000:3.0:1787826824.412626:0:78638:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7e0408 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1368 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.412638:0:78638:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7e0408 lrc: 4/0,1 mode: --/PR res: [0x200015bc1:0x13:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1368 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.412643:0:78638:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7e0408 lrc: 4/0,1 mode: --/PR res: [0x200015bc1:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1368 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.412648:0:78638:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7e0408 lrc: 4/0,1 mode: PR/PR res: [0x200015bc1:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1368 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.412652:0:78638:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7e0408 lrc: 4/0,1 mode: PR/PR res: [0x200015bc1:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1368 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.412658:0:78638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7e0408 lrc: 5/1,1 mode: PR/PR res: [0x200015bc1:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1368 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.412662:0:78638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7e0408 lrc: 5/1,1 mode: PR/PR res: [0x200015bc1:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1368 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.412666:0:78638:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7e0408 lrc: 4/1,0 mode: PR/PR res: [0x200015bc1:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1368 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826824.412674:0:78638:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff985610e4de00 x1874669225552256/t193273528367(193273528367) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/664 e 0 to 0 dl 1787826840 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:3.0:1787826824.412685:0:78638:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff985610e4de00 x1874669225552256/t193273528367(193273528367) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/664 e 0 to 0 dl 1787826840 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826824.412721:0:78638:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7e0408 lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1368 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:3.0:1787826824.412729:0:78638:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7e0408 lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1368 expref: -99 pid: 78638 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.412735:0:78638:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8590023617/19 (0 0) 00000080:00200000:3.0:1787826824.412738:0:78638:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:3.0:1787826824.412747:0:78638:0:(namei.c:1628:ll_create_it()) VFS Op:name=f33b.replay-single-8, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=open|creat 00000080:00200000:3.0:1787826824.412763:0:78638:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200015bc1:0x13:0x0](ffff98561011a1c8) 00000080:00200000:3.0:1787826824.412799:0:78638:0:(file.c:6854:ll_layout_conf()) [0x200015bc1:0x13:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:3.0:1787826824.412802:0:78638:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x200015bc1:0x13:0x0](ffff98561011a1c8) 00000080:00010000:3.0:1787826824.412804:0:78638:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200015bc1:0x13:0x0](ffff98561011a1c8) for lock 0xb3f83dad1b7e0408 00000080:00010000:3.0:1787826824.412809:0:78638:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200015bc1:0x13:0x0](ffff98561011a1c8) 00000080:00200008:3.0:1787826824.412814:0:78638:0:(file.c:1090:ll_file_open()) START file f33b.replay-single-8:[0x200015bc1:0x13:0x0](ffff98561011a1c8), flags 100102 00000002:00100000:3.0:1787826824.412824:0:78638:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff985610e4de00 x1874669225552256/t193273528367(193273528367) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826840 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:3.0:1787826824.412842:0:78638:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0408 from it ffff985610c4a4e0 00010000:00010000:3.0:1787826824.412845:0:78638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7e0408 lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a1368 expref: -99 pid: 78638 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.412853:0:78638:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7e0408 lrc: 2/0,0 mode: PR/PR res: [0x200015bc1:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a1368 expref: -99 pid: 78638 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826824.412871:0:78638:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200015bc1:0x13:0x0](ffff98561011a1c8), inode mode 81a4 mask 40 00000080:00200008:3.0:1787826824.412888:0:78638:0:(file.c:479:ll_file_release()) START file f33b.replay-single-8:[0x200015bc1:0x13:0x0](ffff98561011a1c8), flags 100002 00000002:00100000:3.0:1787826824.412911:0:78638:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff985610e4de00 x1874669225552256/t193273528367(193273528367) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826840 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:3.0:1787826824.412917:0:78638:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:3.0:1787826824.412920:0:78638:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610e4da40 x1874669225552512/t0(0) o35->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:3.0:1787826824.412929:0:78638:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e4da40 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78638:1874669225552512:192.168.203.114@tcp:35:createmany.0 00000100:00100000:3.0:1787826824.412956:0:78638:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607e06000 going to sleep for 11 seconds 00000100:00100000:3.0:1787826824.413888:0:78638:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e4da40 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78638:1874669225552512:192.168.203.114@tcp:35:createmany.0 00000002:00100000:3.0:1787826824.413904:0:78638:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff985610e4de00 x1874669225552256/t193273528367(193273528367) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826840 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:3.0:1787826824.413946:0:78638:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:3.0:1787826824.413956:0:78638:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826824.413984:0:78638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e040f lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 78638 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.413993:0:78638:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e040f lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78638 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.414000:0:78638:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e040f lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78638 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826824.414012:0:78638:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e4f0c0 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78638:1874669225552640:192.168.203.114@tcp:101:createmany.0 00000100:00100000:3.0:1787826824.414053:0:78638:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607e06000 going to sleep for 11 seconds 00000100:00100000:3.0:1787826824.414940:0:78638:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e4f0c0 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78638:1874669225552640:192.168.203.114@tcp:101:createmany.0 00010000:00010000:3.0:1787826824.414951:0:78638:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e040f lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a137d expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.414960:0:78638:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e040f lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a137d expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.414968:0:78638:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e040f lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a137d expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.414976:0:78638:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e040f lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a137d expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.414984:0:78638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e040f lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a137d expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.414991:0:78638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e040f lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a137d expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.415008:0:78638:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e040f lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a137d expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826824.415018:0:78638:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff985610e4f0c0 x1874669225552640/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826840 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826824.415028:0:78638:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e040f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a137d expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.415042:0:78638:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:3.0:1787826824.415047:0:78638:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:3.0:1787826824.415060:0:78638:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:3.0:1787826824.415065:0:78638:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e040f 00000080:00010000:3.0:1787826824.415069:0:78638:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:3.0:1787826824.415072:0:78638:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e040f from it ffffab3ac167bbf8 00010000:00010000:3.0:1787826824.415076:0:78638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e040f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a137d expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.415084:0:78638:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e040f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a137d expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826824.415099:0:78638:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:3.0:1787826824.415110:0:78638:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:3.0:1787826824.415118:0:78638:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826824.415125:0:78638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e040f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a137d expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826824.415133:0:78638:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e040f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a137d expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826824.415139:0:78638:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826824.415142:0:78638:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:3.0:1787826824.415145:0:78638:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e040f 00000080:00010000:3.0:1787826824.415147:0:78638:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e040f from it ffffab3ac167bb38 00010000:00010000:3.0:1787826824.415150:0:78638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e040f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a137d expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826824.415157:0:78638:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e040f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a137d expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:3.0:1787826824.415164:0:78638:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 3 00000080:00200000:3.0:1787826824.415175:0:78638:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f33b.replay-single-9, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), file ffff98561436f680, open_flags 8042, mode 81a4 opened 0 00000080:00200000:3.0:1787826824.415183:0:78638:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f33b.replay-single-9, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=open|creat 00000002:00010000:3.0:1787826824.415197:0:78638:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f33b.replay-single-9,[0x200015bc1:0x14:0x0]) in obj [0x200000007:0x1:0x0], intent: open|creat flags 0100103 00010000:00010000:3.0:1787826824.415217:0:78638:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e040f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x5aa6d504d91a137d expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826824.415247:0:78638:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e040f lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a137d expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826824.415254:0:78638:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826824.415256:0:78638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07e00/0xb3f83dad1b7e040f lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a137d expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826824.415280:0:78638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e0416 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 78638 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.415287:0:78638:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e0416 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78638 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:3.0:1787826824.415295:0:78638:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:3.0:1787826824.415301:0:78638:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610e4f0c0 x1874669225552896/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:3.0:1787826824.415310:0:78638:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e0416 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78638 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826824.415323:0:78638:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e4f0c0 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78638:1874669225552896:192.168.203.114@tcp:101:createmany.0 00000100:00100000:3.0:1787826824.415400:0:78638:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607e06000 going to sleep for 11 seconds 00000100:00100000:3.0:1787826824.417131:0:78638:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e4f0c0 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78638:1874669225552896:192.168.203.114@tcp:101:createmany.0 00010000:00010000:3.0:1787826824.417144:0:78638:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e0416 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1399 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.417160:0:78638:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e0416 lrc: 4/0,1 mode: --/PR res: [0x200015bc1:0x14:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1399 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.417165:0:78638:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e0416 lrc: 4/0,1 mode: --/PR res: [0x200015bc1:0x14:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1399 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.417171:0:78638:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e0416 lrc: 4/0,1 mode: PR/PR res: [0x200015bc1:0x14:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1399 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.417176:0:78638:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e0416 lrc: 4/0,1 mode: PR/PR res: [0x200015bc1:0x14:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1399 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.417183:0:78638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e0416 lrc: 5/1,1 mode: PR/PR res: [0x200015bc1:0x14:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1399 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.417187:0:78638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e0416 lrc: 5/1,1 mode: PR/PR res: [0x200015bc1:0x14:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1399 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.417192:0:78638:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e0416 lrc: 4/1,0 mode: PR/PR res: [0x200015bc1:0x14:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1399 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826824.417203:0:78638:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff985610e4f0c0 x1874669225552896/t193273528369(193273528369) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/664 e 0 to 0 dl 1787826840 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:3.0:1787826824.417227:0:78638:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff985610e4f0c0 x1874669225552896/t193273528369(193273528369) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/664 e 0 to 0 dl 1787826840 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826824.417239:0:78638:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e0416 lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x14:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1399 expref: -99 pid: 78638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:3.0:1787826824.417249:0:78638:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e0416 lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x14:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1399 expref: -99 pid: 78638 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.417258:0:78638:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8590023617/20 (0 0) 00000080:00200000:3.0:1787826824.417263:0:78638:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:3.0:1787826824.417273:0:78638:0:(namei.c:1628:ll_create_it()) VFS Op:name=f33b.replay-single-9, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=open|creat 00000080:00200000:3.0:1787826824.417299:0:78638:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200015bc1:0x14:0x0](ffff98561011ec08) 00000080:00200000:3.0:1787826824.417351:0:78638:0:(file.c:6854:ll_layout_conf()) [0x200015bc1:0x14:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:3.0:1787826824.417356:0:78638:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x200015bc1:0x14:0x0](ffff98561011ec08) 00000080:00010000:3.0:1787826824.417360:0:78638:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200015bc1:0x14:0x0](ffff98561011ec08) for lock 0xb3f83dad1b7e0416 00000080:00010000:3.0:1787826824.417367:0:78638:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200015bc1:0x14:0x0](ffff98561011ec08) 00000080:00200008:3.0:1787826824.417378:0:78638:0:(file.c:1090:ll_file_open()) START file f33b.replay-single-9:[0x200015bc1:0x14:0x0](ffff98561011ec08), flags 100102 00000002:00100000:3.0:1787826824.417395:0:78638:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff985610e4f0c0 x1874669225552896/t193273528369(193273528369) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826840 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:3.0:1787826824.417425:0:78638:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0416 from it ffff985610c4a4e0 00010000:00010000:3.0:1787826824.417430:0:78638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e0416 lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x14:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a1399 expref: -99 pid: 78638 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826824.417444:0:78638:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e0416 lrc: 2/0,0 mode: PR/PR res: [0x200015bc1:0x14:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a1399 expref: -99 pid: 78638 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826824.417459:0:78638:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200015bc1:0x14:0x0](ffff98561011ec08), inode mode 81a4 mask 40 00000080:00200008:3.0:1787826824.417479:0:78638:0:(file.c:479:ll_file_release()) START file f33b.replay-single-9:[0x200015bc1:0x14:0x0](ffff98561011ec08), flags 100002 00000002:00100000:3.0:1787826824.417503:0:78638:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff985610e4f0c0 x1874669225552896/t193273528369(193273528369) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826840 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:3.0:1787826824.417510:0:78638:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:3.0:1787826824.417513:0:78638:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610e4c780 x1874669225553152/t0(0) o35->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:3.0:1787826824.417524:0:78638:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e4c780 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78638:1874669225553152:192.168.203.114@tcp:35:createmany.0 00000100:00100000:3.0:1787826824.417577:0:78638:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607e06600 going to sleep for 11 seconds 00000100:00100000:3.0:1787826824.418521:0:78638:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e4c780 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:78638:1874669225553152:192.168.203.114@tcp:35:createmany.0 00000002:00100000:3.0:1787826824.418538:0:78638:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff985610e4f0c0 x1874669225552896/t193273528369(193273528369) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826840 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00010000:00010000:3.0:1787826825.013379:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826826.484910:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e7bc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:3c946890-15ca-497a-b838-fe1cba61f4dd:2384:1874669225553280:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787826826.485009:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e79a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669225553536:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787826826.485044:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e79680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669225553792:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787826826.487226:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e7bc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:3c946890-15ca-497a-b838-fe1cba61f4dd:2384:1874669225553280:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787826826.487256:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e79a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669225553536:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787826826.487492:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e79680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669225553792:192.168.203.114@tcp:400:kworker.0 00000001:02000400:1.0:1787826826.687342:0:78699:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: local REPLAY BARRIER on lustre-MDT0000 00010000:00010000:3.0:1787826827.060814:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826827.365160:0:15156:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:3.0:1787826827.365195:0:15156:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874669235487872 00000100:00100000:3.0:1787826827.365207:0:15156:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874669235487872 00000100:00100000:3.0:1787826827.365267:0:15156:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874669235488000 00000100:00100000:3.0:1787826827.365273:0:15156:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874669235488000 00000100:00100000:3.0:1787826827.365294:0:15156:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874669235488256 00000100:00100000:3.0:1787826827.365299:0:15156:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874669235488256 00000100:00100000:3.0:1787826827.365327:0:15156:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874669235488512 00000100:00100000:3.0:1787826827.365331:0:15156:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874669235488512 00000100:00100000:3.0:1787826827.365347:0:15156:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874669235488640 00000100:00100000:3.0:1787826827.365351:0:15156:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874669235488640 00010000:00010000:1.0:1787826827.365444:0:15163:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787826827.365464:0:15163:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-OST0001-osc-ffff985602bc9800 lock: 0000000005f20dd6/0xb3f83dad1b7e01d1 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x5e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x420000010000 nid: local remote: 0x5aa6d504d91a10d6 expref: -99 pid: 78053 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826827.365528:0:15163:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9856129ecc00 already unused, calling callback (ffffffffc13b01e0) 00010000:00010000:1.0:1787826827.365536:0:15163:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0001-osc-ffff985602bc9800 lock: 0000000005f20dd6/0xb3f83dad1b7e01d1 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x5e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x428400010000 nid: local remote: 0x5aa6d504d91a10d6 expref: -99 pid: 78053 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000100:00100000:3.0:1787826827.365594:0:15156:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.114@tcp, seq: 239 00010000:00010000:1.0:1787826827.365603:0:15163:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-OST0001-osc-ffff985602bc9800 lock: 0000000005f20dd6/0xb3f83dad1b7e01d1 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x5e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x4c29400010020 nid: local remote: 0x5aa6d504d91a10d6 expref: -99 pid: 78053 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000100:00100000:3.0:1787826827.365611:0:15156:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff985602ee0780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb01_001:LOV_OSC_UUID+7:17106:x1874669235488000:12345-192.168.203.114@tcp:104: 00010000:00010000:3.0:1787826827.365637:0:15156:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-OST0000-osc-ffff985602bc9800 lock: 00000000c9e21032/0xb3f83dad1b7e0192 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5e4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x420000010000 nid: local remote: 0x5aa6d504d91a1082 expref: -99 pid: 78052 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826827.365641:0:15163:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826827.365666:0:15163:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-OST0001-osc-ffff985602bc9800 lock: 0000000005f20dd6/0xb3f83dad1b7e01d1 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x5e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x4c29400010020 nid: local remote: 0x5aa6d504d91a10d6 expref: -99 pid: 78053 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826827.365739:0:15163:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0001-osc-ffff985602bc9800 lock: 0000000005f20dd6/0xb3f83dad1b7e01d1 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x5e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x4c29400010020 nid: local remote: 0x5aa6d504d91a10d6 expref: -99 pid: 78053 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826827.365770:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826827.365775:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826827.365779:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826827.365788:0:15156:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/0 locks to priority blp list, 0 blwis in pool 00000100:00100000:3.0:1787826827.365797:0:15156:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff985602ee0780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb01_001:LOV_OSC_UUID+7:17106:x1874669235488000:12345-192.168.203.114@tcp:104: Request processed in 193us (787us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826827.365802:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610d3ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669225554048:192.168.203.114@tcp:103:ldlm_bl.0 00000100:00100000:3.0:1787826827.365810:0:15156:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.114@tcp, seq: 239 00000100:00100000:1.0:1787826827.365882:0:2384:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1787826827.365887:0:15156:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.114@tcp, seq: 240 00000100:00100000:1.0:1787826827.365895:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e792c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669225553920:192.168.203.114@tcp:103:ldlm_bl.0 00000100:00100000:3.0:1787826827.365907:0:15156:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff985602ee1680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb01_001:LOV_OSC_UUID+5:43928:x1874669235488128:12345-192.168.203.114@tcp:104: 00010000:00010000:3.0:1787826827.365925:0:15156:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-OST0000-osc-ffff985602bc9800 lock: 000000007f725c53/0xb3f83dad1b7e01d8 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x420000010000 nid: local remote: 0x5aa6d504d91a10dd expref: -99 pid: 78053 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000100:00100000:1.0:1787826827.365941:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e78000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669225554176:192.168.203.114@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1787826827.366012:0:2383:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00010000:00010000:3.0:1787826827.366019:0:15156:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to priority blp list, 2 blwis in pool 00000100:00100000:1.0:1787826827.366030:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e78b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669225554304:192.168.203.114@tcp:103:ldlm_bl.0 00000100:00100000:3.0:1787826827.366037:0:15156:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff985602ee1680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb01_001:LOV_OSC_UUID+5:43928:x1874669235488128:12345-192.168.203.114@tcp:104: Request processed in 131us (996us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826827.366080:0:15156:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.114@tcp, seq: 240 00010000:00010000:1.0:1787826827.366101:0:15163:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787826827.366109:0:15163:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-OST0000-osc-ffff985602bc9800 lock: 000000007f725c53/0xb3f83dad1b7e01d8 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x420000010000 nid: local remote: 0x5aa6d504d91a10dd expref: -99 pid: 78053 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000100:00100000:3.0:1787826827.366124:0:15156:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.114@tcp, seq: 243 00010000:00010000:1.0:1787826827.366127:0:15163:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9856129ece00 already unused, calling callback (ffffffffc13b01e0) 00010000:00010000:1.0:1787826827.366139:0:15163:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0000-osc-ffff985602bc9800 lock: 000000007f725c53/0xb3f83dad1b7e01d8 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x428400010000 nid: local remote: 0x5aa6d504d91a10dd expref: -99 pid: 78053 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000100:00100000:3.0:1787826827.366147:0:15156:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff985602ee0f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb01_001:LOV_OSC_UUID+5:43896:x1874669235488384:12345-192.168.203.114@tcp:104: 00010000:00010000:3.0:1787826827.366172:0:15156:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-OST0001-osc-ffff985602bc9800 lock: 00000000af8864da/0xb3f83dad1b7e01a0 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x5e5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x420000010000 nid: local remote: 0x5aa6d504d91a10ba expref: -99 pid: 78052 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826827.366208:0:15163:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-OST0000-osc-ffff985602bc9800 lock: 000000007f725c53/0xb3f83dad1b7e01d8 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x4c29400010020 nid: local remote: 0x5aa6d504d91a10dd expref: -99 pid: 78053 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826827.366269:0:15156:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:3.0:1787826827.366287:0:15156:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff985602ee0f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb01_001:LOV_OSC_UUID+5:43896:x1874669235488384:12345-192.168.203.114@tcp:104: Request processed in 146us (1219us total) trans 0 rc 0/0 00010000:00010000:1.0:1787826827.366309:0:15163:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:3.0:1787826827.366318:0:15156:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.114@tcp, seq: 243 00010000:00010000:1.0:1787826827.366325:0:15163:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-OST0000-osc-ffff985602bc9800 lock: 000000007f725c53/0xb3f83dad1b7e01d8 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x5e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x4c29400010020 nid: local remote: 0x5aa6d504d91a10dd expref: -99 pid: 78053 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826827.366341:0:15163:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0000-osc-ffff985602bc9800 lock: 000000007f725c53/0xb3f83dad1b7e01d8 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x4c29400010020 nid: local remote: 0x5aa6d504d91a10dd expref: -99 pid: 78053 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000100:00100000:3.0:1787826827.366345:0:15156:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.114@tcp, seq: 244 00000100:00100000:3.0:1787826827.366373:0:15156:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff985605621450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb01_001:LOV_OSC_UUID+4:43898:x1874669235488640:12345-192.168.203.114@tcp:104: 00010000:00010000:3.0:1787826827.366397:0:15156:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-OST0001-osc-ffff985602bc9800 lock: 00000000b57f056c/0xb3f83dad1b7e014c lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x5e2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x420000010000 nid: local remote: 0x5aa6d504d91a0f8d expref: -99 pid: 78052 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826827.366402:0:15163:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787826827.366416:0:15163:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-OST0000-osc-ffff985602bc9800 lock: 0000000089b1b66a/0xb3f83dad1b7e0176 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5e3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x420000010000 nid: local remote: 0x5aa6d504d91a1020 expref: -99 pid: 78052 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826827.366435:0:15163:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985609eb1400 already unused, calling callback (ffffffffc13b01e0) 00010000:00010000:1.0:1787826827.366441:0:15163:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0000-osc-ffff985602bc9800 lock: 0000000089b1b66a/0xb3f83dad1b7e0176 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5e3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x428400010000 nid: local remote: 0x5aa6d504d91a1020 expref: -99 pid: 78052 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826827.366456:0:15156:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:3.0:1787826827.366469:0:15156:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff985605621450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb01_001:LOV_OSC_UUID+4:43898:x1874669235488640:12345-192.168.203.114@tcp:104: Request processed in 105us (1353us total) trans 0 rc 0/0 00010000:00010000:1.0:1787826827.366476:0:15163:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-OST0000-osc-ffff985602bc9800 lock: 0000000089b1b66a/0xb3f83dad1b7e0176 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5e3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x4c29400010020 nid: local remote: 0x5aa6d504d91a1020 expref: -99 pid: 78052 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000100:00100000:3.0:1787826827.366479:0:15156:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.114@tcp, seq: 244 00010000:00010000:1.0:1787826827.366500:0:15163:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826827.366508:0:15163:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-OST0000-osc-ffff985602bc9800 lock: 0000000089b1b66a/0xb3f83dad1b7e0176 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x5e3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x4c29400010020 nid: local remote: 0x5aa6d504d91a1020 expref: -99 pid: 78052 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826827.366527:0:15163:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0000-osc-ffff985602bc9800 lock: 0000000089b1b66a/0xb3f83dad1b7e0176 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5e3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x4c29400010020 nid: local remote: 0x5aa6d504d91a1020 expref: -99 pid: 78052 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826827.366531:0:15164:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787826827.366540:0:15164:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-OST0001-osc-ffff985602bc9800 lock: 00000000b57f056c/0xb3f83dad1b7e014c lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x5e2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x420000010000 nid: local remote: 0x5aa6d504d91a0f8d expref: -99 pid: 78052 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826827.366578:0:15164:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985610e07000 already unused, calling callback (ffffffffc13b01e0) 00010000:00010000:3.0:1787826827.366585:0:15164:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0001-osc-ffff985602bc9800 lock: 00000000b57f056c/0xb3f83dad1b7e014c lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x5e2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x428400010000 nid: local remote: 0x5aa6d504d91a0f8d expref: -99 pid: 78052 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826827.366589:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826827.366594:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826827.366597:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826827.366621:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610d3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669225554560:192.168.203.114@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826827.366622:0:15164:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-OST0001-osc-ffff985602bc9800 lock: 00000000b57f056c/0xb3f83dad1b7e014c lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x5e2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x4c29400010020 nid: local remote: 0x5aa6d504d91a0f8d expref: -99 pid: 78052 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826827.366636:0:15164:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:1.0:1787826827.366664:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610d3e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669225554816:192.168.203.114@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826827.366665:0:15164:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-OST0001-osc-ffff985602bc9800 lock: 00000000b57f056c/0xb3f83dad1b7e014c lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x5e2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x4c29400010020 nid: local remote: 0x5aa6d504d91a0f8d expref: -99 pid: 78052 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826827.366678:0:15164:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0001-osc-ffff985602bc9800 lock: 00000000b57f056c/0xb3f83dad1b7e014c lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x5e2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x4c29400010020 nid: local remote: 0x5aa6d504d91a0f8d expref: -99 pid: 78052 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826827.366721:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826827.366723:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826827.366729:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826827.366748:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e4fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669225554944:192.168.203.114@tcp:103:ldlm_bl.0 00000100:00100000:3.0:1787826827.366831:0:33343:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874669235489024 00000100:00100000:3.0:1787826827.366837:0:33343:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874669235489024 00000100:00100000:3.0:1787826827.366875:0:33343:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.114@tcp, seq: 245 00000100:00100000:3.0:1787826827.366886:0:33343:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff985602ee03c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb01_002:LOV_OSC_UUID+4:43900:x1874669235489024:12345-192.168.203.114@tcp:104: 00010000:00010000:3.0:1787826827.366896:0:33343:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-OST0001-osc-ffff985602bc9800 lock: 00000000e6dcfdb4/0xb3f83dad1b7e0184 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x5e4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x420000010000 nid: local remote: 0x5aa6d504d91a1051 expref: -99 pid: 78052 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826827.366943:0:33343:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:3.0:1787826827.366950:0:33343:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff985602ee03c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb01_002:LOV_OSC_UUID+4:43900:x1874669235489024:12345-192.168.203.114@tcp:104: Request processed in 67us (420us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826827.366960:0:33343:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.114@tcp, seq: 245 00000100:00100000:1.0:1787826827.367109:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e79680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669225555072:192.168.203.114@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1787826827.370103:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610d3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669225554560:192.168.203.114@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1787826827.370131:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610d3e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669225554816:192.168.203.114@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1787826827.370145:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e792c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669225553920:192.168.203.114@tcp:103:ldlm_bl.0 00000100:00100000:3.0:1787826827.370145:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e4fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669225554944:192.168.203.114@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1787826827.370155:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e78000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669225554176:192.168.203.114@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1787826827.370174:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610d3ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669225554048:192.168.203.114@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787826828.084837:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826830.132862:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826831.604834:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610d3e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669225555328:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787826831.604934:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610d3d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669225555584:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787826831.604969:0:2383:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1787826831.604977:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610d3d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669225555712:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787826831.606201:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610d3d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669225555584:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787826831.606902:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610d3d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669225555712:192.168.203.114@tcp:400:kworker.0 00000100:00080000:1.0:1787826831.606916:0:2383:0:(recover.c:205:ptlrpc_request_handle_notconn()) import lustre-MDT0000-mdc-ffff985602bc9800 of lustre-MDT0000_UUID@192.168.203.114@tcp abruptly disconnected: reconnecting 00000100:00080000:1.0:1787826831.606921:0:2383:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from FULL to DISCONN 00000100:02000400:1.0:1787826831.606925:0:2383:0:(import.c:182:ptlrpc_set_import_discon()) lustre-MDT0000-mdc-ffff985602bc9800: Connection to lustre-MDT0000 (at 192.168.203.114@tcp) was lost; in progress operations using this service will wait for recovery to complete 00000100:00080000:1.0:1787826831.606930:0:2383:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787826831.606934:0:2383:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 2961 00000100:00080000:1.0:1787826831.606944:0:2383:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787826831.606953:0:2383:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787826831.606977:0:2383:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610d3d2c0 x1874669225555840/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826831.607009:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610d3e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669225555328:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787826831.607074:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610d3d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669225555840:192.168.203.114@tcp:38:ptlrpcd_00_00.0 00000100:00080000:1.0:1787826831.610067:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787826831.610078:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-19) 00000100:00100000:1.0:1787826831.610083:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610d3d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669225555840:192.168.203.114@tcp:38:ptlrpcd_00_00.0 00010000:00010000:1.0:1787826832.180872:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826834.228974:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826836.276879:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787826836.724916:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826836.724925:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787826836.724929:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787826836.724942:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 2982 00000100:00080000:1.0:1787826836.724955:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 5s 00000100:00080000:1.0:1787826836.724964:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787826836.724978:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787826836.725003:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610d3d680 x1874669225556224/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826836.725080:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610d3d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669225556224:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787826836.725107:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610d3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669225556352:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787826836.725161:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610d3f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669225556608:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787826836.727060:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610d3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669225556352:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787826836.727083:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610d3f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669225556608:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787826836.727737:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610d3e1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:3c946890-15ca-497a-b838-fe1cba61f4dd:2384:1874669225555200:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787826836.727888:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e79e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669225556736:192.168.203.114@tcp:250:ptlrpcd_00_00.0 00010000:00080000:3.0:1787826836.728955:0:2382:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=46162) req@ffff985610d3d680 x1874669225556224/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/416 e 0 to 0 dl 1787826851 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826836.728974:0:2382:0:(import.c:1132:ptlrpc_connect_interpret()) lustre-MDT0000-mdc-ffff985602bc9800: connect to target with instance 46 00000100:00080000:3.0:1787826836.729040:0:2382:0:(import.c:1338:ptlrpc_connect_interpret()) @@@ lustre-MDT0000-mdc-ffff985602bc9800: evicting (reconnect/recover flags not set: 4) req@ffff985610d3d680 x1874669225556224/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/416 e 0 to 0 dl 1787826851 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826836.729058:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to EVICTED 00000100:02020000:3.0:1787826836.729064:0:2382:0:(import.c:1663:ptlrpc_import_recovery_state_machine()) lustre-MDT0000-mdc-ffff985602bc9800: This client was evicted by lustre-MDT0000; in progress operations using this service will fail. 00000100:00080000:3.0:1787826836.733539:0:2382:0:(import.c:1674:ptlrpc_import_recovery_state_machine()) evicted from lustre-MDT0000_UUID@192.168.203.114@tcp; invalidating 00000100:00100000:3.0:1787826836.733832:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610d3d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669225556224:192.168.203.114@tcp:38:kworker.0 00000100:00080000:3.0:1787826836.733887:0:2382:0:(import.c:1132:ptlrpc_connect_interpret()) MGC192.168.203.114@tcp: connect to target with instance 0 10000000:01000000:3.0:1787826836.733898:0:2382:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808005 00000100:02000400:3.0:1787826836.733906:0:2382:0:(import.c:1284:ptlrpc_connect_interpret()) Evicted from MGS (at 192.168.203.114@tcp) after server handle changed from 0x5aa6d504d91a0c8b to 0x5aa6d504d91a176d 00000100:00080000:3.0:1787826836.733914:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to EVICTED 00000100:00080000:3.0:1787826836.733928:0:2382:0:(import.c:1674:ptlrpc_import_recovery_state_machine()) evicted from MGS@192.168.203.114@tcp; invalidating 00000100:00100000:3.0:1787826836.734085:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e79e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669225556736:192.168.203.114@tcp:250:ptlrpcd_00_00.0 00000100:00080000:1.0:1787826836.734108:0:79023:0:(import.c:1598:ptlrpc_invalidate_import_thread()) thread invalidate import MGC192.168.203.114@tcp to MGS@192.168.203.114@tcp 00000100:00100000:1.0:1787826836.734116:0:79023:0:(import.c:323:ptlrpc_invalidate_import()) Sleeping 20s for inflight to error out 10000000:01000000:1.0:1787826836.734128:0:79023:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808003 00010000:00010000:1.0:1787826836.734149:0:79023:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.114@tcp lock: ffff985610e9b800/0xb3f83dad1b7e00a4 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x110e400000000 nid: local remote: 0x5aa6d504d91a0d5d expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787826836.734161:0:79023:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.114@tcp lock: ffff985610e9b800/0xb3f83dad1b7e00a4 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x110f400000000 nid: local remote: 0x5aa6d504d91a0d5d expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787826836.734174:0:79023:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:1.0:1787826836.734193:0:79023:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787826836.734210:0:79023:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.114@tcp lock: ffff985610e9b800/0xb3f83dad1b7e00a4 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x590f400000000 nid: local remote: 0x5aa6d504d91a0d5d expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826836.734236:0:79023:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.114@tcp lock: ffff985610e9b600/0xb3f83dad1b7e00b2 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x110e400000000 nid: local remote: 0x5aa6d504d91a0d6b expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787826836.734250:0:79023:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.114@tcp lock: ffff985610e9b600/0xb3f83dad1b7e00b2 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x110f400000000 nid: local remote: 0x5aa6d504d91a0d6b expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787826836.734257:0:79023:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:1.0:1787826836.734266:0:79023:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787826836.734271:0:79023:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.114@tcp lock: ffff985610e9b600/0xb3f83dad1b7e00b2 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x590f400000000 nid: local remote: 0x5aa6d504d91a0d6b expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826836.734298:0:79023:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.114@tcp lock: ffff985610e9b000/0xb3f83dad1b7e009d lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x110e400000000 nid: local remote: 0x5aa6d504d91a0d56 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787826836.734305:0:79023:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.114@tcp lock: ffff985610e9b000/0xb3f83dad1b7e009d lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x110f400000000 nid: local remote: 0x5aa6d504d91a0d56 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787826836.734309:0:79023:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x2:0x0].0x0 (lustre) 00010000:00010000:1.0:1787826836.734317:0:79023:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787826836.734321:0:79023:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.114@tcp lock: ffff985610e9b000/0xb3f83dad1b7e009d lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x590f400000000 nid: local remote: 0x5aa6d504d91a0d56 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826836.734338:0:79023:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.114@tcp lock: ffff985609eb0c00/0xb3f83dad1b7e00ab lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x110e400000000 nid: local remote: 0x5aa6d504d91a0d64 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787826836.734347:0:79023:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.114@tcp lock: ffff985609eb0c00/0xb3f83dad1b7e00ab lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x110f400000000 nid: local remote: 0x5aa6d504d91a0d64 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787826836.734352:0:79023:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x736d61726170:0x3:0x0].0x0 (params) 00010000:00010000:1.0:1787826836.734358:0:79023:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787826836.734368:0:79023:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.114@tcp lock: ffff985609eb0c00/0xb3f83dad1b7e00ab lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x590f400000000 nid: local remote: 0x5aa6d504d91a0d64 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1787826836.734394:0:79023:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from EVICTED to RECOVER 00000100:00080000:1.0:1787826836.734406:0:79023:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from RECOVER to FULL 10000000:01000000:1.0:1787826836.734413:0:79023:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808004 00000100:00080000:1.0:1787826836.734427:0:79023:0:(pinger.c:186:ptlrpc_pinger_ir_up()) IR up 00000100:02000000:1.0:1787826836.734436:0:79023:0:(import.c:1743:ptlrpc_import_recovery_state_machine()) MGC192.168.203.114@tcp: Connection restored to 192.168.203.114@tcp (at 192.168.203.114@tcp) 00000100:02000000:1.0:1787826836.737577:0:79022:0:(import.c:1743:ptlrpc_import_recovery_state_machine()) lustre-MDT0000-mdc-ffff985602bc9800: Connection restored to 192.168.203.114@tcp (at 192.168.203.114@tcp) 10000000:01000000:3.0:1787826837.876759:0:15166:0:(mgc_request.c:611:do_requeue()) updating log lustre-cliir 10000000:01000000:3.0:1787826837.876765:0:15166:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-cliir-ffff985602bc9800 from 184 10000000:01000000:3.0:1787826837.876770:0:15166:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-cliir (res 0x65727473756c) 00010000:00010000:3.0:1787826837.876813:0:15166:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.114@tcp lock: ffff985610e06e00/0xb3f83dad1b7e0424 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826837.876824:0:15166:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.114@tcp lock: ffff985610e06e00/0xb3f83dad1b7e0424 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826837.876835:0:15166:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.114@tcp lock: ffff985610e06e00/0xb3f83dad1b7e0424 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826837.876859:0:15166:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e4c000 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669225556864:192.168.203.114@tcp:101:ll_cfg_requeue.0 00000100:00100000:3.0:1787826837.876928:0:15166:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607e06f00 going to sleep for 11 seconds 00000100:00100000:3.0:1787826837.878064:0:15166:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e4c000 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669225556864:192.168.203.114@tcp:101:ll_cfg_requeue.0 00010000:00010000:3.0:1787826837.878078:0:15166:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.114@tcp lock: ffff985610e06e00/0xb3f83dad1b7e0424 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x5aa6d504d91a1797 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826837.878085:0:15166:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.114@tcp lock: ffff985610e06e00/0xb3f83dad1b7e0424 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x5aa6d504d91a1797 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826837.878141:0:15166:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e4c000 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669225556992:192.168.203.114@tcp:256:ll_cfg_requeue.0 00000100:00100000:3.0:1787826837.878181:0:15166:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607e06f00 going to sleep for 11 seconds 00000100:00100000:1.0:1787826837.880126:0:15166:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e4c000 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669225556992:192.168.203.114@tcp:256:ll_cfg_requeue.0 10000000:01000000:1.0:1787826837.880205:0:15166:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.114@tcp: configuration from log 'lustre-cliir' succeeded (0). 00010000:00010000:1.0:1787826837.880213:0:15166:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.114@tcp lock: ffff985610e06e00/0xb3f83dad1b7e0424 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x5aa6d504d91a1797 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826837.880223:0:15166:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.114@tcp lock: ffff985610e06e00/0xb3f83dad1b7e0424 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x5aa6d504d91a1797 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787826837.880232:0:15166:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 10000000:01000000:1.0:1787826837.880235:0:15166:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff985602bc9800 from 52 10000000:01000000:1.0:1787826837.880238:0:15166:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00010000:00010000:1.0:1787826837.880261:0:15166:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.114@tcp lock: ffff985642051800/0xb3f83dad1b7e042b lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826837.880268:0:15166:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.114@tcp lock: ffff985642051800/0xb3f83dad1b7e042b lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826837.880277:0:15166:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.114@tcp lock: ffff985642051800/0xb3f83dad1b7e042b lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826837.880292:0:15166:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610d3f840 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669225557120:192.168.203.114@tcp:101:ll_cfg_requeue.0 00000100:00100000:1.0:1787826837.880339:0:15166:0:(client.c:2715:ptlrpc_set_wait()) set ffff98560883c780 going to sleep for 11 seconds 00000100:00100000:1.0:1787826837.881248:0:15166:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610d3f840 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669225557120:192.168.203.114@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787826837.881261:0:15166:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.114@tcp lock: ffff985642051800/0xb3f83dad1b7e042b lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x5aa6d504d91a179e expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826837.881270:0:15166:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.114@tcp lock: ffff985642051800/0xb3f83dad1b7e042b lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x5aa6d504d91a179e expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826837.881334:0:15166:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610d3f840 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669225557248:192.168.203.114@tcp:501:ll_cfg_requeue.0 00000100:00100000:1.0:1787826837.881375:0:15166:0:(client.c:2715:ptlrpc_set_wait()) set ffff98560883c780 going to sleep for 11 seconds 00000100:00100000:1.0:1787826837.882280:0:15166:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610d3f840 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669225557248:192.168.203.114@tcp:501:ll_cfg_requeue.0 00000100:00100000:1.0:1787826837.882314:0:15166:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610d3f840 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669225557376:192.168.203.114@tcp:503:ll_cfg_requeue.0 00000100:00100000:1.0:1787826837.882364:0:15166:0:(client.c:2715:ptlrpc_set_wait()) set ffff98560883c780 going to sleep for 11 seconds 00000100:00100000:1.0:1787826837.883254:0:15166:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610d3f840 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669225557376:192.168.203.114@tcp:503:ll_cfg_requeue.0 00000040:00080000:3.0:1787826837.883941:0:79051:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x3:0x0] index 64768 count 51 00000100:00100000:3.0:1787826837.885135:0:15166:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee1a40 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669225557504:192.168.203.114@tcp:101:ll_cfg_requeue.0 00010000:00010000:3.0:1787826837.885152:0:15166:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.114@tcp lock: ffff985610dc1e00/0xb3f83dad1b7e0432 lrc: 4/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x5aa6d504d91a17a5 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826837.885160:0:15166:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.114@tcp lock: ffff985610dc1e00/0xb3f83dad1b7e0432 lrc: 4/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x5aa6d504d91a17a5 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826837.885196:0:15166:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e4c780 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669225557632:192.168.203.114@tcp:501:ll_cfg_requeue.0 00000100:00100000:3.0:1787826837.885228:0:15166:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607e06600 going to sleep for 11 seconds 00000100:00100000:3.0:1787826837.886138:0:15166:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e4c780 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669225557632:192.168.203.114@tcp:501:ll_cfg_requeue.0 00000100:00100000:3.0:1787826837.886168:0:15166:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e4c780 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669225557760:192.168.203.114@tcp:503:ll_cfg_requeue.0 00000100:00100000:3.0:1787826837.886226:0:15166:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607e06600 going to sleep for 11 seconds 00000100:00100000:3.0:1787826837.887185:0:15166:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e4c780 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669225557760:192.168.203.114@tcp:503:ll_cfg_requeue.0 00000040:00080000:3.0:1787826837.887861:0:79053:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x2:0x0] index 64768 count 7 00000100:00100000:3.0:1787826837.889165:0:15166:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee2580 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669225557888:192.168.203.114@tcp:101:ll_cfg_requeue.0 00010000:00010000:3.0:1787826837.889182:0:15166:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.114@tcp lock: ffff985610dc0200/0xb3f83dad1b7e0439 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0x5aa6d504d91a17ac expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826837.889192:0:15166:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.114@tcp lock: ffff985610dc0200/0xb3f83dad1b7e0439 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1000000000000 nid: local remote: 0x5aa6d504d91a17ac expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826837.889229:0:15166:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e4c780 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669225558016:192.168.203.114@tcp:501:ll_cfg_requeue.0 00000100:00100000:3.0:1787826837.889266:0:15166:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607e06540 going to sleep for 11 seconds 00000100:00100000:3.0:1787826837.890014:0:15166:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e4c780 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669225558016:192.168.203.114@tcp:501:ll_cfg_requeue.0 10000000:01000000:3.0:1787826837.890040:0:15166:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.114@tcp: configuration from log 'lustre-sptlrpc' failed (-2). 00010000:00010000:3.0:1787826837.890046:0:15166:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.114@tcp lock: ffff985610dc0200/0xb3f83dad1b7e0439 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0x5aa6d504d91a17ac expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826837.890054:0:15166:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.114@tcp lock: ffff985610dc0200/0xb3f83dad1b7e0439 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0x5aa6d504d91a17ac expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826838.324825:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826840.372923:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826841.845017:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e4f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669225558912:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787826841.845307:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e4de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669225559168:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787826841.846794:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e4f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669225558912:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787826841.846808:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e4de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669225559168:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787826841.846845:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e4c780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2385:1874669225558784:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787826841.846855:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e4da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669225559040:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787826841.846864:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e4d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669225559296:192.168.203.114@tcp:400:kworker.0 00010000:00010000:3.0:1787826842.420876:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826843.445860:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826844.294195:0:79201:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee21c0 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:79201:1874669225559424:192.168.203.114@tcp:700:createmany.0 00000002:00010000:1.0:1787826844.294245:0:79201:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f33b.replay-single--0,[0x200016b60:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: open|creat flags 0100103 00010000:00010000:1.0:1787826844.294270:0:79201:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1800/0xb3f83dad1b7e0440 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x5aa6d504d91a17ba expref: -99 pid: 79064 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00010000:00010000:1.0:1787826844.294353:0:79201:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1800/0xb3f83dad1b7e0440 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a17ba expref: -99 pid: 79064 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00010000:00010000:1.0:1787826844.294359:0:79201:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826844.294362:0:79201:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1800/0xb3f83dad1b7e0440 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a17ba expref: -99 pid: 79064 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00010000:00010000:1.0:1787826844.294408:0:79201:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e0447 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 79201 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.294417:0:79201:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e0447 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 79201 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:1.0:1787826844.294429:0:79201:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:1.0:1787826844.294451:0:79201:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610d3f840 x1874669225559680/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:1.0:1787826844.294462:0:79201:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e0447 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 79201 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826844.294492:0:79201:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610d3f840 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:79201:1874669225559680:192.168.203.114@tcp:101:createmany.0 00000100:00100000:1.0:1787826844.294681:0:79201:0:(client.c:2715:ptlrpc_set_wait()) set ffff98560883cd80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826844.298992:0:79201:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610d3f840 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:79201:1874669225559680:192.168.203.114@tcp:101:createmany.0 00010000:00010000:1.0:1787826844.299026:0:79201:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e0447 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a17d6 expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.299059:0:79201:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e0447 lrc: 4/0,1 mode: --/PR res: [0x200016b60:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a17d6 expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.299081:0:79201:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e0447 lrc: 4/0,1 mode: --/PR res: [0x200016b60:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a17d6 expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.299094:0:79201:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e0447 lrc: 4/0,1 mode: PR/PR res: [0x200016b60:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a17d6 expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.299106:0:79201:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e0447 lrc: 4/0,1 mode: PR/PR res: [0x200016b60:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a17d6 expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.299127:0:79201:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e0447 lrc: 5/1,1 mode: PR/PR res: [0x200016b60:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a17d6 expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.299137:0:79201:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e0447 lrc: 5/1,1 mode: PR/PR res: [0x200016b60:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a17d6 expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.299145:0:79201:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e0447 lrc: 4/1,0 mode: PR/PR res: [0x200016b60:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a17d6 expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826844.299159:0:79201:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff985610d3f840 x1874669225559680/t197568495617(197568495617) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/664 e 0 to 0 dl 1787826860 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:1.0:1787826844.299172:0:79201:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff985610d3f840 x1874669225559680/t197568495617(197568495617) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/664 e 0 to 0 dl 1787826860 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826844.299185:0:79201:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e0447 lrc: 3/1,0 mode: PR/PR res: [0x200016b60:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a17d6 expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:1.0:1787826844.299198:0:79201:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e0447 lrc: 3/1,0 mode: PR/PR res: [0x200016b60:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a17d6 expref: -99 pid: 79201 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.299210:0:79201:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8590027616/1 (0 0) 00000080:00200000:1.0:1787826844.299217:0:79201:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:1.0:1787826844.299234:0:79201:0:(namei.c:1628:ll_create_it()) VFS Op:name=f33b.replay-single--0, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=open|creat 00000080:00200000:1.0:1787826844.299269:0:79201:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200016b60:0x1:0x0](ffff98561158aa08) 00000080:00200000:1.0:1787826844.299360:0:79201:0:(file.c:6854:ll_layout_conf()) [0x200016b60:0x1:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:1.0:1787826844.299367:0:79201:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x200016b60:0x1:0x0](ffff98561158aa08) 00000080:00010000:1.0:1787826844.299371:0:79201:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200016b60:0x1:0x0](ffff98561158aa08) for lock 0xb3f83dad1b7e0447 00000080:00010000:1.0:1787826844.299382:0:79201:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200016b60:0x1:0x0](ffff98561158aa08) 00000080:00200008:1.0:1787826844.299401:0:79201:0:(file.c:1090:ll_file_open()) START file f33b.replay-single--0:[0x200016b60:0x1:0x0](ffff98561158aa08), flags 100102 00000002:00100000:1.0:1787826844.299423:0:79201:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff985610d3f840 x1874669225559680/t197568495617(197568495617) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826860 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:1.0:1787826844.299460:0:79201:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0447 from it ffff9856095bb720 00010000:00010000:1.0:1787826844.299465:0:79201:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e0447 lrc: 3/1,0 mode: PR/PR res: [0x200016b60:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a17d6 expref: -99 pid: 79201 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.299474:0:79201:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e0447 lrc: 2/0,0 mode: PR/PR res: [0x200016b60:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a17d6 expref: -99 pid: 79201 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826844.299500:0:79201:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200016b60:0x1:0x0](ffff98561158aa08), inode mode 81a4 mask 40 00000080:00200008:1.0:1787826844.299537:0:79201:0:(file.c:479:ll_file_release()) START file f33b.replay-single--0:[0x200016b60:0x1:0x0](ffff98561158aa08), flags 100002 00000002:00100000:1.0:1787826844.299589:0:79201:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff985610d3f840 x1874669225559680/t197568495617(197568495617) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826860 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:1.0:1787826844.299601:0:79201:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:1.0:1787826844.299606:0:79201:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610d3c3c0 x1874669225559936/t0(0) o35->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:1.0:1787826844.299631:0:79201:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610d3c3c0 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:79201:1874669225559936:192.168.203.114@tcp:35:createmany.0 00000100:00100000:1.0:1787826844.299737:0:79201:0:(client.c:2715:ptlrpc_set_wait()) set ffff98560883ca80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826844.301249:0:79201:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610d3c3c0 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:79201:1874669225559936:192.168.203.114@tcp:35:createmany.0 00000002:00100000:1.0:1787826844.301350:0:79201:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff985610d3f840 x1874669225559680/t197568495617(197568495617) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826860 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:1.0:1787826844.301435:0:79201:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826844.301454:0:79201:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826844.301524:0:79201:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e044e lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 79201 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.301535:0:79201:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e044e lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 79201 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.301545:0:79201:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e044e lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 79201 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826844.301563:0:79201:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610d3c780 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:79201:1874669225560064:192.168.203.114@tcp:101:createmany.0 00000100:00100000:1.0:1787826844.301633:0:79201:0:(client.c:2715:ptlrpc_set_wait()) set ffff98560883ca80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826844.304797:0:79201:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610d3c780 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:79201:1874669225560064:192.168.203.114@tcp:101:createmany.0 00010000:00010000:1.0:1787826844.304832:0:79201:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e044e lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a17eb expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.304847:0:79201:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e044e lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a17eb expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.304862:0:79201:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e044e lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a17eb expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.304874:0:79201:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e044e lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a17eb expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.304885:0:79201:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e044e lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a17eb expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.304891:0:79201:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e044e lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a17eb expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.304897:0:79201:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e044e lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a17eb expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826844.304910:0:79201:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff985610d3c780 x1874669225560064/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826860 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826844.304925:0:79201:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e044e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a17eb expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.304940:0:79201:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:1.0:1787826844.304947:0:79201:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:1.0:1787826844.304987:0:79201:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:1.0:1787826844.304994:0:79201:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e044e 00000080:00010000:1.0:1787826844.305001:0:79201:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826844.305005:0:79201:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e044e from it ffffab3ac1cefbf8 00010000:00010000:1.0:1787826844.305008:0:79201:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e044e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a17eb expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.305019:0:79201:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e044e lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a17eb expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826844.305048:0:79201:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787826844.305092:0:79201:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826844.305109:0:79201:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826844.305119:0:79201:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e044e lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a17eb expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826844.305134:0:79201:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e044e lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a17eb expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826844.305141:0:79201:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826844.305144:0:79201:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826844.305147:0:79201:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e044e 00000080:00010000:1.0:1787826844.305151:0:79201:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e044e from it ffffab3ac1cefb38 00010000:00010000:1.0:1787826844.305154:0:79201:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e044e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a17eb expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826844.305160:0:79201:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e044e lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a17eb expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826844.305168:0:79201:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787826844.305173:0:79201:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f33b.replay-single--1, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), file ffff9856206c1980, open_flags 8042, mode 81a4 opened 0 00000080:00200000:1.0:1787826844.305182:0:79201:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f33b.replay-single--1, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=open|creat 00000002:00010000:1.0:1787826844.305204:0:79201:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f33b.replay-single--1,[0x200016b60:0x2:0x0]) in obj [0x200000007:0x1:0x0], intent: open|creat flags 0100103 00010000:00010000:1.0:1787826844.305216:0:79201:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e044e lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x5aa6d504d91a17eb expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826844.305268:0:79201:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e044e lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a17eb expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826844.305274:0:79201:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826844.305276:0:79201:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e044e lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a17eb expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826844.305319:0:79201:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050a00/0xb3f83dad1b7e0455 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 79201 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.305326:0:79201:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050a00/0xb3f83dad1b7e0455 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 79201 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:1.0:1787826844.305337:0:79201:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:1.0:1787826844.305343:0:79201:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610d3c780 x1874669225560320/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:1.0:1787826844.305350:0:79201:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050a00/0xb3f83dad1b7e0455 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 79201 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826844.305370:0:79201:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610d3c780 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:79201:1874669225560320:192.168.203.114@tcp:101:createmany.0 00000100:00100000:1.0:1787826844.305773:0:79201:0:(client.c:2715:ptlrpc_set_wait()) set ffff98560883ca80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826844.310796:0:79201:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610d3c780 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:79201:1874669225560320:192.168.203.114@tcp:101:createmany.0 00010000:00010000:1.0:1787826844.310838:0:79201:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050a00/0xb3f83dad1b7e0455 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1807 expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.310871:0:79201:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050a00/0xb3f83dad1b7e0455 lrc: 4/0,1 mode: --/PR res: [0x200016b60:0x2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1807 expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.310882:0:79201:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050a00/0xb3f83dad1b7e0455 lrc: 4/0,1 mode: --/PR res: [0x200016b60:0x2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1807 expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.310896:0:79201:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050a00/0xb3f83dad1b7e0455 lrc: 4/0,1 mode: PR/PR res: [0x200016b60:0x2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1807 expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.310914:0:79201:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050a00/0xb3f83dad1b7e0455 lrc: 4/0,1 mode: PR/PR res: [0x200016b60:0x2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1807 expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.310929:0:79201:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050a00/0xb3f83dad1b7e0455 lrc: 5/1,1 mode: PR/PR res: [0x200016b60:0x2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1807 expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.310938:0:79201:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050a00/0xb3f83dad1b7e0455 lrc: 5/1,1 mode: PR/PR res: [0x200016b60:0x2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1807 expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.310946:0:79201:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050a00/0xb3f83dad1b7e0455 lrc: 4/1,0 mode: PR/PR res: [0x200016b60:0x2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1807 expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826844.310970:0:79201:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff985610d3c780 x1874669225560320/t197568495619(197568495619) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/664 e 0 to 0 dl 1787826860 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:1.0:1787826844.310984:0:79201:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff985610d3c780 x1874669225560320/t197568495619(197568495619) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/664 e 0 to 0 dl 1787826860 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826844.310999:0:79201:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050a00/0xb3f83dad1b7e0455 lrc: 3/1,0 mode: PR/PR res: [0x200016b60:0x2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1807 expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:1.0:1787826844.311012:0:79201:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050a00/0xb3f83dad1b7e0455 lrc: 3/1,0 mode: PR/PR res: [0x200016b60:0x2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1807 expref: -99 pid: 79201 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.311022:0:79201:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8590027616/2 (0 0) 00000080:00200000:1.0:1787826844.311030:0:79201:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:1.0:1787826844.311047:0:79201:0:(namei.c:1628:ll_create_it()) VFS Op:name=f33b.replay-single--1, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=open|creat 00000080:00200000:1.0:1787826844.311090:0:79201:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200016b60:0x2:0x0](ffff98561158a1c8) 00000080:00200000:1.0:1787826844.311189:0:79201:0:(file.c:6854:ll_layout_conf()) [0x200016b60:0x2:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:1.0:1787826844.311194:0:79201:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x200016b60:0x2:0x0](ffff98561158a1c8) 00000080:00010000:1.0:1787826844.311197:0:79201:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200016b60:0x2:0x0](ffff98561158a1c8) for lock 0xb3f83dad1b7e0455 00000080:00010000:1.0:1787826844.311209:0:79201:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200016b60:0x2:0x0](ffff98561158a1c8) 00000080:00200008:1.0:1787826844.311226:0:79201:0:(file.c:1090:ll_file_open()) START file f33b.replay-single--1:[0x200016b60:0x2:0x0](ffff98561158a1c8), flags 100102 00000002:00100000:1.0:1787826844.311252:0:79201:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff985610d3c780 x1874669225560320/t197568495619(197568495619) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826860 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:1.0:1787826844.311308:0:79201:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0455 from it ffff985610e3f480 00010000:00010000:1.0:1787826844.311313:0:79201:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050a00/0xb3f83dad1b7e0455 lrc: 3/1,0 mode: PR/PR res: [0x200016b60:0x2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a1807 expref: -99 pid: 79201 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.311323:0:79201:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050a00/0xb3f83dad1b7e0455 lrc: 2/0,0 mode: PR/PR res: [0x200016b60:0x2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a1807 expref: -99 pid: 79201 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826844.311349:0:79201:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200016b60:0x2:0x0](ffff98561158a1c8), inode mode 81a4 mask 40 00000080:00200008:1.0:1787826844.311386:0:79201:0:(file.c:479:ll_file_release()) START file f33b.replay-single--1:[0x200016b60:0x2:0x0](ffff98561158a1c8), flags 100002 00000002:00100000:1.0:1787826844.311439:0:79201:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff985610d3c780 x1874669225560320/t197568495619(197568495619) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826860 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:1.0:1787826844.311451:0:79201:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:1.0:1787826844.311457:0:79201:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610d3cf00 x1874669225560576/t0(0) o35->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:1.0:1787826844.311482:0:79201:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610d3cf00 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:79201:1874669225560576:192.168.203.114@tcp:35:createmany.0 00000100:00100000:1.0:1787826844.311583:0:79201:0:(client.c:2715:ptlrpc_set_wait()) set ffff98560883cc00 going to sleep for 11 seconds 00000100:00100000:1.0:1787826844.316923:0:79201:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610d3cf00 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:79201:1874669225560576:192.168.203.114@tcp:35:createmany.0 00000002:00100000:1.0:1787826844.316985:0:79201:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff985610d3c780 x1874669225560320/t197568495619(197568495619) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826860 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:1.0:1787826844.317209:0:79201:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826844.317233:0:79201:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826844.317308:0:79201:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e045c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 79201 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.317320:0:79201:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e045c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 79201 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.317330:0:79201:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e045c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 79201 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826844.317354:0:79201:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610d3cb40 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:79201:1874669225560704:192.168.203.114@tcp:101:createmany.0 00000100:00100000:1.0:1787826844.317453:0:79201:0:(client.c:2715:ptlrpc_set_wait()) set ffff98560883cc00 going to sleep for 11 seconds 00000100:00100000:1.0:1787826844.319044:0:79201:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610d3cb40 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:79201:1874669225560704:192.168.203.114@tcp:101:createmany.0 00010000:00010000:1.0:1787826844.319080:0:79201:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e045c lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a181c expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.319093:0:79201:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e045c lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a181c expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.319109:0:79201:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e045c lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a181c expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.319126:0:79201:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e045c lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a181c expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.319145:0:79201:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e045c lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a181c expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.319153:0:79201:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e045c lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a181c expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.319161:0:79201:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e045c lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a181c expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826844.319174:0:79201:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff985610d3cb40 x1874669225560704/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826860 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826844.319188:0:79201:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e045c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a181c expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.319201:0:79201:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:1.0:1787826844.319209:0:79201:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:1.0:1787826844.319241:0:79201:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:1.0:1787826844.319249:0:79201:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e045c 00000080:00010000:1.0:1787826844.319255:0:79201:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826844.319259:0:79201:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e045c from it ffffab3ac1cefbf8 00010000:00010000:1.0:1787826844.319262:0:79201:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e045c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a181c expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.319272:0:79201:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e045c lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a181c expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826844.319306:0:79201:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787826844.319334:0:79201:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826844.319351:0:79201:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826844.319361:0:79201:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e045c lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a181c expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826844.319373:0:79201:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e045c lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a181c expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826844.319380:0:79201:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826844.319384:0:79201:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826844.319387:0:79201:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e045c 00000080:00010000:1.0:1787826844.319391:0:79201:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e045c from it ffffab3ac1cefb38 00010000:00010000:1.0:1787826844.319394:0:79201:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e045c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a181c expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826844.319401:0:79201:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e045c lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a181c expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826844.319409:0:79201:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787826844.319414:0:79201:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f33b.replay-single--2, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), file ffff9856206c0900, open_flags 8042, mode 81a4 opened 0 00000080:00200000:1.0:1787826844.319425:0:79201:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f33b.replay-single--2, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=open|creat 00000002:00010000:1.0:1787826844.319447:0:79201:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f33b.replay-single--2,[0x200016b60:0x3:0x0]) in obj [0x200000007:0x1:0x0], intent: open|creat flags 0100103 00010000:00010000:1.0:1787826844.319459:0:79201:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e045c lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x5aa6d504d91a181c expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826844.319513:0:79201:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e045c lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a181c expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826844.319521:0:79201:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826844.319524:0:79201:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e045c lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a181c expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826844.319560:0:79201:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e0463 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 79201 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.319568:0:79201:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e0463 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 79201 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:1.0:1787826844.319580:0:79201:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:1.0:1787826844.319588:0:79201:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610d3cb40 x1874669225560960/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:1.0:1787826844.319598:0:79201:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e0463 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 79201 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826844.319622:0:79201:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610d3cb40 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:79201:1874669225560960:192.168.203.114@tcp:101:createmany.0 00000100:00100000:1.0:1787826844.319833:0:79201:0:(client.c:2715:ptlrpc_set_wait()) set ffff98560883cc00 going to sleep for 11 seconds 00000100:00100000:1.0:1787826844.323422:0:79201:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610d3cb40 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:79201:1874669225560960:192.168.203.114@tcp:101:createmany.0 00010000:00010000:1.0:1787826844.323460:0:79201:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e0463 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1838 expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.323498:0:79201:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e0463 lrc: 4/0,1 mode: --/PR res: [0x200016b60:0x3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1838 expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.323514:0:79201:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e0463 lrc: 4/0,1 mode: --/PR res: [0x200016b60:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1838 expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.323529:0:79201:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e0463 lrc: 4/0,1 mode: PR/PR res: [0x200016b60:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1838 expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.323542:0:79201:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e0463 lrc: 4/0,1 mode: PR/PR res: [0x200016b60:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1838 expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.323557:0:79201:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e0463 lrc: 5/1,1 mode: PR/PR res: [0x200016b60:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1838 expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.323566:0:79201:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e0463 lrc: 5/1,1 mode: PR/PR res: [0x200016b60:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1838 expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.323573:0:79201:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e0463 lrc: 4/1,0 mode: PR/PR res: [0x200016b60:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1838 expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826844.323586:0:79201:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff985610d3cb40 x1874669225560960/t197568495621(197568495621) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/664 e 0 to 0 dl 1787826860 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:1.0:1787826844.323600:0:79201:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff985610d3cb40 x1874669225560960/t197568495621(197568495621) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/664 e 0 to 0 dl 1787826860 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826844.323614:0:79201:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e0463 lrc: 3/1,0 mode: PR/PR res: [0x200016b60:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1838 expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:1.0:1787826844.323627:0:79201:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e0463 lrc: 3/1,0 mode: PR/PR res: [0x200016b60:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1838 expref: -99 pid: 79201 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.323639:0:79201:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8590027616/3 (0 0) 00000080:00200000:1.0:1787826844.323646:0:79201:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:1.0:1787826844.323664:0:79201:0:(namei.c:1628:ll_create_it()) VFS Op:name=f33b.replay-single--2, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=open|creat 00000080:00200000:1.0:1787826844.323750:0:79201:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200016b60:0x3:0x0](ffff98561158d348) 00000080:00200000:1.0:1787826844.323847:0:79201:0:(file.c:6854:ll_layout_conf()) [0x200016b60:0x3:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:1.0:1787826844.323853:0:79201:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x200016b60:0x3:0x0](ffff98561158d348) 00000080:00010000:1.0:1787826844.323857:0:79201:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200016b60:0x3:0x0](ffff98561158d348) for lock 0xb3f83dad1b7e0463 00000080:00010000:1.0:1787826844.323869:0:79201:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200016b60:0x3:0x0](ffff98561158d348) 00000080:00200008:1.0:1787826844.323886:0:79201:0:(file.c:1090:ll_file_open()) START file f33b.replay-single--2:[0x200016b60:0x3:0x0](ffff98561158d348), flags 100102 00000002:00100000:1.0:1787826844.323911:0:79201:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff985610d3cb40 x1874669225560960/t197568495621(197568495621) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826860 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:1.0:1787826844.323975:0:79201:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0463 from it ffff985610e3f480 00010000:00010000:1.0:1787826844.323980:0:79201:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e0463 lrc: 3/1,0 mode: PR/PR res: [0x200016b60:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a1838 expref: -99 pid: 79201 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.323990:0:79201:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e0463 lrc: 2/0,0 mode: PR/PR res: [0x200016b60:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a1838 expref: -99 pid: 79201 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826844.324015:0:79201:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200016b60:0x3:0x0](ffff98561158d348), inode mode 81a4 mask 40 00000080:00200008:1.0:1787826844.324055:0:79201:0:(file.c:479:ll_file_release()) START file f33b.replay-single--2:[0x200016b60:0x3:0x0](ffff98561158d348), flags 100002 00000002:00100000:1.0:1787826844.324119:0:79201:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff985610d3cb40 x1874669225560960/t197568495621(197568495621) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826860 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:1.0:1787826844.324131:0:79201:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:1.0:1787826844.324137:0:79201:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610d3e1c0 x1874669225561216/t0(0) o35->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:1.0:1787826844.324170:0:79201:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610d3e1c0 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:79201:1874669225561216:192.168.203.114@tcp:35:createmany.0 00000100:00100000:1.0:1787826844.324274:0:79201:0:(client.c:2715:ptlrpc_set_wait()) set ffff98560883c780 going to sleep for 11 seconds 00000100:00100000:1.0:1787826844.325916:0:79201:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610d3e1c0 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:79201:1874669225561216:192.168.203.114@tcp:35:createmany.0 00000002:00100000:1.0:1787826844.325968:0:79201:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff985610d3cb40 x1874669225560960/t197568495621(197568495621) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826860 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:1.0:1787826844.326083:0:79201:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826844.326103:0:79201:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826844.326169:0:79201:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050800/0xb3f83dad1b7e046a lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 79201 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.326180:0:79201:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050800/0xb3f83dad1b7e046a lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 79201 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.326191:0:79201:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050800/0xb3f83dad1b7e046a lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 79201 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826844.326225:0:79201:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610d3d680 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:79201:1874669225561344:192.168.203.114@tcp:101:createmany.0 00000100:00100000:1.0:1787826844.326315:0:79201:0:(client.c:2715:ptlrpc_set_wait()) set ffff98560883c780 going to sleep for 11 seconds 00000100:00100000:1.0:1787826844.333164:0:79201:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610d3d680 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:79201:1874669225561344:192.168.203.114@tcp:101:createmany.0 00010000:00010000:1.0:1787826844.333201:0:79201:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050800/0xb3f83dad1b7e046a lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a184d expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.333216:0:79201:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050800/0xb3f83dad1b7e046a lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a184d expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.333236:0:79201:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050800/0xb3f83dad1b7e046a lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a184d expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.333250:0:79201:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050800/0xb3f83dad1b7e046a lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a184d expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.333264:0:79201:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050800/0xb3f83dad1b7e046a lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a184d expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.333273:0:79201:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050800/0xb3f83dad1b7e046a lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a184d expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.333281:0:79201:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050800/0xb3f83dad1b7e046a lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a184d expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826844.333305:0:79201:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff985610d3d680 x1874669225561344/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826860 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826844.333320:0:79201:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050800/0xb3f83dad1b7e046a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a184d expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.333336:0:79201:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:1.0:1787826844.333343:0:79201:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:1.0:1787826844.333386:0:79201:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:1.0:1787826844.333425:0:79201:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e046a 00000080:00010000:1.0:1787826844.333433:0:79201:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826844.333436:0:79201:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e046a from it ffffab3ac1cefbf8 00010000:00010000:1.0:1787826844.333440:0:79201:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050800/0xb3f83dad1b7e046a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a184d expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.333450:0:79201:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050800/0xb3f83dad1b7e046a lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a184d expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826844.333480:0:79201:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787826844.333512:0:79201:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826844.333531:0:79201:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826844.333552:0:79201:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050800/0xb3f83dad1b7e046a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a184d expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826844.333564:0:79201:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050800/0xb3f83dad1b7e046a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a184d expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826844.333571:0:79201:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826844.333575:0:79201:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826844.333579:0:79201:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e046a 00000080:00010000:1.0:1787826844.333583:0:79201:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e046a from it ffffab3ac1cefb38 00010000:00010000:1.0:1787826844.333587:0:79201:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050800/0xb3f83dad1b7e046a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a184d expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826844.333593:0:79201:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050800/0xb3f83dad1b7e046a lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a184d expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826844.333601:0:79201:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787826844.333622:0:79201:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f33b.replay-single--3, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), file ffff9856206c1980, open_flags 8042, mode 81a4 opened 0 00000080:00200000:1.0:1787826844.333633:0:79201:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f33b.replay-single--3, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=open|creat 00000002:00010000:1.0:1787826844.333658:0:79201:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f33b.replay-single--3,[0x200016b60:0x4:0x0]) in obj [0x200000007:0x1:0x0], intent: open|creat flags 0100103 00010000:00010000:1.0:1787826844.333673:0:79201:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050800/0xb3f83dad1b7e046a lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x5aa6d504d91a184d expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826844.333752:0:79201:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050800/0xb3f83dad1b7e046a lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a184d expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826844.333760:0:79201:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826844.333763:0:79201:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050800/0xb3f83dad1b7e046a lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a184d expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826844.333821:0:79201:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e0471 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 79201 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.333829:0:79201:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e0471 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 79201 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:1.0:1787826844.333842:0:79201:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:1.0:1787826844.333848:0:79201:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610d3d680 x1874669225561600/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:1.0:1787826844.333855:0:79201:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e0471 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 79201 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826844.333874:0:79201:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610d3d680 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:79201:1874669225561600:192.168.203.114@tcp:101:createmany.0 00000100:00100000:1.0:1787826844.334056:0:79201:0:(client.c:2715:ptlrpc_set_wait()) set ffff98560883c780 going to sleep for 11 seconds 00000100:00100000:1.0:1787826844.337330:0:79201:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610d3d680 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:79201:1874669225561600:192.168.203.114@tcp:101:createmany.0 00010000:00010000:1.0:1787826844.337365:0:79201:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e0471 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1869 expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.337417:0:79201:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e0471 lrc: 4/0,1 mode: --/PR res: [0x200016b60:0x4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1869 expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.337427:0:79201:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e0471 lrc: 4/0,1 mode: --/PR res: [0x200016b60:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1869 expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.337442:0:79201:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e0471 lrc: 4/0,1 mode: PR/PR res: [0x200016b60:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1869 expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.337453:0:79201:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e0471 lrc: 4/0,1 mode: PR/PR res: [0x200016b60:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1869 expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.337468:0:79201:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e0471 lrc: 5/1,1 mode: PR/PR res: [0x200016b60:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1869 expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.337478:0:79201:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e0471 lrc: 5/1,1 mode: PR/PR res: [0x200016b60:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1869 expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.337486:0:79201:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e0471 lrc: 4/1,0 mode: PR/PR res: [0x200016b60:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1869 expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826844.337498:0:79201:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff985610d3d680 x1874669225561600/t197568495623(197568495623) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/664 e 0 to 0 dl 1787826860 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:1.0:1787826844.337513:0:79201:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff985610d3d680 x1874669225561600/t197568495623(197568495623) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/664 e 0 to 0 dl 1787826860 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826844.337527:0:79201:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e0471 lrc: 3/1,0 mode: PR/PR res: [0x200016b60:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1869 expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:1.0:1787826844.337540:0:79201:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e0471 lrc: 3/1,0 mode: PR/PR res: [0x200016b60:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1869 expref: -99 pid: 79201 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.337551:0:79201:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8590027616/4 (0 0) 00000080:00200000:1.0:1787826844.337559:0:79201:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:1.0:1787826844.337575:0:79201:0:(namei.c:1628:ll_create_it()) VFS Op:name=f33b.replay-single--3, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=open|creat 00000080:00200000:1.0:1787826844.337608:0:79201:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200016b60:0x4:0x0](ffff98561158db88) 00000080:00200000:1.0:1787826844.337725:0:79201:0:(file.c:6854:ll_layout_conf()) [0x200016b60:0x4:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:1.0:1787826844.337731:0:79201:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x200016b60:0x4:0x0](ffff98561158db88) 00000080:00010000:1.0:1787826844.337735:0:79201:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200016b60:0x4:0x0](ffff98561158db88) for lock 0xb3f83dad1b7e0471 00000080:00010000:1.0:1787826844.337746:0:79201:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200016b60:0x4:0x0](ffff98561158db88) 00000080:00200008:1.0:1787826844.337764:0:79201:0:(file.c:1090:ll_file_open()) START file f33b.replay-single--3:[0x200016b60:0x4:0x0](ffff98561158db88), flags 100102 00000002:00100000:1.0:1787826844.337796:0:79201:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff985610d3d680 x1874669225561600/t197568495623(197568495623) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826860 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:1.0:1787826844.337915:0:79201:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0471 from it ffff985610e3f480 00010000:00010000:1.0:1787826844.337921:0:79201:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e0471 lrc: 3/1,0 mode: PR/PR res: [0x200016b60:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a1869 expref: -99 pid: 79201 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.337931:0:79201:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e0471 lrc: 2/0,0 mode: PR/PR res: [0x200016b60:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a1869 expref: -99 pid: 79201 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826844.337955:0:79201:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200016b60:0x4:0x0](ffff98561158db88), inode mode 81a4 mask 40 00000080:00200008:1.0:1787826844.337996:0:79201:0:(file.c:479:ll_file_release()) START file f33b.replay-single--3:[0x200016b60:0x4:0x0](ffff98561158db88), flags 100002 00000002:00100000:1.0:1787826844.338059:0:79201:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff985610d3d680 x1874669225561600/t197568495623(197568495623) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826860 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:1.0:1787826844.338087:0:79201:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:1.0:1787826844.338093:0:79201:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610d3e580 x1874669225561856/t0(0) o35->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:1.0:1787826844.338118:0:79201:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610d3e580 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:79201:1874669225561856:192.168.203.114@tcp:35:createmany.0 00000100:00100000:1.0:1787826844.338212:0:79201:0:(client.c:2715:ptlrpc_set_wait()) set ffff98560957d300 going to sleep for 11 seconds 00000100:00100000:1.0:1787826844.342320:0:79201:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610d3e580 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:79201:1874669225561856:192.168.203.114@tcp:35:createmany.0 00000002:00100000:1.0:1787826844.342376:0:79201:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff985610d3d680 x1874669225561600/t197568495623(197568495623) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826860 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:1.0:1787826844.342468:0:79201:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826844.342488:0:79201:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826844.342571:0:79201:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e0478 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 79201 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.342582:0:79201:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e0478 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 79201 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.342592:0:79201:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e0478 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 79201 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826844.342612:0:79201:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610d3f0c0 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:79201:1874669225561984:192.168.203.114@tcp:101:createmany.0 00000100:00100000:1.0:1787826844.342689:0:79201:0:(client.c:2715:ptlrpc_set_wait()) set ffff98560957d300 going to sleep for 11 seconds 00000100:00100000:1.0:1787826844.344407:0:79201:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610d3f0c0 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:79201:1874669225561984:192.168.203.114@tcp:101:createmany.0 00010000:00010000:1.0:1787826844.344433:0:79201:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e0478 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a187e expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.344452:0:79201:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e0478 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a187e expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.344466:0:79201:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e0478 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a187e expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.344476:0:79201:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e0478 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a187e expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.344488:0:79201:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e0478 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a187e expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.344497:0:79201:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e0478 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a187e expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.344505:0:79201:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e0478 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a187e expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826844.344515:0:79201:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff985610d3f0c0 x1874669225561984/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826860 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826844.344623:0:79201:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e0478 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a187e expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.344638:0:79201:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:1.0:1787826844.344644:0:79201:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:1.0:1787826844.344673:0:79201:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:1.0:1787826844.344678:0:79201:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e0478 00000080:00010000:1.0:1787826844.344684:0:79201:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826844.344687:0:79201:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0478 from it ffffab3ac1cefbf8 00010000:00010000:1.0:1787826844.344832:0:79201:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e0478 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a187e expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.344843:0:79201:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e0478 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a187e expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826844.344921:0:79201:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787826844.344952:0:79201:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826844.344967:0:79201:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826844.344977:0:79201:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e0478 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a187e expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826844.344987:0:79201:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e0478 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a187e expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826844.344994:0:79201:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826844.344998:0:79201:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826844.345001:0:79201:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e0478 00000080:00010000:1.0:1787826844.345005:0:79201:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0478 from it ffffab3ac1cefb38 00010000:00010000:1.0:1787826844.345008:0:79201:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e0478 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a187e expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826844.345015:0:79201:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e0478 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a187e expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826844.345036:0:79201:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787826844.345042:0:79201:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f33b.replay-single--4, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), file ffff9856206c0900, open_flags 8042, mode 81a4 opened 0 00000080:00200000:1.0:1787826844.345053:0:79201:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f33b.replay-single--4, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=open|creat 00000002:00010000:1.0:1787826844.345083:0:79201:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f33b.replay-single--4,[0x200016b60:0x5:0x0]) in obj [0x200000007:0x1:0x0], intent: open|creat flags 0100103 00010000:00010000:1.0:1787826844.345095:0:79201:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e0478 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x5aa6d504d91a187e expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826844.345153:0:79201:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e0478 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a187e expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826844.345160:0:79201:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826844.345163:0:79201:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e0478 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a187e expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826844.345203:0:79201:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050800/0xb3f83dad1b7e047f lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 79201 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.345210:0:79201:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050800/0xb3f83dad1b7e047f lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 79201 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:1.0:1787826844.345221:0:79201:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:1.0:1787826844.345229:0:79201:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610d3f0c0 x1874669225562240/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:1.0:1787826844.345238:0:79201:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050800/0xb3f83dad1b7e047f lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 79201 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826844.345259:0:79201:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610d3f0c0 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:79201:1874669225562240:192.168.203.114@tcp:101:createmany.0 00000100:00100000:1.0:1787826844.345420:0:79201:0:(client.c:2715:ptlrpc_set_wait()) set ffff98560957d300 going to sleep for 11 seconds 00000100:00100000:1.0:1787826844.349576:0:79201:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610d3f0c0 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:79201:1874669225562240:192.168.203.114@tcp:101:createmany.0 00010000:00010000:1.0:1787826844.349612:0:79201:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050800/0xb3f83dad1b7e047f lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a189a expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.349647:0:79201:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050800/0xb3f83dad1b7e047f lrc: 4/0,1 mode: --/PR res: [0x200016b60:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a189a expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.349657:0:79201:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050800/0xb3f83dad1b7e047f lrc: 4/0,1 mode: --/PR res: [0x200016b60:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a189a expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.349670:0:79201:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050800/0xb3f83dad1b7e047f lrc: 4/0,1 mode: PR/PR res: [0x200016b60:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a189a expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.349683:0:79201:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050800/0xb3f83dad1b7e047f lrc: 4/0,1 mode: PR/PR res: [0x200016b60:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a189a expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.349740:0:79201:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050800/0xb3f83dad1b7e047f lrc: 5/1,1 mode: PR/PR res: [0x200016b60:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a189a expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.349750:0:79201:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050800/0xb3f83dad1b7e047f lrc: 5/1,1 mode: PR/PR res: [0x200016b60:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a189a expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.349756:0:79201:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050800/0xb3f83dad1b7e047f lrc: 4/1,0 mode: PR/PR res: [0x200016b60:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a189a expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826844.349765:0:79201:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff985610d3f0c0 x1874669225562240/t197568495625(197568495625) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/664 e 0 to 0 dl 1787826860 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:1.0:1787826844.349777:0:79201:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff985610d3f0c0 x1874669225562240/t197568495625(197568495625) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/664 e 0 to 0 dl 1787826860 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826844.349798:0:79201:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050800/0xb3f83dad1b7e047f lrc: 3/1,0 mode: PR/PR res: [0x200016b60:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a189a expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:1.0:1787826844.349821:0:79201:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050800/0xb3f83dad1b7e047f lrc: 3/1,0 mode: PR/PR res: [0x200016b60:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a189a expref: -99 pid: 79201 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.349833:0:79201:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8590027616/5 (0 0) 00000080:00200000:1.0:1787826844.349841:0:79201:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:1.0:1787826844.349858:0:79201:0:(namei.c:1628:ll_create_it()) VFS Op:name=f33b.replay-single--4, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=open|creat 00000080:00200000:1.0:1787826844.349906:0:79201:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200016b60:0x5:0x0](ffff985611589148) 00000080:00200000:1.0:1787826844.350040:0:79201:0:(file.c:6854:ll_layout_conf()) [0x200016b60:0x5:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:1.0:1787826844.350046:0:79201:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x200016b60:0x5:0x0](ffff985611589148) 00000080:00010000:1.0:1787826844.350049:0:79201:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200016b60:0x5:0x0](ffff985611589148) for lock 0xb3f83dad1b7e047f 00000080:00010000:1.0:1787826844.350061:0:79201:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200016b60:0x5:0x0](ffff985611589148) 00000080:00200008:1.0:1787826844.350087:0:79201:0:(file.c:1090:ll_file_open()) START file f33b.replay-single--4:[0x200016b60:0x5:0x0](ffff985611589148), flags 100102 00000002:00100000:1.0:1787826844.350114:0:79201:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff985610d3f0c0 x1874669225562240/t197568495625(197568495625) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826860 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:1.0:1787826844.350175:0:79201:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e047f from it ffff985610e3f480 00010000:00010000:1.0:1787826844.350181:0:79201:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050800/0xb3f83dad1b7e047f lrc: 3/1,0 mode: PR/PR res: [0x200016b60:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a189a expref: -99 pid: 79201 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.350191:0:79201:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050800/0xb3f83dad1b7e047f lrc: 2/0,0 mode: PR/PR res: [0x200016b60:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a189a expref: -99 pid: 79201 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826844.350220:0:79201:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200016b60:0x5:0x0](ffff985611589148), inode mode 81a4 mask 40 00000080:00200008:1.0:1787826844.350256:0:79201:0:(file.c:479:ll_file_release()) START file f33b.replay-single--4:[0x200016b60:0x5:0x0](ffff985611589148), flags 100002 00000002:00100000:1.0:1787826844.350320:0:79201:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff985610d3f0c0 x1874669225562240/t197568495625(197568495625) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826860 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:1.0:1787826844.350340:0:79201:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:1.0:1787826844.350351:0:79201:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610d3d2c0 x1874669225562496/t0(0) o35->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:1.0:1787826844.350377:0:79201:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610d3d2c0 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:79201:1874669225562496:192.168.203.114@tcp:35:createmany.0 00000100:00100000:1.0:1787826844.350469:0:79201:0:(client.c:2715:ptlrpc_set_wait()) set ffff98560957d240 going to sleep for 11 seconds 00000100:00100000:1.0:1787826844.352730:0:79201:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610d3d2c0 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:79201:1874669225562496:192.168.203.114@tcp:35:createmany.0 00000002:00100000:1.0:1787826844.352762:0:79201:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff985610d3f0c0 x1874669225562240/t197568495625(197568495625) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826860 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:1.0:1787826844.352840:0:79201:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826844.352858:0:79201:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826844.352913:0:79201:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e0486 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 79201 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.352924:0:79201:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e0486 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 79201 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.352937:0:79201:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e0486 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 79201 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826844.352956:0:79201:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610d3de00 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:79201:1874669225562624:192.168.203.114@tcp:101:createmany.0 00000100:00100000:1.0:1787826844.353044:0:79201:0:(client.c:2715:ptlrpc_set_wait()) set ffff98560957d240 going to sleep for 11 seconds 00000100:00100000:1.0:1787826844.355029:0:79201:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610d3de00 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:79201:1874669225562624:192.168.203.114@tcp:101:createmany.0 00010000:00010000:1.0:1787826844.355056:0:79201:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e0486 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a18af expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.355091:0:79201:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e0486 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a18af expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.355106:0:79201:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e0486 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a18af expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.355125:0:79201:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e0486 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a18af expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.355139:0:79201:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e0486 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a18af expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.355149:0:79201:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e0486 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a18af expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.355157:0:79201:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e0486 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a18af expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826844.355170:0:79201:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff985610d3de00 x1874669225562624/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826860 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826844.355184:0:79201:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e0486 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a18af expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.355204:0:79201:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:1.0:1787826844.355212:0:79201:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:1.0:1787826844.355249:0:79201:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:1.0:1787826844.355256:0:79201:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e0486 00000080:00010000:1.0:1787826844.355262:0:79201:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826844.355265:0:79201:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0486 from it ffffab3ac1cefbf8 00010000:00010000:1.0:1787826844.355276:0:79201:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e0486 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a18af expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.355287:0:79201:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e0486 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a18af expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826844.355314:0:79201:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787826844.355340:0:79201:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826844.355356:0:79201:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826844.355365:0:79201:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e0486 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a18af expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826844.355374:0:79201:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e0486 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a18af expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826844.355392:0:79201:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826844.355396:0:79201:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826844.355399:0:79201:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e0486 00000080:00010000:1.0:1787826844.355403:0:79201:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0486 from it ffffab3ac1cefb38 00010000:00010000:1.0:1787826844.355406:0:79201:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e0486 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a18af expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826844.355414:0:79201:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e0486 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a18af expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826844.355421:0:79201:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787826844.355427:0:79201:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f33b.replay-single--5, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), file ffff9856206c1980, open_flags 8042, mode 81a4 opened 0 00000080:00200000:1.0:1787826844.355435:0:79201:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f33b.replay-single--5, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=open|creat 00000002:00010000:1.0:1787826844.355456:0:79201:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f33b.replay-single--5,[0x200016b60:0x6:0x0]) in obj [0x200000007:0x1:0x0], intent: open|creat flags 0100103 00010000:00010000:1.0:1787826844.355474:0:79201:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e0486 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x5aa6d504d91a18af expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826844.355524:0:79201:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e0486 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a18af expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826844.355532:0:79201:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826844.355535:0:79201:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e0486 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a18af expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826844.355583:0:79201:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051000/0xb3f83dad1b7e048d lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 79201 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.355596:0:79201:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051000/0xb3f83dad1b7e048d lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 79201 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:1.0:1787826844.355608:0:79201:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:1.0:1787826844.355615:0:79201:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610d3de00 x1874669225562880/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:1.0:1787826844.355624:0:79201:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051000/0xb3f83dad1b7e048d lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 79201 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826844.355649:0:79201:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610d3de00 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:79201:1874669225562880:192.168.203.114@tcp:101:createmany.0 00000100:00100000:1.0:1787826844.355866:0:79201:0:(client.c:2715:ptlrpc_set_wait()) set ffff98560957d240 going to sleep for 11 seconds 00000100:00100000:1.0:1787826844.359808:0:79201:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610d3de00 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:79201:1874669225562880:192.168.203.114@tcp:101:createmany.0 00010000:00010000:1.0:1787826844.359837:0:79201:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051000/0xb3f83dad1b7e048d lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a18cb expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.359869:0:79201:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051000/0xb3f83dad1b7e048d lrc: 4/0,1 mode: --/PR res: [0x200016b60:0x6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a18cb expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.359877:0:79201:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051000/0xb3f83dad1b7e048d lrc: 4/0,1 mode: --/PR res: [0x200016b60:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a18cb expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.359888:0:79201:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051000/0xb3f83dad1b7e048d lrc: 4/0,1 mode: PR/PR res: [0x200016b60:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a18cb expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.359899:0:79201:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051000/0xb3f83dad1b7e048d lrc: 4/0,1 mode: PR/PR res: [0x200016b60:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a18cb expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.359912:0:79201:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051000/0xb3f83dad1b7e048d lrc: 5/1,1 mode: PR/PR res: [0x200016b60:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a18cb expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.359919:0:79201:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051000/0xb3f83dad1b7e048d lrc: 5/1,1 mode: PR/PR res: [0x200016b60:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a18cb expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.359927:0:79201:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051000/0xb3f83dad1b7e048d lrc: 4/1,0 mode: PR/PR res: [0x200016b60:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a18cb expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826844.359944:0:79201:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff985610d3de00 x1874669225562880/t197568495627(197568495627) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/664 e 0 to 0 dl 1787826860 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:1.0:1787826844.359957:0:79201:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff985610d3de00 x1874669225562880/t197568495627(197568495627) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/664 e 0 to 0 dl 1787826860 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826844.359969:0:79201:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051000/0xb3f83dad1b7e048d lrc: 3/1,0 mode: PR/PR res: [0x200016b60:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a18cb expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:1.0:1787826844.359982:0:79201:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051000/0xb3f83dad1b7e048d lrc: 3/1,0 mode: PR/PR res: [0x200016b60:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a18cb expref: -99 pid: 79201 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.359993:0:79201:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8590027616/6 (0 0) 00000080:00200000:1.0:1787826844.359999:0:79201:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:1.0:1787826844.360013:0:79201:0:(namei.c:1628:ll_create_it()) VFS Op:name=f33b.replay-single--5, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=open|creat 00000080:00200000:1.0:1787826844.360042:0:79201:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200016b60:0x6:0x0](ffff985611588908) 00000080:00200000:1.0:1787826844.360197:0:79201:0:(file.c:6854:ll_layout_conf()) [0x200016b60:0x6:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:1.0:1787826844.360202:0:79201:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x200016b60:0x6:0x0](ffff985611588908) 00000080:00010000:1.0:1787826844.360217:0:79201:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200016b60:0x6:0x0](ffff985611588908) for lock 0xb3f83dad1b7e048d 00000080:00010000:1.0:1787826844.360227:0:79201:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200016b60:0x6:0x0](ffff985611588908) 00000080:00200008:1.0:1787826844.360239:0:79201:0:(file.c:1090:ll_file_open()) START file f33b.replay-single--5:[0x200016b60:0x6:0x0](ffff985611588908), flags 100102 00000002:00100000:1.0:1787826844.360260:0:79201:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff985610d3de00 x1874669225562880/t197568495627(197568495627) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826860 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:1.0:1787826844.360296:0:79201:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e048d from it ffff985610e3f480 00010000:00010000:1.0:1787826844.360301:0:79201:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051000/0xb3f83dad1b7e048d lrc: 3/1,0 mode: PR/PR res: [0x200016b60:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a18cb expref: -99 pid: 79201 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.360316:0:79201:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051000/0xb3f83dad1b7e048d lrc: 2/0,0 mode: PR/PR res: [0x200016b60:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a18cb expref: -99 pid: 79201 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826844.360345:0:79201:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200016b60:0x6:0x0](ffff985611588908), inode mode 81a4 mask 40 00000080:00200008:1.0:1787826844.360372:0:79201:0:(file.c:479:ll_file_release()) START file f33b.replay-single--5:[0x200016b60:0x6:0x0](ffff985611588908), flags 100002 00000002:00100000:1.0:1787826844.360413:0:79201:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff985610d3de00 x1874669225562880/t197568495627(197568495627) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826860 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:1.0:1787826844.360424:0:79201:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:1.0:1787826844.360429:0:79201:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610d3fc00 x1874669225563136/t0(0) o35->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:1.0:1787826844.360453:0:79201:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610d3fc00 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:79201:1874669225563136:192.168.203.114@tcp:35:createmany.0 00000100:00100000:1.0:1787826844.360527:0:79201:0:(client.c:2715:ptlrpc_set_wait()) set ffff98560957d480 going to sleep for 11 seconds 00000100:00100000:1.0:1787826844.361888:0:79201:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610d3fc00 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:79201:1874669225563136:192.168.203.114@tcp:35:createmany.0 00000002:00100000:1.0:1787826844.361908:0:79201:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff985610d3de00 x1874669225562880/t197568495627(197568495627) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826860 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:1.0:1787826844.361960:0:79201:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826844.361974:0:79201:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826844.362014:0:79201:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e0494 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 79201 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.362022:0:79201:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e0494 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 79201 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.362032:0:79201:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e0494 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 79201 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826844.362045:0:79201:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610d3e940 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:79201:1874669225563264:192.168.203.114@tcp:101:createmany.0 00000100:00100000:1.0:1787826844.362427:0:79201:0:(client.c:2715:ptlrpc_set_wait()) set ffff98560957d480 going to sleep for 11 seconds 00000100:00100000:1.0:1787826844.364319:0:79201:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610d3e940 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:79201:1874669225563264:192.168.203.114@tcp:101:createmany.0 00010000:00010000:1.0:1787826844.364338:0:79201:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e0494 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a18e0 expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.364347:0:79201:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e0494 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a18e0 expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.364358:0:79201:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e0494 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a18e0 expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.364369:0:79201:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e0494 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a18e0 expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.364380:0:79201:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e0494 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a18e0 expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.364389:0:79201:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e0494 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a18e0 expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.364565:0:79201:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e0494 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a18e0 expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826844.364581:0:79201:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff985610d3e940 x1874669225563264/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826860 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826844.364602:0:79201:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e0494 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a18e0 expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.364611:0:79201:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:1.0:1787826844.364616:0:79201:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:1.0:1787826844.364638:0:79201:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:1.0:1787826844.364643:0:79201:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e0494 00000080:00010000:1.0:1787826844.364648:0:79201:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826844.364652:0:79201:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0494 from it ffffab3ac1cefbf8 00010000:00010000:1.0:1787826844.364655:0:79201:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e0494 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a18e0 expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.364663:0:79201:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e0494 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a18e0 expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826844.364685:0:79201:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787826844.364742:0:79201:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826844.364758:0:79201:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826844.364768:0:79201:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e0494 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a18e0 expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826844.364778:0:79201:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e0494 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a18e0 expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826844.364785:0:79201:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826844.364789:0:79201:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826844.364792:0:79201:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e0494 00000080:00010000:1.0:1787826844.364796:0:79201:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0494 from it ffffab3ac1cefb38 00010000:00010000:1.0:1787826844.364799:0:79201:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e0494 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a18e0 expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826844.364806:0:79201:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e0494 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a18e0 expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826844.364814:0:79201:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787826844.364819:0:79201:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f33b.replay-single--6, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), file ffff9856206c0180, open_flags 8042, mode 81a4 opened 0 00000080:00200000:1.0:1787826844.364828:0:79201:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f33b.replay-single--6, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=open|creat 00000002:00010000:1.0:1787826844.364847:0:79201:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f33b.replay-single--6,[0x200016b60:0x7:0x0]) in obj [0x200000007:0x1:0x0], intent: open|creat flags 0100103 00010000:00010000:1.0:1787826844.364860:0:79201:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e0494 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x5aa6d504d91a18e0 expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826844.364903:0:79201:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e0494 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a18e0 expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826844.365139:0:79201:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826844.365141:0:79201:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e0494 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a18e0 expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826844.365176:0:79201:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050400/0xb3f83dad1b7e049b lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 79201 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.365184:0:79201:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050400/0xb3f83dad1b7e049b lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 79201 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:1.0:1787826844.365193:0:79201:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:1.0:1787826844.365201:0:79201:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610d3e940 x1874669225563520/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:1.0:1787826844.365211:0:79201:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050400/0xb3f83dad1b7e049b lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 79201 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826844.365283:0:79201:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610d3e940 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:79201:1874669225563520:192.168.203.114@tcp:101:createmany.0 00000100:00100000:1.0:1787826844.365468:0:79201:0:(client.c:2715:ptlrpc_set_wait()) set ffff98560957d480 going to sleep for 11 seconds 00000100:00100000:1.0:1787826844.371018:0:79201:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610d3e940 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:79201:1874669225563520:192.168.203.114@tcp:101:createmany.0 00010000:00010000:1.0:1787826844.371039:0:79201:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050400/0xb3f83dad1b7e049b lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a18fc expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.371058:0:79201:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050400/0xb3f83dad1b7e049b lrc: 4/0,1 mode: --/PR res: [0x200016b60:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a18fc expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.371063:0:79201:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050400/0xb3f83dad1b7e049b lrc: 4/0,1 mode: --/PR res: [0x200016b60:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a18fc expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.371083:0:79201:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050400/0xb3f83dad1b7e049b lrc: 4/0,1 mode: PR/PR res: [0x200016b60:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a18fc expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.371091:0:79201:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050400/0xb3f83dad1b7e049b lrc: 4/0,1 mode: PR/PR res: [0x200016b60:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a18fc expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.371101:0:79201:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050400/0xb3f83dad1b7e049b lrc: 5/1,1 mode: PR/PR res: [0x200016b60:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a18fc expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.371106:0:79201:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050400/0xb3f83dad1b7e049b lrc: 5/1,1 mode: PR/PR res: [0x200016b60:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a18fc expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.371114:0:79201:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050400/0xb3f83dad1b7e049b lrc: 4/1,0 mode: PR/PR res: [0x200016b60:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a18fc expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826844.371123:0:79201:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff985610d3e940 x1874669225563520/t197568495629(197568495629) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/664 e 0 to 0 dl 1787826860 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:1.0:1787826844.371135:0:79201:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff985610d3e940 x1874669225563520/t197568495629(197568495629) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/664 e 0 to 0 dl 1787826860 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826844.371149:0:79201:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050400/0xb3f83dad1b7e049b lrc: 3/1,0 mode: PR/PR res: [0x200016b60:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a18fc expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:1.0:1787826844.371161:0:79201:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050400/0xb3f83dad1b7e049b lrc: 3/1,0 mode: PR/PR res: [0x200016b60:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a18fc expref: -99 pid: 79201 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.371173:0:79201:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8590027616/7 (0 0) 00000080:00200000:1.0:1787826844.371179:0:79201:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:1.0:1787826844.371190:0:79201:0:(namei.c:1628:ll_create_it()) VFS Op:name=f33b.replay-single--6, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=open|creat 00000080:00200000:1.0:1787826844.371214:0:79201:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200016b60:0x7:0x0](ffff98561158f448) 00000080:00200000:1.0:1787826844.371288:0:79201:0:(file.c:6854:ll_layout_conf()) [0x200016b60:0x7:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:1.0:1787826844.371293:0:79201:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x200016b60:0x7:0x0](ffff98561158f448) 00000080:00010000:1.0:1787826844.371297:0:79201:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200016b60:0x7:0x0](ffff98561158f448) for lock 0xb3f83dad1b7e049b 00000080:00010000:1.0:1787826844.371305:0:79201:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200016b60:0x7:0x0](ffff98561158f448) 00000080:00200008:1.0:1787826844.371317:0:79201:0:(file.c:1090:ll_file_open()) START file f33b.replay-single--6:[0x200016b60:0x7:0x0](ffff98561158f448), flags 100102 00000002:00100000:1.0:1787826844.371338:0:79201:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff985610d3e940 x1874669225563520/t197568495629(197568495629) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826860 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:1.0:1787826844.371376:0:79201:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e049b from it ffff985610e3f480 00010000:00010000:1.0:1787826844.371380:0:79201:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050400/0xb3f83dad1b7e049b lrc: 3/1,0 mode: PR/PR res: [0x200016b60:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a18fc expref: -99 pid: 79201 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.371386:0:79201:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050400/0xb3f83dad1b7e049b lrc: 2/0,0 mode: PR/PR res: [0x200016b60:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a18fc expref: -99 pid: 79201 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826844.371405:0:79201:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200016b60:0x7:0x0](ffff98561158f448), inode mode 81a4 mask 40 00000080:00200008:1.0:1787826844.371431:0:79201:0:(file.c:479:ll_file_release()) START file f33b.replay-single--6:[0x200016b60:0x7:0x0](ffff98561158f448), flags 100002 00000002:00100000:1.0:1787826844.371472:0:79201:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff985610d3e940 x1874669225563520/t197568495629(197568495629) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826860 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:1.0:1787826844.371482:0:79201:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:1.0:1787826844.371487:0:79201:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610d3ed00 x1874669225563776/t0(0) o35->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:1.0:1787826844.371505:0:79201:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610d3ed00 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:79201:1874669225563776:192.168.203.114@tcp:35:createmany.0 00000100:00100000:1.0:1787826844.371571:0:79201:0:(client.c:2715:ptlrpc_set_wait()) set ffff98560957dc00 going to sleep for 11 seconds 00000100:00100000:1.0:1787826844.373126:0:79201:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610d3ed00 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:79201:1874669225563776:192.168.203.114@tcp:35:createmany.0 00000002:00100000:1.0:1787826844.373159:0:79201:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff985610d3e940 x1874669225563520/t197568495629(197568495629) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826860 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:1.0:1787826844.373219:0:79201:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826844.373234:0:79201:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826844.373273:0:79201:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e04a2 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 79201 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.373286:0:79201:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e04a2 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 79201 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.373296:0:79201:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e04a2 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 79201 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826844.373311:0:79201:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610d3f480 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:79201:1874669225563904:192.168.203.114@tcp:101:createmany.0 00000100:00100000:1.0:1787826844.373362:0:79201:0:(client.c:2715:ptlrpc_set_wait()) set ffff98560957dc00 going to sleep for 11 seconds 00000100:00100000:1.0:1787826844.375600:0:79201:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610d3f480 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:79201:1874669225563904:192.168.203.114@tcp:101:createmany.0 00010000:00010000:1.0:1787826844.375623:0:79201:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e04a2 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1911 expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.375635:0:79201:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e04a2 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1911 expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.375646:0:79201:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e04a2 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1911 expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.375656:0:79201:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e04a2 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1911 expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.375667:0:79201:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e04a2 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1911 expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.375675:0:79201:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e04a2 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1911 expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.375686:0:79201:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e04a2 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1911 expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826844.375727:0:79201:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff985610d3f480 x1874669225563904/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826860 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826844.375741:0:79201:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e04a2 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1911 expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.375752:0:79201:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:1.0:1787826844.375758:0:79201:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:1.0:1787826844.375782:0:79201:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:1.0:1787826844.375788:0:79201:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e04a2 00000080:00010000:1.0:1787826844.375792:0:79201:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826844.375795:0:79201:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e04a2 from it ffffab3ac1cefbf8 00010000:00010000:1.0:1787826844.375798:0:79201:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e04a2 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1911 expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.375805:0:79201:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e04a2 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1911 expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826844.375826:0:79201:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787826844.375851:0:79201:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826844.375865:0:79201:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826844.375874:0:79201:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e04a2 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1911 expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826844.375889:0:79201:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e04a2 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1911 expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826844.375896:0:79201:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826844.375900:0:79201:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826844.375903:0:79201:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e04a2 00000080:00010000:1.0:1787826844.375907:0:79201:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e04a2 from it ffffab3ac1cefb38 00010000:00010000:1.0:1787826844.375909:0:79201:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e04a2 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1911 expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826844.375917:0:79201:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e04a2 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1911 expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826844.375925:0:79201:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787826844.375931:0:79201:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f33b.replay-single--7, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), file ffff9856206c1980, open_flags 8042, mode 81a4 opened 0 00000080:00200000:1.0:1787826844.375939:0:79201:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f33b.replay-single--7, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=open|creat 00000002:00010000:1.0:1787826844.375957:0:79201:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f33b.replay-single--7,[0x200016b60:0x8:0x0]) in obj [0x200000007:0x1:0x0], intent: open|creat flags 0100103 00010000:00010000:1.0:1787826844.375967:0:79201:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e04a2 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x5aa6d504d91a1911 expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826844.376006:0:79201:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e04a2 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a1911 expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826844.376013:0:79201:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826844.376016:0:79201:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e04a2 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a1911 expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826844.376051:0:79201:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e04a9 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 79201 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.376058:0:79201:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e04a9 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 79201 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:1.0:1787826844.376078:0:79201:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:1.0:1787826844.376086:0:79201:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610d3f480 x1874669225564160/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:1.0:1787826844.376094:0:79201:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e04a9 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 79201 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826844.376111:0:79201:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610d3f480 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:79201:1874669225564160:192.168.203.114@tcp:101:createmany.0 00000100:00100000:1.0:1787826844.376227:0:79201:0:(client.c:2715:ptlrpc_set_wait()) set ffff98560957dc00 going to sleep for 11 seconds 00000100:00100000:1.0:1787826844.378921:0:79201:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610d3f480 pname:cluuid:pid:xid:nid:opc:job createmany:7b4e8535-9db7-485d-abf6-bda966b63320:79201:1874669225564160:192.168.203.114@tcp:101:createmany.0 00010000:00010000:1.0:1787826844.378944:0:79201:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e04a9 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a192d expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.378976:0:79201:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e04a9 lrc: 4/0,1 mode: --/PR res: [0x200016b60:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a192d expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.378987:0:79201:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e04a9 lrc: 4/0,1 mode: --/PR res: [0x200016b60:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a192d expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.378996:0:79201:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e04a9 lrc: 4/0,1 mode: PR/PR res: [0x200016b60:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a192d expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.379010:0:79201:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e04a9 lrc: 4/0,1 mode: PR/PR res: [0x200016b60:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a192d expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.379021:0:79201:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e04a9 lrc: 5/1,1 mode: PR/PR res: [0x200016b60:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a192d expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.379029:0:79201:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e04a9 lrc: 5/1,1 mode: PR/PR res: [0x200016b60:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a192d expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.379035:0:79201:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e04a9 lrc: 4/1,0 mode: PR/PR res: [0x200016b60:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a192d expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826844.379045:0:79201:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff985610d3f480 x1874669225564160/t197568495631(197568495631) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/664 e 0 to 0 dl 1787826860 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:1.0:1787826844.379056:0:79201:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff985610d3f480 x1874669225564160/t197568495631(197568495631) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/664 e 0 to 0 dl 1787826860 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826844.379078:0:79201:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e04a9 lrc: 3/1,0 mode: PR/PR res: [0x200016b60:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a192d expref: -99 pid: 79201 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000080:00200000:1.0:1787826844.399792:0:78486:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826844.399813:0:78486:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826844.399862:0:78486:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e04cc lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 78486 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.399873:0:78486:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e04cc lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78486 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.399886:0:78486:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e04cc lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78486 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826844.399907:0:78486:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610d3da40 pname:cluuid:pid:xid:nid:opc:job bash:7b4e8535-9db7-485d-abf6-bda966b63320:78486:1874669225565824:192.168.203.114@tcp:101:bash.0 00000100:00100000:1.0:1787826844.400148:0:78486:0:(client.c:2715:ptlrpc_set_wait()) set ffff98560957dc00 going to sleep for 11 seconds 00000100:00100000:1.0:1787826844.403678:0:78486:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610d3da40 pname:cluuid:pid:xid:nid:opc:job bash:7b4e8535-9db7-485d-abf6-bda966b63320:78486:1874669225565824:192.168.203.114@tcp:101:bash.0 00010000:00010000:1.0:1787826844.403714:0:78486:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e04cc lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a19a4 expref: -99 pid: 78486 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.403726:0:78486:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e04cc lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a19a4 expref: -99 pid: 78486 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.403738:0:78486:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e04cc lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a19a4 expref: -99 pid: 78486 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.403748:0:78486:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e04cc lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a19a4 expref: -99 pid: 78486 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.403760:0:78486:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e04cc lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a19a4 expref: -99 pid: 78486 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.403768:0:78486:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e04cc lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a19a4 expref: -99 pid: 78486 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.403776:0:78486:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e04cc lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a19a4 expref: -99 pid: 78486 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826844.403791:0:78486:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff985610d3da40 x1874669225565824/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826860 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'bash.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826844.403805:0:78486:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e04cc lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a19a4 expref: -99 pid: 78486 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.403816:0:78486:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:1.0:1787826844.403822:0:78486:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:1.0:1787826844.403849:0:78486:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:1.0:1787826844.403854:0:78486:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e04cc 00000080:00010000:1.0:1787826844.403859:0:78486:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826844.403862:0:78486:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e04cc from it ffffab3ac1897bb0 00010000:00010000:1.0:1787826844.403865:0:78486:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e04cc lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a19a4 expref: -99 pid: 78486 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.403874:0:78486:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e04cc lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a19a4 expref: -99 pid: 78486 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826844.403894:0:78486:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 44 00000080:00200008:1.0:1787826844.403905:0:78486:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 304000 00000080:00200008:1.0:1787826844.403925:0:78486:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:1.0:1787826844.403931:0:78486:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826844.403945:0:78486:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826844.403954:0:78486:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e04cc lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a19a4 expref: -99 pid: 78486 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826844.403963:0:78486:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e04cc lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a19a4 expref: -99 pid: 78486 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826844.403970:0:78486:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826844.403973:0:78486:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826844.403976:0:78486:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e04cc 00000080:00010000:1.0:1787826844.403979:0:78486:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e04cc from it ffffab3ac1897cd8 00010000:00010000:1.0:1787826844.403982:0:78486:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e04cc lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a19a4 expref: -99 pid: 78486 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826844.403989:0:78486:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e04cc lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a19a4 expref: -99 pid: 78486 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826844.404331:0:78486:0:(dir.c:332:ll_iterate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) pos/size0/20480 32bit_api 0 00000002:00010000:1.0:1787826844.404346:0:78486:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: readdir flags 0 00010000:00010000:1.0:1787826844.404353:0:78486:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e04cc lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a19a4 expref: -99 pid: 78486 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826844.404361:0:78486:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e04cc lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a19a4 expref: -99 pid: 78486 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000100:00100000:1.0:1787826844.404956:0:78486:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610d3da40 pname:cluuid:pid:xid:nid:opc:job bash:7b4e8535-9db7-485d-abf6-bda966b63320:78486:1874669225565952:192.168.203.114@tcp:37:bash.0 00000100:00100000:1.0:1787826844.405037:0:78486:0:(client.c:2715:ptlrpc_set_wait()) set ffff98560957dc00 going to sleep for 11 seconds 00000100:00100000:1.0:1787826844.411845:0:78486:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610d3da40 pname:cluuid:pid:xid:nid:opc:job bash:7b4e8535-9db7-485d-abf6-bda966b63320:78486:1874669225565952:192.168.203.114@tcp:37:bash.0 00010000:00010000:1.0:1787826844.412486:0:78486:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e04cc lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a19a4 expref: -99 pid: 78486 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826844.412500:0:78486:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e04cc lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a19a4 expref: -99 pid: 78486 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787826844.412630:0:78486:0:(dir.c:332:ll_iterate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) pos/size18446744073709551614/20480 32bit_api 0 00000080:00200008:1.0:1787826844.412646:0:78486:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 304000 00000080:00200008:1.0:1787826844.416650:0:79202:0:(file.c:6313:ll_getattr_dentry()) START file f33b.replay-single--0:[0x200016b60:0x1:0x0](ffff98561158aa08), request_mask 2047, flags 0, foreign 0 00000080:00200000:1.0:1787826844.416657:0:79202:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200016b60:0x1:0x0](ffff98561158aa08),name=f33b.replay-single--0 00000002:00010000:1.0:1787826844.416673:0:79202:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200016b60:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826844.416682:0:79202:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e0447 lrc: 2/1,0 mode: PR/PR res: [0x200016b60:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a17d6 expref: -99 pid: 79201 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826844.416754:0:79202:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e0447 lrc: 2/1,0 mode: PR/PR res: [0x200016b60:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a17d6 expref: -99 pid: 79201 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826844.416763:0:79202:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826844.416767:0:79202:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200016b60:0x1:0x0](ffff98561158aa08) 00000080:00010000:1.0:1787826844.416770:0:79202:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200016b60:0x1:0x0](ffff98561158aa08) for lock 0xb3f83dad1b7e0447 00000080:00010000:1.0:1787826844.416775:0:79202:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0447 from it ffffab3ac0dd3cb8 00010000:00010000:1.0:1787826844.416778:0:79202:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e0447 lrc: 3/1,0 mode: PR/PR res: [0x200016b60:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a17d6 expref: -99 pid: 79201 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826844.416786:0:79202:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e0447 lrc: 2/0,0 mode: PR/PR res: [0x200016b60:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a17d6 expref: -99 pid: 79201 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826844.416809:0:79202:0:(vvp_io.c:1885:vvp_io_init()) [0x200016b60:0x1:0x0] ignore/verify layout 0/1, layout version 0 restore needed 0 00000001:00010000:1.0:1787826844.416820:0:79202:0:(glimpse.c:73:cl_glimpse_lock()) Glimpsing inode [0x200016b60:0x1:0x0] 00000008:00010000:1.0:1787826844.416839:0:79202:0:(lustre_osc.h:166:osc_enq2ldlm_flags()) flags: a 00000008:00010000:1.0:1787826844.416842:0:79202:0:(osc_lock.c:1320:osc_lock_init()) ### lock ffff98560560cbb0, osc lock ffff985602f8f110, flags 0x400001002 00010000:00010000:1.0:1787826844.416891:0:79202:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0000-osc-ffff985602bc9800 lock: 000000006563a0df/0xb3f83dad1b7e04d3 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x602:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 79202 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.416903:0:79202:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x400001000 ns: lustre-OST0000-osc-ffff985602bc9800 lock: 000000006563a0df/0xb3f83dad1b7e04d3 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x602:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 79202 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000020:00200000:1.0:1787826844.416936:0:79202:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000100:00100000:1.0:1787826844.416949:0:2384:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1787826844.416968:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610d3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669225566080:192.168.203.114@tcp:101:rm.0 00000100:00100000:3.0:1787826844.417294:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669225566208:192.168.203.114@tcp:101:ll_agl.0 00000100:00100000:3.0:1787826844.417398:0:2386:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1787826844.417411:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee2d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669225566336:192.168.203.114@tcp:101:ll_agl.0 00000100:00100000:3.0:1787826844.417516:0:2385:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1787826844.417528:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669225566464:192.168.203.114@tcp:101:ll_agl.0 00000100:00100000:3.0:1787826844.417620:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669225566592:192.168.203.114@tcp:101:ll_agl.0 00000100:00100000:3.0:1787826844.417657:0:2385:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1787826844.417666:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669225566720:192.168.203.114@tcp:101:ll_agl.0 00000100:00100000:3.0:1787826844.418204:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669225566848:192.168.203.114@tcp:101:ll_agl.0 00000100:00100000:1.0:1787826844.418325:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e78f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669225566976:192.168.203.114@tcp:101:ll_sa.0 00000100:00100000:3.0:1787826844.418333:0:2385:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1787826844.418354:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669225567104:192.168.203.114@tcp:101:ll_agl.0 00010000:00010000:3.0:1787826844.419386:0:2386:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0000-osc-ffff985602bc9800 lock: 00000000a32e421a/0xb3f83dad1b7e04e1 lrc: 6/2,0 mode: --/PR res: [0x280000401:0x603:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.419399:0:2386:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-OST0000-osc-ffff985602bc9800 lock: 00000000a32e421a/0xb3f83dad1b7e04e1 lrc: 6/2,0 mode: --/PR res: [0x280000401:0x603:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a19b2 expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.419411:0:2386:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0000-osc-ffff985602bc9800 lock: 00000000a32e421a/0xb3f83dad1b7e04e1 lrc: 6/2,0 mode: PR/PR res: [0x280000401:0x603:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a19b2 expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.419416:0:2384:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0000-osc-ffff985602bc9800 lock: 000000006563a0df/0xb3f83dad1b7e04d3 lrc: 6/2,0 mode: --/PR res: [0x280000401:0x602:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 79202 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.419422:0:2386:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-OST0000-osc-ffff985602bc9800 lock: 00000000a32e421a/0xb3f83dad1b7e04e1 lrc: 6/2,0 mode: PR/PR res: [0x280000401:0x603:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a19b2 expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.419446:0:2384:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-OST0000-osc-ffff985602bc9800 lock: 000000006563a0df/0xb3f83dad1b7e04d3 lrc: 6/2,0 mode: --/PR res: [0x280000401:0x602:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a19b9 expref: -99 pid: 79202 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:3.0:1787826844.419454:0:2386:0:(osc_lock.c:192:osc_lock_lvb_update()) ### acquired size 0, setting rss=0; kms=0, end=18446744073709551615 ns: lustre-OST0000-osc-ffff985602bc9800 lock: 00000000a32e421a/0xb3f83dad1b7e04e1 lrc: 5/2,0 mode: PR/PR res: [0x280000401:0x603:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a19b2 expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.419468:0:2384:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0000-osc-ffff985602bc9800 lock: 000000006563a0df/0xb3f83dad1b7e04d3 lrc: 6/2,0 mode: PR/PR res: [0x280000401:0x602:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a19b9 expref: -99 pid: 79202 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.419487:0:2384:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-OST0000-osc-ffff985602bc9800 lock: 000000006563a0df/0xb3f83dad1b7e04d3 lrc: 6/2,0 mode: PR/PR res: [0x280000401:0x602:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a19b9 expref: -99 pid: 79202 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787826844.419508:0:2384:0:(osc_lock.c:192:osc_lock_lvb_update()) ### acquired size 0, setting rss=0; kms=0, end=18446744073709551615 ns: lustre-OST0000-osc-ffff985602bc9800 lock: 000000006563a0df/0xb3f83dad1b7e04d3 lrc: 5/2,0 mode: PR/PR res: [0x280000401:0x602:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a19b9 expref: -99 pid: 79202 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000020:00200000:1.0:1787826844.419539:0:2384:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00010000:00010000:3.0:1787826844.419544:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff985602bc9800 lock: 00000000a32e421a/0xb3f83dad1b7e04e1 lrc: 5/2,0 mode: PR/PR res: [0x280000401:0x603:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a19b2 expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.419567:0:2386:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-OST0000-osc-ffff985602bc9800 lock: 00000000a32e421a/0xb3f83dad1b7e04e1 lrc: 4/1,0 mode: PR/PR res: [0x280000401:0x603:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a19b2 expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.419579:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff985602bc9800 lock: 00000000a32e421a/0xb3f83dad1b7e04e1 lrc: 4/1,0 mode: PR/PR res: [0x280000401:0x603:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a19b2 expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.419594:0:2386:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0000-osc-ffff985602bc9800 lock: 00000000a32e421a/0xb3f83dad1b7e04e1 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x603:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a19b2 expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000100:00100000:3.0:1787826844.419627:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee2d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669225566336:192.168.203.114@tcp:101:ll_agl.0 00010000:00010000:3.0:1787826844.419652:0:2386:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0001-osc-ffff985602bc9800 lock: 0000000024db9e92/0xb3f83dad1b7e04da lrc: 6/2,0 mode: --/PR res: [0x2c0000401:0x602:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.419665:0:2386:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-OST0001-osc-ffff985602bc9800 lock: 0000000024db9e92/0xb3f83dad1b7e04da lrc: 6/2,0 mode: --/PR res: [0x2c0000401:0x602:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a19ab expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.419672:0:2386:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0001-osc-ffff985602bc9800 lock: 0000000024db9e92/0xb3f83dad1b7e04da lrc: 6/2,0 mode: PR/PR res: [0x2c0000401:0x602:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a19ab expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.419679:0:2386:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-OST0001-osc-ffff985602bc9800 lock: 0000000024db9e92/0xb3f83dad1b7e04da lrc: 6/2,0 mode: PR/PR res: [0x2c0000401:0x602:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a19ab expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:3.0:1787826844.419686:0:2386:0:(osc_lock.c:192:osc_lock_lvb_update()) ### acquired size 0, setting rss=0; kms=0, end=18446744073709551615 ns: lustre-OST0001-osc-ffff985602bc9800 lock: 0000000024db9e92/0xb3f83dad1b7e04da lrc: 5/2,0 mode: PR/PR res: [0x2c0000401:0x602:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a19ab expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.419712:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0001-osc-ffff985602bc9800 lock: 0000000024db9e92/0xb3f83dad1b7e04da lrc: 5/2,0 mode: PR/PR res: [0x2c0000401:0x602:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a19ab expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.419717:0:2386:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-OST0001-osc-ffff985602bc9800 lock: 0000000024db9e92/0xb3f83dad1b7e04da lrc: 4/1,0 mode: PR/PR res: [0x2c0000401:0x602:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a19ab expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.419722:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0001-osc-ffff985602bc9800 lock: 0000000024db9e92/0xb3f83dad1b7e04da lrc: 4/1,0 mode: PR/PR res: [0x2c0000401:0x602:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a19ab expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.419729:0:2386:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0001-osc-ffff985602bc9800 lock: 0000000024db9e92/0xb3f83dad1b7e04da lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x602:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a19ab expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000100:00100000:3.0:1787826844.419741:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669225566208:192.168.203.114@tcp:101:ll_agl.0 00010000:00010000:3.0:1787826844.419852:0:2385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0001-osc-ffff985602bc9800 lock: 00000000b69c5f27/0xb3f83dad1b7e04f6 lrc: 6/2,0 mode: --/PR res: [0x2c0000401:0x604:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.419863:0:2385:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-OST0001-osc-ffff985602bc9800 lock: 00000000b69c5f27/0xb3f83dad1b7e04f6 lrc: 6/2,0 mode: --/PR res: [0x2c0000401:0x604:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a19ce expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.419871:0:2385:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0001-osc-ffff985602bc9800 lock: 00000000b69c5f27/0xb3f83dad1b7e04f6 lrc: 6/2,0 mode: PR/PR res: [0x2c0000401:0x604:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a19ce expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.419879:0:2385:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-OST0001-osc-ffff985602bc9800 lock: 00000000b69c5f27/0xb3f83dad1b7e04f6 lrc: 6/2,0 mode: PR/PR res: [0x2c0000401:0x604:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a19ce expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:3.0:1787826844.419888:0:2385:0:(osc_lock.c:192:osc_lock_lvb_update()) ### acquired size 0, setting rss=0; kms=0, end=18446744073709551615 ns: lustre-OST0001-osc-ffff985602bc9800 lock: 00000000b69c5f27/0xb3f83dad1b7e04f6 lrc: 5/2,0 mode: PR/PR res: [0x2c0000401:0x604:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a19ce expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.419898:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0001-osc-ffff985602bc9800 lock: 00000000b69c5f27/0xb3f83dad1b7e04f6 lrc: 5/2,0 mode: PR/PR res: [0x2c0000401:0x604:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a19ce expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.419905:0:2385:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-OST0001-osc-ffff985602bc9800 lock: 00000000b69c5f27/0xb3f83dad1b7e04f6 lrc: 4/1,0 mode: PR/PR res: [0x2c0000401:0x604:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a19ce expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.419912:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0001-osc-ffff985602bc9800 lock: 00000000b69c5f27/0xb3f83dad1b7e04f6 lrc: 4/1,0 mode: PR/PR res: [0x2c0000401:0x604:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a19ce expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.419919:0:2385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0001-osc-ffff985602bc9800 lock: 00000000b69c5f27/0xb3f83dad1b7e04f6 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x604:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a19ce expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826844.419923:0:2384:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff985602bc9800 lock: 000000006563a0df/0xb3f83dad1b7e04d3 lrc: 6/2,0 mode: PR/PR res: [0x280000401:0x602:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x5aa6d504d91a19b9 expref: -99 pid: 79202 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826844.419932:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669225566720:192.168.203.114@tcp:101:ll_agl.0 00010000:00010000:1.0:1787826844.419944:0:2384:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-OST0000-osc-ffff985602bc9800 lock: 000000006563a0df/0xb3f83dad1b7e04d3 lrc: 5/1,0 mode: PR/PR res: [0x280000401:0x602:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x5aa6d504d91a19b9 expref: -99 pid: 79202 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.419960:0:2384:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff985602bc9800 lock: 000000006563a0df/0xb3f83dad1b7e04d3 lrc: 4/1,0 mode: PR/PR res: [0x280000401:0x602:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x5aa6d504d91a19b9 expref: -99 pid: 79202 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.419970:0:2384:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0000-osc-ffff985602bc9800 lock: 000000006563a0df/0xb3f83dad1b7e04d3 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x602:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x5aa6d504d91a19b9 expref: -99 pid: 79202 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000100:00100000:1.0:1787826844.419994:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610d3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669225566080:192.168.203.114@tcp:101:rm.0 00010000:00010000:1.0:1787826844.420020:0:2384:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0001-osc-ffff985602bc9800 lock: 000000008d58046f/0xb3f83dad1b7e0504 lrc: 6/2,0 mode: --/PR res: [0x2c0000401:0x605:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 79203 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.420031:0:2384:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-OST0001-osc-ffff985602bc9800 lock: 000000008d58046f/0xb3f83dad1b7e0504 lrc: 6/2,0 mode: --/PR res: [0x2c0000401:0x605:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a19d5 expref: -99 pid: 79203 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.420039:0:2384:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0001-osc-ffff985602bc9800 lock: 000000008d58046f/0xb3f83dad1b7e0504 lrc: 6/2,0 mode: PR/PR res: [0x2c0000401:0x605:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a19d5 expref: -99 pid: 79203 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.420046:0:2384:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-OST0001-osc-ffff985602bc9800 lock: 000000008d58046f/0xb3f83dad1b7e0504 lrc: 6/2,0 mode: PR/PR res: [0x2c0000401:0x605:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a19d5 expref: -99 pid: 79203 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787826844.420107:0:2384:0:(osc_lock.c:192:osc_lock_lvb_update()) ### acquired size 0, setting rss=0; kms=0, end=18446744073709551615 ns: lustre-OST0001-osc-ffff985602bc9800 lock: 000000008d58046f/0xb3f83dad1b7e0504 lrc: 5/2,0 mode: PR/PR res: [0x2c0000401:0x605:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a19d5 expref: -99 pid: 79203 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.420125:0:2384:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0001-osc-ffff985602bc9800 lock: 000000008d58046f/0xb3f83dad1b7e0504 lrc: 5/2,0 mode: PR/PR res: [0x2c0000401:0x605:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a19d5 expref: -99 pid: 79203 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.420133:0:2384:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-OST0001-osc-ffff985602bc9800 lock: 000000008d58046f/0xb3f83dad1b7e0504 lrc: 4/1,0 mode: PR/PR res: [0x2c0000401:0x605:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a19d5 expref: -99 pid: 79203 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.420139:0:2384:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0001-osc-ffff985602bc9800 lock: 000000008d58046f/0xb3f83dad1b7e0504 lrc: 4/1,0 mode: PR/PR res: [0x2c0000401:0x605:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a19d5 expref: -99 pid: 79203 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.420145:0:2384:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0001-osc-ffff985602bc9800 lock: 000000008d58046f/0xb3f83dad1b7e0504 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x605:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a19d5 expref: -99 pid: 79203 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000100:00100000:1.0:1787826844.420166:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e78f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669225566976:192.168.203.114@tcp:101:ll_sa.0 00000080:00200000:3.0:1787826844.425463:0:79204:0:(vvp_io.c:1885:vvp_io_init()) [0x200016b60:0xa:0x0] ignore/verify layout 0/1, layout version 0 restore needed 0 00000001:00010000:3.0:1787826844.425489:0:79204:0:(glimpse.c:73:cl_glimpse_lock()) Glimpsing inode [0x200016b60:0xa:0x0] 00000008:00010000:3.0:1787826844.425516:0:79204:0:(lustre_osc.h:166:osc_enq2ldlm_flags()) flags: 2b 00000008:00010000:3.0:1787826844.425527:0:79204:0:(osc_lock.c:1320:osc_lock_init()) ### lock ffff985610fb97b0, osc lock ffff98560897f5f0, flags 0x400041012 00010000:00010000:3.0:1787826844.425598:0:79204:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0001-osc-ffff985602bc9800 lock: 000000009edc46ab/0xb3f83dad1b7e053c lrc: 3/1,0 mode: --/PR res: [0x2c0000401:0x606:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.425652:0:79204:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x400041010 ns: lustre-OST0001-osc-ffff985602bc9800 lock: 000000009edc46ab/0xb3f83dad1b7e053c lrc: 3/1,0 mode: --/PR res: [0x2c0000401:0x606:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000020:00010000:3.0:1787826844.425771:0:79204:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff98560549a290 (ffff985610fe12d8/0) at cl_lock_release():245 00000020:00010000:3.0:1787826844.425775:0:79204:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff98560549a290 (ffff985610fe12d8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1787826844.425777:0:79204:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff985610fb97b0 (ffff9856131a1758/1) at cl_lock_cancel():142 00000020:00010000:3.0:1787826844.425781:0:79204:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff98560549a290 (ffff985610fe12d8/0) at cl_lock_fini():72 00000020:00010000:3.0:1787826844.425792:0:79204:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff985610fb97b0 (ffff985610fe12d8/1) at cl_lock_fini():72 00000080:00200000:3.0:1787826844.425817:0:79204:0:(vvp_io.c:304:vvp_io_fini()) [0x200016b60:0xa:0x0] ignore/verify layout 0/1, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000100:00100000:3.0:1787826844.425857:0:2386:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1787826844.425883:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e4da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669225567616:192.168.203.114@tcp:101:ll_agl.0 00010000:00010000:3.0:1787826844.428822:0:2386:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0001-osc-ffff985602bc9800 lock: 000000009edc46ab/0xb3f83dad1b7e053c lrc: 6/2,0 mode: --/PR res: [0x2c0000401:0x606:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.428896:0:2386:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-OST0001-osc-ffff985602bc9800 lock: 000000009edc46ab/0xb3f83dad1b7e053c lrc: 6/2,0 mode: --/PR res: [0x2c0000401:0x606:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1a14 expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.428925:0:2386:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0001-osc-ffff985602bc9800 lock: 000000009edc46ab/0xb3f83dad1b7e053c lrc: 6/2,0 mode: PR/PR res: [0x2c0000401:0x606:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1a14 expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.428946:0:2386:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-OST0001-osc-ffff985602bc9800 lock: 000000009edc46ab/0xb3f83dad1b7e053c lrc: 6/2,0 mode: PR/PR res: [0x2c0000401:0x606:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1a14 expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:3.0:1787826844.429176:0:2386:0:(osc_lock.c:192:osc_lock_lvb_update()) ### acquired size 0, setting rss=0; kms=0, end=18446744073709551615 ns: lustre-OST0001-osc-ffff985602bc9800 lock: 000000009edc46ab/0xb3f83dad1b7e053c lrc: 5/2,0 mode: PR/PR res: [0x2c0000401:0x606:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a1a14 expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.429212:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0001-osc-ffff985602bc9800 lock: 000000009edc46ab/0xb3f83dad1b7e053c lrc: 5/2,0 mode: PR/PR res: [0x2c0000401:0x606:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a1a14 expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.429230:0:2386:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-OST0001-osc-ffff985602bc9800 lock: 000000009edc46ab/0xb3f83dad1b7e053c lrc: 4/1,0 mode: PR/PR res: [0x2c0000401:0x606:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a1a14 expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.429253:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0001-osc-ffff985602bc9800 lock: 000000009edc46ab/0xb3f83dad1b7e053c lrc: 4/1,0 mode: PR/PR res: [0x2c0000401:0x606:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a1a14 expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.429268:0:2386:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0001-osc-ffff985602bc9800 lock: 000000009edc46ab/0xb3f83dad1b7e053c lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x606:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a1a14 expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000100:00100000:3.0:1787826844.429296:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e4da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669225567616:192.168.203.114@tcp:101:ll_agl.0 00000100:00100000:3.0:1787826844.434325:0:2386:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1787826844.434345:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669225568384:192.168.203.114@tcp:63:ll_sa.0 00010000:00010000:3.0:1787826844.438335:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ed200/0xb3f83dad1b7e055f lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 19 type: IBT flags: 0x0 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.438362:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ed200/0xb3f83dad1b7e055f lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 19 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.438369:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ed200/0xb3f83dad1b7e055f lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 19 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.438375:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ed200/0xb3f83dad1b7e055f lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 19 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.438383:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ed200/0xb3f83dad1b7e055f lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 19 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.438388:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9856129ed200 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826844.438392:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ed200/0xb3f83dad1b7e055f lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 19 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.438399:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ed200/0xb3f83dad1b7e055f lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 19 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.438406:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ed200/0xb3f83dad1b7e055f lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 19 type: IBT flags: 0x20e400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.438427:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826844.438433:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ed200/0xb3f83dad1b7e055f lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 19 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.438439:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ed200/0xb3f83dad1b7e055f lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 19 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.438460:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ec600/0xb3f83dad1b7e0566 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 18 type: IBT flags: 0x0 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.438466:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ec600/0xb3f83dad1b7e0566 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 18 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.438479:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ec600/0xb3f83dad1b7e0566 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 18 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.438485:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ec600/0xb3f83dad1b7e0566 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 18 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.438491:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ec600/0xb3f83dad1b7e0566 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 18 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.438496:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9856129ec600 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826844.438499:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ec600/0xb3f83dad1b7e0566 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 18 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.438504:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ec600/0xb3f83dad1b7e0566 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 18 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.438510:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ec600/0xb3f83dad1b7e0566 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 18 type: IBT flags: 0x20e400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.438515:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826844.438518:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ec600/0xb3f83dad1b7e0566 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 18 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.438524:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ec600/0xb3f83dad1b7e0566 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 18 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.438534:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ede00/0xb3f83dad1b7e056d lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 17 type: IBT flags: 0x0 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.438540:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ede00/0xb3f83dad1b7e056d lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 17 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.438545:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ede00/0xb3f83dad1b7e056d lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 17 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.438550:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ede00/0xb3f83dad1b7e056d lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 17 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.438555:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ede00/0xb3f83dad1b7e056d lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 17 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.438560:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9856129ede00 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826844.438563:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ede00/0xb3f83dad1b7e056d lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 17 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.438568:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ede00/0xb3f83dad1b7e056d lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 17 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.438573:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ede00/0xb3f83dad1b7e056d lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 17 type: IBT flags: 0x20e400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.438578:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826844.438581:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ede00/0xb3f83dad1b7e056d lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 17 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.438586:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ede00/0xb3f83dad1b7e056d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 17 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.438595:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ec000/0xb3f83dad1b7e0574 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 16 type: IBT flags: 0x0 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.438600:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ec000/0xb3f83dad1b7e0574 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 16 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.438619:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ec000/0xb3f83dad1b7e0574 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 16 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.438630:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ec000/0xb3f83dad1b7e0574 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 16 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.438636:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ec000/0xb3f83dad1b7e0574 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 16 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.438640:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9856129ec000 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826844.438643:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ec000/0xb3f83dad1b7e0574 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 16 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.438648:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ec000/0xb3f83dad1b7e0574 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 16 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.438653:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ec000/0xb3f83dad1b7e0574 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 16 type: IBT flags: 0x20e400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.438659:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826844.438661:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ec000/0xb3f83dad1b7e0574 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 16 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.438667:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ec000/0xb3f83dad1b7e0574 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 16 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.438677:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129eca00/0xb3f83dad1b7e057b lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 15 type: IBT flags: 0x0 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.438683:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129eca00/0xb3f83dad1b7e057b lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 15 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.438688:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129eca00/0xb3f83dad1b7e057b lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 15 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.438714:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129eca00/0xb3f83dad1b7e057b lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 15 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.438722:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129eca00/0xb3f83dad1b7e057b lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 15 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.438891:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9856129eca00 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826844.438895:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129eca00/0xb3f83dad1b7e057b lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 15 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.438901:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129eca00/0xb3f83dad1b7e057b lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 15 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.438907:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129eca00/0xb3f83dad1b7e057b lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 15 type: IBT flags: 0x20e400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.438912:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826844.438916:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129eca00/0xb3f83dad1b7e057b lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 15 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.438921:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129eca00/0xb3f83dad1b7e057b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 15 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.438934:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ed600/0xb3f83dad1b7e0582 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 14 type: IBT flags: 0x0 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.438939:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ed600/0xb3f83dad1b7e0582 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 14 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.438945:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ed600/0xb3f83dad1b7e0582 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 14 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.438950:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ed600/0xb3f83dad1b7e0582 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 14 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.438955:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ed600/0xb3f83dad1b7e0582 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 14 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.438959:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9856129ed600 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826844.438962:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ed600/0xb3f83dad1b7e0582 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 14 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.438967:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ed600/0xb3f83dad1b7e0582 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 14 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.438972:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ed600/0xb3f83dad1b7e0582 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 14 type: IBT flags: 0x20e400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.438977:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826844.438980:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ed600/0xb3f83dad1b7e0582 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 14 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.438985:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ed600/0xb3f83dad1b7e0582 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 14 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.438994:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ed400/0xb3f83dad1b7e0589 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 13 type: IBT flags: 0x0 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.439000:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ed400/0xb3f83dad1b7e0589 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 13 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.439005:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ed400/0xb3f83dad1b7e0589 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 13 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.439010:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ed400/0xb3f83dad1b7e0589 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 13 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.439015:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ed400/0xb3f83dad1b7e0589 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 13 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.439019:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9856129ed400 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826844.439022:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ed400/0xb3f83dad1b7e0589 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 13 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.439027:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ed400/0xb3f83dad1b7e0589 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 13 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.439033:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ed400/0xb3f83dad1b7e0589 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 13 type: IBT flags: 0x20e400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.439038:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826844.439041:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ed400/0xb3f83dad1b7e0589 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 13 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.439047:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ed400/0xb3f83dad1b7e0589 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 13 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.439061:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ed800/0xb3f83dad1b7e0590 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 12 type: IBT flags: 0x0 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.439078:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ed800/0xb3f83dad1b7e0590 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 12 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.439084:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ed800/0xb3f83dad1b7e0590 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 12 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.439090:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ed800/0xb3f83dad1b7e0590 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 12 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.439095:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ed800/0xb3f83dad1b7e0590 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 12 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.439099:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9856129ed800 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826844.439102:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ed800/0xb3f83dad1b7e0590 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 12 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.439106:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ed800/0xb3f83dad1b7e0590 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 12 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.439111:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ed800/0xb3f83dad1b7e0590 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 12 type: IBT flags: 0x20e400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.439116:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826844.439119:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ed800/0xb3f83dad1b7e0590 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 12 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.439370:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ed800/0xb3f83dad1b7e0590 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 12 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.439399:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ec200/0xb3f83dad1b7e0597 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 11 type: IBT flags: 0x0 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.439407:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ec200/0xb3f83dad1b7e0597 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 11 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.439412:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ec200/0xb3f83dad1b7e0597 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 11 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.439419:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ec200/0xb3f83dad1b7e0597 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 11 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.439425:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ec200/0xb3f83dad1b7e0597 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 11 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.439430:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9856129ec200 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826844.439433:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ec200/0xb3f83dad1b7e0597 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 11 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.439439:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ec200/0xb3f83dad1b7e0597 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 11 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.439446:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ec200/0xb3f83dad1b7e0597 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 11 type: IBT flags: 0x20e400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.439452:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826844.439456:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ec200/0xb3f83dad1b7e0597 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 11 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.439462:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ec200/0xb3f83dad1b7e0597 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 11 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.439474:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ec400/0xb3f83dad1b7e059e lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 10 type: IBT flags: 0x0 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.439480:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ec400/0xb3f83dad1b7e059e lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 10 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.439486:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ec400/0xb3f83dad1b7e059e lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 10 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.439492:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ec400/0xb3f83dad1b7e059e lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 10 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.439498:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ec400/0xb3f83dad1b7e059e lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 10 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.439503:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9856129ec400 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826844.439506:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ec400/0xb3f83dad1b7e059e lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 10 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.439512:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ec400/0xb3f83dad1b7e059e lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 10 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.439517:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ec400/0xb3f83dad1b7e059e lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 10 type: IBT flags: 0x20e400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.439523:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826844.439526:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ec400/0xb3f83dad1b7e059e lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 10 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.439532:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ec400/0xb3f83dad1b7e059e lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 10 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.439542:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129eda00/0xb3f83dad1b7e05a5 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 9 type: IBT flags: 0x0 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.439549:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129eda00/0xb3f83dad1b7e05a5 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 9 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.439554:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129eda00/0xb3f83dad1b7e05a5 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 9 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.439560:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129eda00/0xb3f83dad1b7e05a5 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 9 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.439567:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129eda00/0xb3f83dad1b7e05a5 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 9 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.439572:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9856129eda00 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826844.439574:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129eda00/0xb3f83dad1b7e05a5 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 9 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.439580:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129eda00/0xb3f83dad1b7e05a5 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 9 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.439595:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129eda00/0xb3f83dad1b7e05a5 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 9 type: IBT flags: 0x20e400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.439604:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826844.439608:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129eda00/0xb3f83dad1b7e05a5 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 9 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.439614:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129eda00/0xb3f83dad1b7e05a5 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 9 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.439625:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ec800/0xb3f83dad1b7e05ac lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 8 type: IBT flags: 0x0 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.439631:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ec800/0xb3f83dad1b7e05ac lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 8 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.439863:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ec800/0xb3f83dad1b7e05ac lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 8 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.439872:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ec800/0xb3f83dad1b7e05ac lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 8 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.439879:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ec800/0xb3f83dad1b7e05ac lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 8 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.439885:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9856129ec800 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826844.439889:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ec800/0xb3f83dad1b7e05ac lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 8 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.439895:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ec800/0xb3f83dad1b7e05ac lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 8 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.439902:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ec800/0xb3f83dad1b7e05ac lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 8 type: IBT flags: 0x20e400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.439912:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826844.439917:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ec800/0xb3f83dad1b7e05ac lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 8 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.439923:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ec800/0xb3f83dad1b7e05ac lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 8 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.439937:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129edc00/0xb3f83dad1b7e05b3 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 7 type: IBT flags: 0x0 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.439943:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129edc00/0xb3f83dad1b7e05b3 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 7 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.439949:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129edc00/0xb3f83dad1b7e05b3 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 7 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.439955:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129edc00/0xb3f83dad1b7e05b3 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 7 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.439961:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129edc00/0xb3f83dad1b7e05b3 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 7 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.439966:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9856129edc00 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826844.439970:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129edc00/0xb3f83dad1b7e05b3 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 7 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.439976:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129edc00/0xb3f83dad1b7e05b3 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 7 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.439983:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129edc00/0xb3f83dad1b7e05b3 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 7 type: IBT flags: 0x20e400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.439995:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826844.440000:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129edc00/0xb3f83dad1b7e05b3 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 7 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.440008:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129edc00/0xb3f83dad1b7e05b3 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 7 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.440021:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ecc00/0xb3f83dad1b7e05ba lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 7 type: IBT flags: 0x0 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.440040:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ecc00/0xb3f83dad1b7e05ba lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 7 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.440077:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ecc00/0xb3f83dad1b7e05ba lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 6 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.440096:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ecc00/0xb3f83dad1b7e05ba lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 7 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.440104:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ecc00/0xb3f83dad1b7e05ba lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 6 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.440127:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9856129ecc00 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826844.440137:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ecc00/0xb3f83dad1b7e05ba lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 6 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.440148:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ecc00/0xb3f83dad1b7e05ba lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 6 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.440163:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ecc00/0xb3f83dad1b7e05ba lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 6 type: IBT flags: 0x20e400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.440182:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826844.440188:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ecc00/0xb3f83dad1b7e05ba lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 6 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.440200:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ecc00/0xb3f83dad1b7e05ba lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 6 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.440214:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ece00/0xb3f83dad1b7e05c1 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x0 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.440221:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ece00/0xb3f83dad1b7e05c1 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.440227:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ece00/0xb3f83dad1b7e05c1 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.440233:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ece00/0xb3f83dad1b7e05c1 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.440241:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ece00/0xb3f83dad1b7e05c1 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.440250:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9856129ece00 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826844.440256:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ece00/0xb3f83dad1b7e05c1 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 5 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.440277:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ece00/0xb3f83dad1b7e05c1 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 6 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.440295:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ece00/0xb3f83dad1b7e05c1 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 5 type: IBT flags: 0x20e400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.440306:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826844.440311:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ece00/0xb3f83dad1b7e05c1 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 5 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.440322:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856129ece00/0xb3f83dad1b7e05c1 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 5 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.440338:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dce000/0xb3f83dad1b7e05c8 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x0 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.440347:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dce000/0xb3f83dad1b7e05c8 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.440363:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dce000/0xb3f83dad1b7e05c8 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.440376:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dce000/0xb3f83dad1b7e05c8 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.440388:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dce000/0xb3f83dad1b7e05c8 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.440398:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985607dce000 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826844.440401:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dce000/0xb3f83dad1b7e05c8 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 4 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.440407:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dce000/0xb3f83dad1b7e05c8 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 4 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.440412:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dce000/0xb3f83dad1b7e05c8 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 4 type: IBT flags: 0x20e400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.440422:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826844.440428:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dce000/0xb3f83dad1b7e05c8 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.440440:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dce000/0xb3f83dad1b7e05c8 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.440455:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dce200/0xb3f83dad1b7e05cf lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x0 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.440466:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dce200/0xb3f83dad1b7e05cf lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.440485:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dce200/0xb3f83dad1b7e05cf lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.440496:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dce200/0xb3f83dad1b7e05cf lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.440509:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dce200/0xb3f83dad1b7e05cf lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.440522:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985607dce200 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826844.440529:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dce200/0xb3f83dad1b7e05cf lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 3 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.440548:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dce200/0xb3f83dad1b7e05cf lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 3 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.440556:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dce200/0xb3f83dad1b7e05cf lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 3 type: IBT flags: 0x20e400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.440572:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826844.440576:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dce200/0xb3f83dad1b7e05cf lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.440583:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dce200/0xb3f83dad1b7e05cf lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826844.440600:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669225568384:192.168.203.114@tcp:63:ll_sa.0 00000100:00100000:3.0:1787826844.441960:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669225569024:192.168.203.114@tcp:63:ll_sa.0 00010000:00010000:3.0:1787826844.445181:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcf000/0xb3f83dad1b7e05dd lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 35 type: IBT flags: 0x0 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.445193:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcf000/0xb3f83dad1b7e05dd lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 35 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.445200:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcf000/0xb3f83dad1b7e05dd lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 35 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.445206:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcf000/0xb3f83dad1b7e05dd lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 35 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.445213:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcf000/0xb3f83dad1b7e05dd lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 35 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.445219:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985607dcf000 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826844.445223:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcf000/0xb3f83dad1b7e05dd lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 35 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.445230:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcf000/0xb3f83dad1b7e05dd lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 35 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.445237:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcf000/0xb3f83dad1b7e05dd lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 35 type: IBT flags: 0x20e400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.445244:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826844.445250:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcf000/0xb3f83dad1b7e05dd lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 35 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.445256:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcf000/0xb3f83dad1b7e05dd lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 35 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.445905:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcec00/0xb3f83dad1b7e05e4 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 34 type: IBT flags: 0x0 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.445915:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcec00/0xb3f83dad1b7e05e4 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 34 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.445921:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcec00/0xb3f83dad1b7e05e4 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 34 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.445926:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcec00/0xb3f83dad1b7e05e4 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 34 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.445933:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcec00/0xb3f83dad1b7e05e4 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 34 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.445939:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985607dcec00 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826844.445942:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcec00/0xb3f83dad1b7e05e4 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 34 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.445948:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcec00/0xb3f83dad1b7e05e4 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 34 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.445959:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcec00/0xb3f83dad1b7e05e4 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 34 type: IBT flags: 0x20e400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.445964:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826844.445968:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcec00/0xb3f83dad1b7e05e4 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 34 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.445974:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcec00/0xb3f83dad1b7e05e4 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 34 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.445988:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcfc00/0xb3f83dad1b7e05eb lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 33 type: IBT flags: 0x0 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.445994:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcfc00/0xb3f83dad1b7e05eb lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 33 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.445998:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcfc00/0xb3f83dad1b7e05eb lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 33 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446003:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcfc00/0xb3f83dad1b7e05eb lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 33 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446008:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcfc00/0xb3f83dad1b7e05eb lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 33 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446013:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985607dcfc00 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826844.446015:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcfc00/0xb3f83dad1b7e05eb lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 33 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446021:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcfc00/0xb3f83dad1b7e05eb lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 33 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446030:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcfc00/0xb3f83dad1b7e05eb lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 33 type: IBT flags: 0x20e400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446035:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826844.446038:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcfc00/0xb3f83dad1b7e05eb lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 33 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446044:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcfc00/0xb3f83dad1b7e05eb lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 33 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446064:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcea00/0xb3f83dad1b7e05f2 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 32 type: IBT flags: 0x0 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446084:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcea00/0xb3f83dad1b7e05f2 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 32 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446090:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcea00/0xb3f83dad1b7e05f2 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 32 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446100:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcea00/0xb3f83dad1b7e05f2 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 32 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446109:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcea00/0xb3f83dad1b7e05f2 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 32 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446114:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985607dcea00 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826844.446116:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcea00/0xb3f83dad1b7e05f2 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 32 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446122:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcea00/0xb3f83dad1b7e05f2 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 32 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446131:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcea00/0xb3f83dad1b7e05f2 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 32 type: IBT flags: 0x20e400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446136:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826844.446140:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcea00/0xb3f83dad1b7e05f2 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 32 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446145:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcea00/0xb3f83dad1b7e05f2 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 32 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446157:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcf800/0xb3f83dad1b7e05f9 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 31 type: IBT flags: 0x0 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446163:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcf800/0xb3f83dad1b7e05f9 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 31 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446169:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcf800/0xb3f83dad1b7e05f9 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 31 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446180:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcf800/0xb3f83dad1b7e05f9 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 31 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446186:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcf800/0xb3f83dad1b7e05f9 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 31 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446191:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985607dcf800 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826844.446194:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcf800/0xb3f83dad1b7e05f9 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 31 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446199:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcf800/0xb3f83dad1b7e05f9 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 31 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446208:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcf800/0xb3f83dad1b7e05f9 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 31 type: IBT flags: 0x20e400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446213:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826844.446216:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcf800/0xb3f83dad1b7e05f9 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 31 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446222:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcf800/0xb3f83dad1b7e05f9 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 31 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446234:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcfa00/0xb3f83dad1b7e0600 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 30 type: IBT flags: 0x0 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446239:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcfa00/0xb3f83dad1b7e0600 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 30 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446245:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcfa00/0xb3f83dad1b7e0600 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 30 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446252:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcfa00/0xb3f83dad1b7e0600 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 30 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446256:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcfa00/0xb3f83dad1b7e0600 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 30 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446259:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985607dcfa00 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826844.446261:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcfa00/0xb3f83dad1b7e0600 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 30 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446264:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcfa00/0xb3f83dad1b7e0600 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 30 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446271:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcfa00/0xb3f83dad1b7e0600 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 30 type: IBT flags: 0x20e400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446274:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826844.446276:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcfa00/0xb3f83dad1b7e0600 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 30 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446279:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcfa00/0xb3f83dad1b7e0600 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 30 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446287:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcf200/0xb3f83dad1b7e0607 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 29 type: IBT flags: 0x0 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446291:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcf200/0xb3f83dad1b7e0607 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 29 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446294:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcf200/0xb3f83dad1b7e0607 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 29 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446297:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcf200/0xb3f83dad1b7e0607 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 29 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446300:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcf200/0xb3f83dad1b7e0607 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 29 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446303:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985607dcf200 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826844.446305:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcf200/0xb3f83dad1b7e0607 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 29 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446307:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcf200/0xb3f83dad1b7e0607 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 29 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446313:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcf200/0xb3f83dad1b7e0607 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 29 type: IBT flags: 0x20e400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446319:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826844.446321:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcf200/0xb3f83dad1b7e0607 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 29 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446327:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcf200/0xb3f83dad1b7e0607 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 29 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446338:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dce800/0xb3f83dad1b7e060e lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 28 type: IBT flags: 0x0 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446343:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dce800/0xb3f83dad1b7e060e lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 28 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446357:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dce800/0xb3f83dad1b7e060e lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 28 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446366:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dce800/0xb3f83dad1b7e060e lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 28 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446372:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dce800/0xb3f83dad1b7e060e lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 28 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446377:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985607dce800 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826844.446380:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dce800/0xb3f83dad1b7e060e lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 28 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446393:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dce800/0xb3f83dad1b7e060e lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 28 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446405:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dce800/0xb3f83dad1b7e060e lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 28 type: IBT flags: 0x20e400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446411:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826844.446414:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dce800/0xb3f83dad1b7e060e lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 28 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446421:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dce800/0xb3f83dad1b7e060e lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 28 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446433:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dce400/0xb3f83dad1b7e0615 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 27 type: IBT flags: 0x0 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446439:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dce400/0xb3f83dad1b7e0615 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 27 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446445:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dce400/0xb3f83dad1b7e0615 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 27 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446450:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dce400/0xb3f83dad1b7e0615 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 27 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446454:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dce400/0xb3f83dad1b7e0615 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 27 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446457:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985607dce400 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826844.446459:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dce400/0xb3f83dad1b7e0615 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 27 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446462:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dce400/0xb3f83dad1b7e0615 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 27 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446468:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dce400/0xb3f83dad1b7e0615 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 27 type: IBT flags: 0x20e400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446471:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826844.446473:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dce400/0xb3f83dad1b7e0615 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 27 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446484:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dce400/0xb3f83dad1b7e0615 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 27 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446496:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcf600/0xb3f83dad1b7e061c lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 26 type: IBT flags: 0x0 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446501:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcf600/0xb3f83dad1b7e061c lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 26 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446519:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcf600/0xb3f83dad1b7e061c lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 26 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446525:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcf600/0xb3f83dad1b7e061c lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 26 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446532:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcf600/0xb3f83dad1b7e061c lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 26 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446537:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985607dcf600 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826844.446540:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcf600/0xb3f83dad1b7e061c lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 26 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446546:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcf600/0xb3f83dad1b7e061c lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 26 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446555:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcf600/0xb3f83dad1b7e061c lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 26 type: IBT flags: 0x20e400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446560:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826844.446564:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcf600/0xb3f83dad1b7e061c lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 26 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446571:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcf600/0xb3f83dad1b7e061c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 26 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446582:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dce600/0xb3f83dad1b7e0623 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 25 type: IBT flags: 0x0 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446586:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dce600/0xb3f83dad1b7e0623 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 25 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446590:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dce600/0xb3f83dad1b7e0623 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 25 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446593:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dce600/0xb3f83dad1b7e0623 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 25 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446596:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dce600/0xb3f83dad1b7e0623 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 25 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446599:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985607dce600 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826844.446600:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dce600/0xb3f83dad1b7e0623 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 25 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446603:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dce600/0xb3f83dad1b7e0623 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 25 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446609:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dce600/0xb3f83dad1b7e0623 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 25 type: IBT flags: 0x20e400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446612:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826844.446614:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dce600/0xb3f83dad1b7e0623 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 25 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446617:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dce600/0xb3f83dad1b7e0623 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 25 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446624:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcfe00/0xb3f83dad1b7e062a lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 24 type: IBT flags: 0x0 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446627:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcfe00/0xb3f83dad1b7e062a lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 24 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446630:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcfe00/0xb3f83dad1b7e062a lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 24 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446633:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcfe00/0xb3f83dad1b7e062a lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 24 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446637:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcfe00/0xb3f83dad1b7e062a lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 24 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446640:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985607dcfe00 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826844.446641:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcfe00/0xb3f83dad1b7e062a lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 24 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446644:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcfe00/0xb3f83dad1b7e062a lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 24 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446650:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcfe00/0xb3f83dad1b7e062a lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 24 type: IBT flags: 0x20e400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446653:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826844.446655:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcfe00/0xb3f83dad1b7e062a lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 24 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446657:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcfe00/0xb3f83dad1b7e062a lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 24 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446663:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcee00/0xb3f83dad1b7e0631 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 23 type: IBT flags: 0x0 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446666:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcee00/0xb3f83dad1b7e0631 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 23 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446669:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcee00/0xb3f83dad1b7e0631 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 23 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446672:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcee00/0xb3f83dad1b7e0631 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 23 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446675:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcee00/0xb3f83dad1b7e0631 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 23 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446678:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985607dcee00 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826844.446679:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcee00/0xb3f83dad1b7e0631 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 23 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446682:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcee00/0xb3f83dad1b7e0631 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 23 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446688:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcee00/0xb3f83dad1b7e0631 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 23 type: IBT flags: 0x20e400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446773:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826844.446778:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcee00/0xb3f83dad1b7e0631 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 23 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446785:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcee00/0xb3f83dad1b7e0631 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 23 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446799:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcf400/0xb3f83dad1b7e0638 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 22 type: IBT flags: 0x0 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446805:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcf400/0xb3f83dad1b7e0638 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 22 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446812:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcf400/0xb3f83dad1b7e0638 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 22 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446819:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcf400/0xb3f83dad1b7e0638 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 22 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446825:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcf400/0xb3f83dad1b7e0638 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 22 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446831:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985607dcf400 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826844.446834:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcf400/0xb3f83dad1b7e0638 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 22 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446840:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcf400/0xb3f83dad1b7e0638 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 22 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446850:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcf400/0xb3f83dad1b7e0638 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 22 type: IBT flags: 0x20e400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446856:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826844.446860:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcf400/0xb3f83dad1b7e0638 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 22 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446866:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607dcf400/0xb3f83dad1b7e0638 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 22 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446881:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1400/0xb3f83dad1b7e063f lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 21 type: IBT flags: 0x0 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446890:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1400/0xb3f83dad1b7e063f lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 21 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446900:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1400/0xb3f83dad1b7e063f lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 22 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446925:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1400/0xb3f83dad1b7e063f lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 22 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446935:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1400/0xb3f83dad1b7e063f lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 21 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446958:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985610dc1400 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826844.446964:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1400/0xb3f83dad1b7e063f lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 22 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.446982:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1400/0xb3f83dad1b7e063f lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 22 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.447003:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1400/0xb3f83dad1b7e063f lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 21 type: IBT flags: 0x20e400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.447031:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826844.447038:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1400/0xb3f83dad1b7e063f lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 21 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.447053:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1400/0xb3f83dad1b7e063f lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 21 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.447081:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0400/0xb3f83dad1b7e0646 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 21 type: IBT flags: 0x0 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.447100:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0400/0xb3f83dad1b7e0646 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 20 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.447108:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0400/0xb3f83dad1b7e0646 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 20 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.447120:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0400/0xb3f83dad1b7e0646 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 20 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.447128:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0400/0xb3f83dad1b7e0646 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 20 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.447133:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985610dc0400 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826844.447136:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0400/0xb3f83dad1b7e0646 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 20 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.447143:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0400/0xb3f83dad1b7e0646 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 20 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.447159:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0400/0xb3f83dad1b7e0646 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 20 type: IBT flags: 0x20e400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.447169:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826844.447176:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0400/0xb3f83dad1b7e0646 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 20 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.447191:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0400/0xb3f83dad1b7e0646 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 21 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.447213:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0000/0xb3f83dad1b7e064d lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 20 type: IBT flags: 0x0 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.447222:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0000/0xb3f83dad1b7e064d lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 20 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.447241:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0000/0xb3f83dad1b7e064d lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 19 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.447246:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0000/0xb3f83dad1b7e064d lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 19 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.447253:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0000/0xb3f83dad1b7e064d lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 19 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.447263:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985610dc0000 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826844.447267:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0000/0xb3f83dad1b7e064d lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 19 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.447277:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0000/0xb3f83dad1b7e064d lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 19 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.447288:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0000/0xb3f83dad1b7e064d lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 19 type: IBT flags: 0x20e400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.447307:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826844.447312:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0000/0xb3f83dad1b7e064d lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 19 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.447323:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc0000/0xb3f83dad1b7e064d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 19 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.447339:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1800/0xb3f83dad1b7e0654 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 18 type: IBT flags: 0x0 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.447347:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1800/0xb3f83dad1b7e0654 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 18 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.447352:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1800/0xb3f83dad1b7e0654 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 18 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.447359:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1800/0xb3f83dad1b7e0654 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 18 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.447365:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1800/0xb3f83dad1b7e0654 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 18 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.447375:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985610dc1800 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826844.447380:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1800/0xb3f83dad1b7e0654 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 18 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.447386:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1800/0xb3f83dad1b7e0654 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 18 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.447398:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1800/0xb3f83dad1b7e0654 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 18 type: IBT flags: 0x20e400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.447405:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826844.447411:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1800/0xb3f83dad1b7e0654 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 18 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.447419:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610dc1800/0xb3f83dad1b7e0654 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 18 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.447430:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605710400/0xb3f83dad1b7e065b lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 17 type: IBT flags: 0x0 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.447441:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605710400/0xb3f83dad1b7e065b lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 17 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.447453:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605710400/0xb3f83dad1b7e065b lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 17 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.447465:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605710400/0xb3f83dad1b7e065b lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 17 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.447475:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605710400/0xb3f83dad1b7e065b lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 17 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.447494:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985605710400 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826844.447498:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605710400/0xb3f83dad1b7e065b lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 17 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.447515:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605710400/0xb3f83dad1b7e065b lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 17 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.447524:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605710400/0xb3f83dad1b7e065b lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 17 type: IBT flags: 0x20e400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.447536:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826844.447542:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605710400/0xb3f83dad1b7e065b lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 17 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.447551:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605710400/0xb3f83dad1b7e065b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 17 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.447562:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605710000/0xb3f83dad1b7e0662 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 16 type: IBT flags: 0x0 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.447574:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605710000/0xb3f83dad1b7e0662 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 16 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.447582:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605710000/0xb3f83dad1b7e0662 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 16 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.447589:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605710000/0xb3f83dad1b7e0662 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 16 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.447610:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605710000/0xb3f83dad1b7e0662 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 16 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.447627:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985605710000 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826844.447631:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605710000/0xb3f83dad1b7e0662 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 16 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.447638:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605710000/0xb3f83dad1b7e0662 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 16 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.447649:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605710000/0xb3f83dad1b7e0662 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 16 type: IBT flags: 0x20e400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.447666:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826844.447670:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605710000/0xb3f83dad1b7e0662 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 16 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.447679:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605710000/0xb3f83dad1b7e0662 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 16 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.447725:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605710200/0xb3f83dad1b7e0669 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 15 type: IBT flags: 0x0 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.447733:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605710200/0xb3f83dad1b7e0669 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 15 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.447741:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605710200/0xb3f83dad1b7e0669 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 15 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.447747:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605710200/0xb3f83dad1b7e0669 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 15 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.447758:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605710200/0xb3f83dad1b7e0669 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 15 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.447772:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985605710200 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826844.447777:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605710200/0xb3f83dad1b7e0669 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 15 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.447790:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605710200/0xb3f83dad1b7e0669 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 15 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.447805:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605710200/0xb3f83dad1b7e0669 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 15 type: IBT flags: 0x20e400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.447810:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826844.447812:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605710200/0xb3f83dad1b7e0669 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 15 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.447817:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605710200/0xb3f83dad1b7e0669 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 15 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.447828:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605711400/0xb3f83dad1b7e0670 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 14 type: IBT flags: 0x0 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.447835:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605711400/0xb3f83dad1b7e0670 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 14 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.447862:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605711400/0xb3f83dad1b7e0670 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 15 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.447869:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605711400/0xb3f83dad1b7e0670 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 15 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.447875:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605711400/0xb3f83dad1b7e0670 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 15 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.447880:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985605711400 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826844.447883:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605711400/0xb3f83dad1b7e0670 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 15 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.447889:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605711400/0xb3f83dad1b7e0670 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 15 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.447897:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605711400/0xb3f83dad1b7e0670 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 15 type: IBT flags: 0x20e400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.447903:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826844.447906:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605711400/0xb3f83dad1b7e0670 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 15 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.447912:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605711400/0xb3f83dad1b7e0670 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 15 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.447924:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605710e00/0xb3f83dad1b7e0677 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 15 type: IBT flags: 0x0 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.447943:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605710e00/0xb3f83dad1b7e0677 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 16 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.447958:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605710e00/0xb3f83dad1b7e0677 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 15 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.447965:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605710e00/0xb3f83dad1b7e0677 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 15 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.447972:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605710e00/0xb3f83dad1b7e0677 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 16 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.449250:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985605710e00 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826844.449257:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605710e00/0xb3f83dad1b7e0677 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 16 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.449276:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605710e00/0xb3f83dad1b7e0677 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 17 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.449290:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605710e00/0xb3f83dad1b7e0677 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 17 type: IBT flags: 0x20e400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.449324:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826844.449332:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605710e00/0xb3f83dad1b7e0677 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 18 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.449349:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605710e00/0xb3f83dad1b7e0677 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 17 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.449379:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605711e00/0xb3f83dad1b7e067e lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 16 type: IBT flags: 0x0 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.449402:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605711e00/0xb3f83dad1b7e067e lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 16 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.449422:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605711e00/0xb3f83dad1b7e067e lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 17 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.449428:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605711e00/0xb3f83dad1b7e067e lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 17 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.449434:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605711e00/0xb3f83dad1b7e067e lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 17 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.449439:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985605711e00 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826844.449441:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605711e00/0xb3f83dad1b7e067e lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 17 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.449450:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605711e00/0xb3f83dad1b7e067e lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 17 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.449460:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605711e00/0xb3f83dad1b7e067e lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 17 type: IBT flags: 0x20e400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.449465:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826844.449468:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605711e00/0xb3f83dad1b7e067e lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 17 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.449476:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605711e00/0xb3f83dad1b7e067e lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 16 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.449490:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605711600/0xb3f83dad1b7e0685 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 16 type: IBT flags: 0x0 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.449507:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605711600/0xb3f83dad1b7e0685 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 17 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.449524:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605711600/0xb3f83dad1b7e0685 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 16 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.449530:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605711600/0xb3f83dad1b7e0685 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 16 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.449536:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605711600/0xb3f83dad1b7e0685 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 16 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.449541:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985605711600 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826844.449544:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605711600/0xb3f83dad1b7e0685 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 16 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.449549:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605711600/0xb3f83dad1b7e0685 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 16 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.449567:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605711600/0xb3f83dad1b7e0685 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 18 type: IBT flags: 0x20e400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.449583:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826844.449588:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605711600/0xb3f83dad1b7e0685 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 17 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.449595:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605711600/0xb3f83dad1b7e0685 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 19 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.449620:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605710600/0xb3f83dad1b7e068c lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 18 type: IBT flags: 0x0 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.449628:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605710600/0xb3f83dad1b7e068c lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 17 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.449642:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605710600/0xb3f83dad1b7e068c lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 19 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.449659:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605710600/0xb3f83dad1b7e068c lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 18 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.449667:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605710600/0xb3f83dad1b7e068c lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 19 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.449680:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985605710600 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826844.449685:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605710600/0xb3f83dad1b7e068c lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 20 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.449713:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605710600/0xb3f83dad1b7e068c lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 20 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.449725:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605710600/0xb3f83dad1b7e068c lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 19 type: IBT flags: 0x20e400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.449740:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826844.449746:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605710600/0xb3f83dad1b7e068c lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 21 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.449756:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605710600/0xb3f83dad1b7e068c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 20 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.449768:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605711000/0xb3f83dad1b7e0693 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 21 type: IBT flags: 0x0 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.449786:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605711000/0xb3f83dad1b7e0693 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 21 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.449794:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605711000/0xb3f83dad1b7e0693 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 20 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.449800:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605711000/0xb3f83dad1b7e0693 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 21 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.449805:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605711000/0xb3f83dad1b7e0693 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 21 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.450116:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985605711000 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826844.450124:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605711000/0xb3f83dad1b7e0693 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 21 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.450133:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605711000/0xb3f83dad1b7e0693 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 20 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.450155:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605711000/0xb3f83dad1b7e0693 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 22 type: IBT flags: 0x20e400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.450171:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826844.450176:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605711000/0xb3f83dad1b7e0693 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 21 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.450186:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605711000/0xb3f83dad1b7e0693 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 23 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.450210:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605710c00/0xb3f83dad1b7e069a lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 22 type: IBT flags: 0x0 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.450217:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605710c00/0xb3f83dad1b7e069a lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 21 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.450233:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605710c00/0xb3f83dad1b7e069a lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 23 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.450249:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605710c00/0xb3f83dad1b7e069a lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 22 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.450256:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605710c00/0xb3f83dad1b7e069a lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 23 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.450268:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985605710c00 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826844.450274:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605710c00/0xb3f83dad1b7e069a lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 24 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.450284:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605710c00/0xb3f83dad1b7e069a lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 23 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.450301:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605710c00/0xb3f83dad1b7e069a lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 25 type: IBT flags: 0x20e400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.450317:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826844.450321:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605710c00/0xb3f83dad1b7e069a lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 24 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.450330:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605710c00/0xb3f83dad1b7e069a lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 26 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.450353:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605711c00/0xb3f83dad1b7e06a1 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 25 type: IBT flags: 0x0 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.450361:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605711c00/0xb3f83dad1b7e06a1 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 24 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.450376:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605711c00/0xb3f83dad1b7e06a1 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 26 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.450393:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605711c00/0xb3f83dad1b7e06a1 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 25 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.450400:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605711c00/0xb3f83dad1b7e06a1 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 25 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.450413:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985605711c00 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826844.450418:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605711c00/0xb3f83dad1b7e06a1 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 27 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.450425:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605711c00/0xb3f83dad1b7e06a1 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 27 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.450434:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605711c00/0xb3f83dad1b7e06a1 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 27 type: IBT flags: 0x20e400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.450439:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826844.450441:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605711c00/0xb3f83dad1b7e06a1 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 27 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.450446:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605711c00/0xb3f83dad1b7e06a1 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 27 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.450456:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605710a00/0xb3f83dad1b7e06a8 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 26 type: IBT flags: 0x0 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.450462:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605710a00/0xb3f83dad1b7e06a8 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 26 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.450467:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605710a00/0xb3f83dad1b7e06a8 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 26 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.450472:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605710a00/0xb3f83dad1b7e06a8 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 26 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.450478:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605710a00/0xb3f83dad1b7e06a8 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 26 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.450485:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985605710a00 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826844.450524:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605710a00/0xb3f83dad1b7e06a8 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 27 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.450533:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605710a00/0xb3f83dad1b7e06a8 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 26 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.450542:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605710a00/0xb3f83dad1b7e06a8 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 26 type: IBT flags: 0x20e400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.450548:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826844.450553:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605710a00/0xb3f83dad1b7e06a8 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 28 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.450562:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605710a00/0xb3f83dad1b7e06a8 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 27 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.450576:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605711800/0xb3f83dad1b7e06af lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 25 type: IBT flags: 0x0 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.450592:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605711800/0xb3f83dad1b7e06af lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 25 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.450611:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605711800/0xb3f83dad1b7e06af lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 25 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.450617:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605711800/0xb3f83dad1b7e06af lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 25 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.450624:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605711800/0xb3f83dad1b7e06af lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 25 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.450629:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985605711800 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826844.450631:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605711800/0xb3f83dad1b7e06af lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 25 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.450638:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605711800/0xb3f83dad1b7e06af lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 25 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.450647:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605711800/0xb3f83dad1b7e06af lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 25 type: IBT flags: 0x20e400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.450653:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826844.450659:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605711800/0xb3f83dad1b7e06af lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 25 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.450671:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605711800/0xb3f83dad1b7e06af lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 26 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.450687:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605711a00/0xb3f83dad1b7e06b6 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 24 type: IBT flags: 0x0 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.450712:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605711a00/0xb3f83dad1b7e06b6 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 24 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.450719:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605711a00/0xb3f83dad1b7e06b6 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 25 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.450737:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605711a00/0xb3f83dad1b7e06b6 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 25 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.450743:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605711a00/0xb3f83dad1b7e06b6 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 23 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.450753:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985605711a00 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826844.450756:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605711a00/0xb3f83dad1b7e06b6 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 23 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.450764:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605711a00/0xb3f83dad1b7e06b6 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 23 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.450785:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605711a00/0xb3f83dad1b7e06b6 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 23 type: IBT flags: 0x20e400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.450792:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826844.450795:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605711a00/0xb3f83dad1b7e06b6 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 23 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.450808:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605711a00/0xb3f83dad1b7e06b6 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 23 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.450822:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605711200/0xb3f83dad1b7e06bd lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 22 type: IBT flags: 0x0 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.450828:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605711200/0xb3f83dad1b7e06bd lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 22 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.450834:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605711200/0xb3f83dad1b7e06bd lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 22 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.450840:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605711200/0xb3f83dad1b7e06bd lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 22 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.450846:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605711200/0xb3f83dad1b7e06bd lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 22 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.450852:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985605711200 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826844.450854:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605711200/0xb3f83dad1b7e06bd lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 22 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.450859:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605711200/0xb3f83dad1b7e06bd lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 22 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.450867:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605711200/0xb3f83dad1b7e06bd lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 22 type: IBT flags: 0x20e400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.450873:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826844.450880:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605711200/0xb3f83dad1b7e06bd lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 22 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826844.450886:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605711200/0xb3f83dad1b7e06bd lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 22 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826844.450904:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669225569024:192.168.203.114@tcp:63:ll_sa.0 00000100:00100000:3.0:1787826844.452221:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669225569536:192.168.203.114@tcp:63:ll_sa.0 00000100:00100000:3.0:1787826844.452332:0:2385:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1787826844.452339:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669225569664:192.168.203.114@tcp:63:ll_sa.0 00000100:00100000:1.0:1787826844.453317:0:79202:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e79a40 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:79202:1874669225569408:192.168.203.114@tcp:36:rm.0 00000080:00200000:1.0:1787826844.453361:0:79202:0:(vvp_io.c:1885:vvp_io_init()) [0x200016b60:0x5:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826844.453374:0:79202:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826844.453429:0:79202:0:(vvp_io.c:304:vvp_io_fini()) [0x200016b60:0x5:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826844.453437:0:79202:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200016b60:0x5:0x0](ffff985611589148) 00000008:00010000:1.0:1787826844.453463:0:79202:0:(osc_object.c:202:osc_object_ast_clear()) ### update lock size 0 blocks 0 [cma]time: 0 0 0 by oinfo size 0 blocks 0 [cma]time 0 0 0 ns: lustre-OST0000-osc-ffff985602bc9800 lock: 000000002c3b7699/0xb3f83dad1b7e04ef lrc: 1/0,0 mode: PR/PR res: [0x280000401:0x604:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x5aa6d504d91a19c7 expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826844.453535:0:79202:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826844.453550:0:79202:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826844.453558:0:79202:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 30 res 8589934599/1 (0 0) 00010000:00010000:1.0:1787826844.453591:0:79202:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e0892 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 67 type: IBT flags: 0x10000000000000 pid: 79202 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.453600:0:79202:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e0892 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 67 type: IBT flags: 0x0 pid: 79202 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.453610:0:79202:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e0892 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 67 type: IBT flags: 0x0 pid: 79202 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826844.453624:0:79202:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610d3da40 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:79202:1874669225569792:192.168.203.114@tcp:101:rm.0 00000100:00100000:1.0:1787826844.453669:0:79202:0:(client.c:2715:ptlrpc_set_wait()) set ffff98560957d300 going to sleep for 11 seconds 00000100:00100000:1.0:1787826844.455239:0:79202:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610d3da40 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:79202:1874669225569792:192.168.203.114@tcp:101:rm.0 00010000:00010000:1.0:1787826844.455251:0:79202:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e0892 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 66 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1b10 expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.455260:0:79202:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e0892 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 66 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1b10 expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.455271:0:79202:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e0892 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 66 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1b10 expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.455281:0:79202:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e0892 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 66 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1b10 expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.455292:0:79202:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e0892 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 66 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1b10 expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.455299:0:79202:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e0892 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 66 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1b10 expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.455307:0:79202:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e0892 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 66 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1b10 expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826844.455319:0:79202:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff985610d3da40 x1874669225569792/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826860 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826844.455331:0:79202:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e0892 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 66 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1b10 expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.455341:0:79202:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:1.0:1787826844.455346:0:79202:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:1.0:1787826844.455366:0:79202:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:1.0:1787826844.455372:0:79202:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e0892 00000080:00010000:1.0:1787826844.455377:0:79202:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826844.455381:0:79202:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0892 from it ffffab3ac0dd3b58 00010000:00010000:1.0:1787826844.455384:0:79202:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e0892 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 66 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1b10 expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.455392:0:79202:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e0892 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 66 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1b10 expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826844.455408:0:79202:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787826844.455417:0:79202:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f33b.replay-single--5, flags=8 00010000:00010000:1.0:1787826844.455430:0:79202:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051000/0xb3f83dad1b7e048d lrc: 3/1,0 mode: PR/PR res: [0x200016b60:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a18cb expref: -99 pid: 79201 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00000080:00010000:1.0:1787826844.455437:0:79202:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e048d from it ffffab3ac0dd3af8 00010000:00010000:1.0:1787826844.455441:0:79202:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051000/0xb3f83dad1b7e048d lrc: 3/1,0 mode: PR/PR res: [0x200016b60:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a18cb expref: -99 pid: 79201 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826844.455448:0:79202:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051000/0xb3f83dad1b7e048d lrc: 2/0,0 mode: PR/PR res: [0x200016b60:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a18cb expref: -99 pid: 79201 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200008:1.0:1787826844.455465:0:79202:0:(file.c:6313:ll_getattr_dentry()) START file f33b.replay-single--5:[0x200016b60:0x6:0x0](ffff985611588908), request_mask 2047, flags 0, foreign 0 00000080:00200000:1.0:1787826844.455471:0:79202:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200016b60:0x6:0x0](ffff985611588908),name=f33b.replay-single--5 00000002:00010000:1.0:1787826844.455483:0:79202:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200016b60:0x6:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826844.455491:0:79202:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051000/0xb3f83dad1b7e048d lrc: 2/1,0 mode: PR/PR res: [0x200016b60:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a18cb expref: -99 pid: 79201 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787826844.455500:0:79202:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051000/0xb3f83dad1b7e048d lrc: 2/1,0 mode: PR/PR res: [0x200016b60:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a18cb expref: -99 pid: 79201 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787826844.455507:0:79202:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826844.455511:0:79202:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200016b60:0x6:0x0](ffff985611588908) 00000080:00010000:1.0:1787826844.455514:0:79202:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200016b60:0x6:0x0](ffff985611588908) for lock 0xb3f83dad1b7e048d 00000080:00010000:1.0:1787826844.455518:0:79202:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e048d from it ffffab3ac0dd3cb8 00010000:00010000:1.0:1787826844.455521:0:79202:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051000/0xb3f83dad1b7e048d lrc: 3/1,0 mode: PR/PR res: [0x200016b60:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a18cb expref: -99 pid: 79201 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787826844.455528:0:79202:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051000/0xb3f83dad1b7e048d lrc: 2/0,0 mode: PR/PR res: [0x200016b60:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a18cb expref: -99 pid: 79201 timeout: 0 lvb_type: 3 lru_score: 4 lru_type: 0 00000080:00200000:1.0:1787826844.455544:0:79202:0:(vvp_io.c:1885:vvp_io_init()) [0x200016b60:0x6:0x0] ignore/verify layout 0/1, layout version 0 restore needed 0 00000001:00010000:1.0:1787826844.455552:0:79202:0:(glimpse.c:73:cl_glimpse_lock()) Glimpsing inode [0x200016b60:0x6:0x0] 00000008:00010000:1.0:1787826844.455565:0:79202:0:(lustre_osc.h:166:osc_enq2ldlm_flags()) flags: a 00000008:00010000:1.0:1787826844.455569:0:79202:0:(osc_lock.c:1320:osc_lock_init()) ### lock ffff98560560c0b0, osc lock ffff985602f8f380, flags 0x400001002 00010000:00010000:1.0:1787826844.455583:0:79202:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0001-osc-ffff985602bc9800 lock: 00000000b69c5f27/0xb3f83dad1b7e04f6 lrc: 2/1,0 mode: PR/PR res: [0x2c0000401:0x604:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a19ce expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826844.455594:0:79202:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 18446744073709551615) ns: lustre-OST0001-osc-ffff985602bc9800 lock: 00000000b69c5f27/0xb3f83dad1b7e04f6 lrc: 2/1,0 mode: PR/PR res: [0x2c0000401:0x604:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a19ce expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000008:00010000:1.0:1787826844.455607:0:79202:0:(osc_lock.c:192:osc_lock_lvb_update()) ### acquired size 0, setting rss=0; kms=0, end=18446744073709551615 ns: lustre-OST0001-osc-ffff985602bc9800 lock: 00000000b69c5f27/0xb3f83dad1b7e04f6 lrc: 4/1,0 mode: PR/PR res: [0x2c0000401:0x604:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a19ce expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000020:00200000:1.0:1787826844.455618:0:79202:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00010000:00010000:1.0:1787826844.455621:0:79202:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0001-osc-ffff985602bc9800 lock: 00000000b69c5f27/0xb3f83dad1b7e04f6 lrc: 5/1,0 mode: PR/PR res: [0x2c0000401:0x604:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x5aa6d504d91a19ce expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826844.455630:0:79202:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0001-osc-ffff985602bc9800 lock: 00000000b69c5f27/0xb3f83dad1b7e04f6 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x604:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x5aa6d504d91a19ce expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000020:00200000:1.0:1787826844.455640:0:79202:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1787826844.455651:0:79202:0:(file.c:1775:ll_merge_attr_nolock()) [0x200016b60:0x6:0x0] updating i_size 0 i_blocks 0 00000020:00010000:1.0:1787826844.455655:0:79202:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff985605499638 (ffff985610fe17e8/0) at cl_lock_release():245 00000020:00010000:1.0:1787826844.455658:0:79202:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff985605499638 (ffff985610fe17e8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1787826844.455661:0:79202:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff98560560c0b0 (ffff985610fe1488/1) at cl_lock_cancel():142 00000020:00010000:1.0:1787826844.455666:0:79202:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff985605499638 (ffff985610fe17e8/0) at cl_lock_fini():72 00000020:00010000:1.0:1787826844.455669:0:79202:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff98560560c0b0 (ffff985610fe17e8/1) at cl_lock_fini():72 00000080:00200000:1.0:1787826844.455682:0:79202:0:(vvp_io.c:304:vvp_io_fini()) [0x200016b60:0x6:0x0] ignore/verify layout 0/1, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826844.455735:0:79202:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826844.455745:0:79202:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826844.455752:0:79202:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e0892 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 67 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1b10 expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826844.455761:0:79202:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e0892 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 66 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1b10 expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826844.455767:0:79202:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826844.455772:0:79202:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826844.455774:0:79202:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e0892 00000080:00010000:1.0:1787826844.455778:0:79202:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0892 from it ffffab3ac0dd3c10 00010000:00010000:1.0:1787826844.455781:0:79202:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e0892 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 66 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1b10 expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826844.455788:0:79202:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e0892 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 66 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1b10 expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826844.455797:0:79202:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787826844.455804:0:79202:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f33b.replay-single--5, flags=0 00000080:00200000:1.0:1787826844.455807:0:79202:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826844.455814:0:79202:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826844.455821:0:79202:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e0892 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 67 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1b10 expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826844.455829:0:79202:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e0892 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 66 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1b10 expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826844.455835:0:79202:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826844.455839:0:79202:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826844.455842:0:79202:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e0892 00000080:00010000:1.0:1787826844.455846:0:79202:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0892 from it ffffab3ac0dd3d68 00010000:00010000:1.0:1787826844.455849:0:79202:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e0892 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 66 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1b10 expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826844.455856:0:79202:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e0892 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 66 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1b10 expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787826844.455864:0:79202:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787826844.455870:0:79202:0:(namei.c:2418:ll_unlink()) VFS Op:name=f33b.replay-single--5, dir=[0x200000007:0x1:0x0](ffff98561158e3c8) 00010000:00010000:1.0:1787826844.455891:0:79202:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e0892 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 67 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x5aa6d504d91a1b10 expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787826844.455911:0:79202:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051000/0xb3f83dad1b7e048d lrc: 2/0,0 mode: PR/PR res: [0x200016b60:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x5aa6d504d91a18cb expref: -99 pid: 79201 timeout: 0 lvb_type: 3 lru_score: 4 lru_type: 0 00000080:00200000:1.0:1787826844.455921:0:79202:0:(vvp_object.c:117:vvp_conf_set()) [0x200016b60:0x6:0x0]: losing layout lock 00010000:00010000:1.0:1787826844.455947:0:79202:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051000/0xb3f83dad1b7e048d lrc: 1/0,0 mode: --/PR res: [0x200016b60:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x5aa6d504d91a18cb expref: -99 pid: 79201 timeout: 0 lvb_type: 3 lru_score: 4 lru_type: 0 00010000:00010000:1.0:1787826844.455960:0:79202:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e0892 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 66 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a1b10 expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787826844.455966:0:79202:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 2 locks packed 00010000:00010000:1.0:1787826844.455969:0:79202:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051000/0xb3f83dad1b7e048d lrc: 0/0,0 mode: --/PR res: [0x200016b60:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x5aa6d504d91a18cb expref: -99 pid: 79201 timeout: 0 lvb_type: 3 lru_score: 4 lru_type: 0 00010000:00010000:1.0:1787826844.455983:0:79202:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e0892 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 66 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a1b10 expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000020:00100000:1.0:1787826844.456004:0:79202:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787826844.456011:0:79202:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610d3da40 x1874669225570048/t0(0) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826844.456028:0:79202:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610d3da40 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:79202:1874669225570048:192.168.203.114@tcp:36:rm.0 00000100:00100000:1.0:1787826844.456095:0:79202:0:(client.c:2715:ptlrpc_set_wait()) set ffff98560957d300 going to sleep for 11 seconds 00000100:00100000:1.0:1787826844.459254:0:79202:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610d3da40 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:79202:1874669225570048:192.168.203.114@tcp:36:rm.0 00000080:00200000:1.0:1787826844.459299:0:79202:0:(vvp_io.c:1885:vvp_io_init()) [0x200016b60:0x6:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826844.459312:0:79202:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826844.459338:0:79202:0:(vvp_io.c:304:vvp_io_fini()) [0x200016b60:0x6:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826844.459350:0:79202:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200016b60:0x6:0x0](ffff985611588908) 00000008:00010000:1.0:1787826844.459381:0:79202:0:(osc_object.c:202:osc_object_ast_clear()) ### update lock size 0 blocks 0 [cma]time: 0 0 0 by oinfo size 0 blocks 0 [cma]time 0 0 0 ns: lustre-OST0001-osc-ffff985602bc9800 lock: 00000000b69c5f27/0xb3f83dad1b7e04f6 lrc: 1/0,0 mode: PR/PR res: [0x2c0000401:0x604:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x5aa6d504d91a19ce expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826844.459469:0:79202:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826844.459488:0:79202:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826844.459499:0:79202:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 30 res 8589934599/1 (0 0) 00010000:00010000:1.0:1787826844.459540:0:79202:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e0899 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 30 type: IBT flags: 0x10000000000000 pid: 79202 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.459554:0:79202:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e0899 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 29 type: IBT flags: 0x0 pid: 79202 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.459568:0:79202:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e0899 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 30 type: IBT flags: 0x0 pid: 79202 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826844.459593:0:79202:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610d3c000 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:79202:1874669225570176:192.168.203.114@tcp:101:rm.0 00000100:00100000:1.0:1787826844.459707:0:79202:0:(client.c:2715:ptlrpc_set_wait()) set ffff98560957d240 going to sleep for 11 seconds 00010000:00010000:2.0:1787826844.459722:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985607da8e00 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:2.0:1787826844.459726:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8e00/0xb3f83dad1b7e07dc lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 27 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.459732:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8e00/0xb3f83dad1b7e07dc lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 27 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.459737:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8e00/0xb3f83dad1b7e07dc lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 27 type: IBT flags: 0x20e400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.459742:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1787826844.459746:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8e00/0xb3f83dad1b7e07dc lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 27 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.459752:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8e00/0xb3f83dad1b7e07dc lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 27 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.459765:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9e00/0xb3f83dad1b7e07e3 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 26 type: IBT flags: 0x0 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.459771:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9e00/0xb3f83dad1b7e07e3 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 26 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.459777:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9e00/0xb3f83dad1b7e07e3 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 26 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.459782:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9e00/0xb3f83dad1b7e07e3 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 26 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.459788:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9e00/0xb3f83dad1b7e07e3 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 26 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.459792:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985607da9e00 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:2.0:1787826844.459795:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9e00/0xb3f83dad1b7e07e3 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 26 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.459801:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9e00/0xb3f83dad1b7e07e3 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 26 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.459807:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9e00/0xb3f83dad1b7e07e3 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 26 type: IBT flags: 0x20e400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.459812:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1787826844.459816:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9e00/0xb3f83dad1b7e07e3 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 26 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.459822:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9e00/0xb3f83dad1b7e07e3 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 26 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.459833:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9600/0xb3f83dad1b7e07ea lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 25 type: IBT flags: 0x0 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.459839:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9600/0xb3f83dad1b7e07ea lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 25 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.459844:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9600/0xb3f83dad1b7e07ea lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 25 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.459849:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9600/0xb3f83dad1b7e07ea lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 25 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.459854:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9600/0xb3f83dad1b7e07ea lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 25 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.459859:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985607da9600 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:2.0:1787826844.459862:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9600/0xb3f83dad1b7e07ea lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 25 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.459867:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9600/0xb3f83dad1b7e07ea lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 25 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.459874:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9600/0xb3f83dad1b7e07ea lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 25 type: IBT flags: 0x20e400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.459879:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1787826844.459882:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9600/0xb3f83dad1b7e07ea lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 25 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.459889:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9600/0xb3f83dad1b7e07ea lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 25 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.459899:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8600/0xb3f83dad1b7e07f1 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 24 type: IBT flags: 0x0 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.459905:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8600/0xb3f83dad1b7e07f1 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 24 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.459911:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8600/0xb3f83dad1b7e07f1 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 24 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.459917:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8600/0xb3f83dad1b7e07f1 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 24 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.459924:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8600/0xb3f83dad1b7e07f1 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 24 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.459928:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985607da8600 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:2.0:1787826844.459931:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8600/0xb3f83dad1b7e07f1 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 24 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.459937:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8600/0xb3f83dad1b7e07f1 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 24 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.459942:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8600/0xb3f83dad1b7e07f1 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 24 type: IBT flags: 0x20e400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.459947:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1787826844.459950:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8600/0xb3f83dad1b7e07f1 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 24 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.459956:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8600/0xb3f83dad1b7e07f1 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 24 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.459967:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9000/0xb3f83dad1b7e07f8 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 23 type: IBT flags: 0x0 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.459973:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9000/0xb3f83dad1b7e07f8 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 23 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.459980:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9000/0xb3f83dad1b7e07f8 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 23 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.459986:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9000/0xb3f83dad1b7e07f8 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 23 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.459992:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9000/0xb3f83dad1b7e07f8 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 23 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.459998:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985607da9000 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:2.0:1787826844.460001:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9000/0xb3f83dad1b7e07f8 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 23 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460008:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9000/0xb3f83dad1b7e07f8 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 23 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460015:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9000/0xb3f83dad1b7e07f8 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 23 type: IBT flags: 0x20e400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460020:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1787826844.460024:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9000/0xb3f83dad1b7e07f8 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 23 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460030:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9000/0xb3f83dad1b7e07f8 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 23 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460042:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8c00/0xb3f83dad1b7e07ff lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 22 type: IBT flags: 0x0 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460048:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8c00/0xb3f83dad1b7e07ff lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 22 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460054:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8c00/0xb3f83dad1b7e07ff lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 22 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460060:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8c00/0xb3f83dad1b7e07ff lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 22 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460078:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8c00/0xb3f83dad1b7e07ff lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 22 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460083:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985607da8c00 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:2.0:1787826844.460086:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8c00/0xb3f83dad1b7e07ff lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 22 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460092:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8c00/0xb3f83dad1b7e07ff lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 22 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460098:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8c00/0xb3f83dad1b7e07ff lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 22 type: IBT flags: 0x20e400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460103:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1787826844.460106:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8c00/0xb3f83dad1b7e07ff lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 22 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460113:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8c00/0xb3f83dad1b7e07ff lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 22 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460123:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9c00/0xb3f83dad1b7e0806 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 21 type: IBT flags: 0x0 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460128:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9c00/0xb3f83dad1b7e0806 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 21 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460134:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9c00/0xb3f83dad1b7e0806 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 21 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460139:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9c00/0xb3f83dad1b7e0806 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 21 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460145:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9c00/0xb3f83dad1b7e0806 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 21 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460150:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985607da9c00 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:2.0:1787826844.460153:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9c00/0xb3f83dad1b7e0806 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 21 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460159:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9c00/0xb3f83dad1b7e0806 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 21 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460164:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9c00/0xb3f83dad1b7e0806 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 21 type: IBT flags: 0x20e400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460170:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1787826844.460172:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9c00/0xb3f83dad1b7e0806 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 21 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460178:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9c00/0xb3f83dad1b7e0806 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 21 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460189:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8a00/0xb3f83dad1b7e080d lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 20 type: IBT flags: 0x0 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460195:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8a00/0xb3f83dad1b7e080d lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 20 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460200:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8a00/0xb3f83dad1b7e080d lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 20 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460206:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8a00/0xb3f83dad1b7e080d lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 20 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460211:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8a00/0xb3f83dad1b7e080d lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 20 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460216:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985607da8a00 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:2.0:1787826844.460219:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8a00/0xb3f83dad1b7e080d lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 20 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460224:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8a00/0xb3f83dad1b7e080d lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 20 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460229:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8a00/0xb3f83dad1b7e080d lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 20 type: IBT flags: 0x20e400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460233:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1787826844.460236:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8a00/0xb3f83dad1b7e080d lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 20 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460241:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8a00/0xb3f83dad1b7e080d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 20 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460252:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9800/0xb3f83dad1b7e0814 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 19 type: IBT flags: 0x0 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460257:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9800/0xb3f83dad1b7e0814 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 19 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460263:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9800/0xb3f83dad1b7e0814 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 19 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460268:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9800/0xb3f83dad1b7e0814 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 19 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460274:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9800/0xb3f83dad1b7e0814 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 19 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460278:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985607da9800 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:2.0:1787826844.460281:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9800/0xb3f83dad1b7e0814 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 19 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460286:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9800/0xb3f83dad1b7e0814 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 19 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460291:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9800/0xb3f83dad1b7e0814 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 19 type: IBT flags: 0x20e400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460296:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1787826844.460298:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9800/0xb3f83dad1b7e0814 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 19 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460304:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9800/0xb3f83dad1b7e0814 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 19 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460315:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9a00/0xb3f83dad1b7e081b lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 18 type: IBT flags: 0x0 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460320:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9a00/0xb3f83dad1b7e081b lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 18 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460325:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9a00/0xb3f83dad1b7e081b lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 18 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460330:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9a00/0xb3f83dad1b7e081b lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 18 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460335:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9a00/0xb3f83dad1b7e081b lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 18 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460340:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985607da9a00 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:2.0:1787826844.460344:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9a00/0xb3f83dad1b7e081b lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 18 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460349:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9a00/0xb3f83dad1b7e081b lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 18 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460354:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9a00/0xb3f83dad1b7e081b lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 18 type: IBT flags: 0x20e400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460359:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1787826844.460363:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9a00/0xb3f83dad1b7e081b lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 18 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460369:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9a00/0xb3f83dad1b7e081b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 18 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460378:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9200/0xb3f83dad1b7e0822 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 17 type: IBT flags: 0x0 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460384:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9200/0xb3f83dad1b7e0822 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 17 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460390:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9200/0xb3f83dad1b7e0822 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 17 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460396:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9200/0xb3f83dad1b7e0822 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 17 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460401:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9200/0xb3f83dad1b7e0822 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 17 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460406:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985607da9200 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:2.0:1787826844.460409:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9200/0xb3f83dad1b7e0822 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 17 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460414:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9200/0xb3f83dad1b7e0822 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 17 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460420:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9200/0xb3f83dad1b7e0822 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 17 type: IBT flags: 0x20e400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460425:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1787826844.460428:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9200/0xb3f83dad1b7e0822 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 17 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460434:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9200/0xb3f83dad1b7e0822 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 17 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460446:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8800/0xb3f83dad1b7e0829 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 16 type: IBT flags: 0x0 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460451:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8800/0xb3f83dad1b7e0829 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 16 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460457:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8800/0xb3f83dad1b7e0829 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 16 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460462:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8800/0xb3f83dad1b7e0829 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 16 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460468:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8800/0xb3f83dad1b7e0829 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 16 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460473:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985607da8800 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:2.0:1787826844.460476:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8800/0xb3f83dad1b7e0829 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 16 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460481:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8800/0xb3f83dad1b7e0829 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 16 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460487:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8800/0xb3f83dad1b7e0829 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 16 type: IBT flags: 0x20e400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460492:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1787826844.460494:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8800/0xb3f83dad1b7e0829 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 16 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460500:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8800/0xb3f83dad1b7e0829 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 16 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460511:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8400/0xb3f83dad1b7e0830 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 15 type: IBT flags: 0x0 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460516:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8400/0xb3f83dad1b7e0830 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 15 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460522:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8400/0xb3f83dad1b7e0830 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 15 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460528:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8400/0xb3f83dad1b7e0830 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 15 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460533:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8400/0xb3f83dad1b7e0830 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 15 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460539:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985607da8400 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:2.0:1787826844.460542:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8400/0xb3f83dad1b7e0830 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 15 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460547:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8400/0xb3f83dad1b7e0830 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 15 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460552:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8400/0xb3f83dad1b7e0830 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 15 type: IBT flags: 0x20e400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460557:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1787826844.460560:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8400/0xb3f83dad1b7e0830 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 15 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460566:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8400/0xb3f83dad1b7e0830 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 15 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460577:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4400/0xb3f83dad1b7e0837 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 14 type: IBT flags: 0x0 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460583:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4400/0xb3f83dad1b7e0837 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 14 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460588:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4400/0xb3f83dad1b7e0837 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 14 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460593:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4400/0xb3f83dad1b7e0837 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 14 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460598:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4400/0xb3f83dad1b7e0837 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 14 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460603:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9856056d4400 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:2.0:1787826844.460607:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4400/0xb3f83dad1b7e0837 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 14 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460612:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4400/0xb3f83dad1b7e0837 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 14 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460617:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4400/0xb3f83dad1b7e0837 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 14 type: IBT flags: 0x20e400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460623:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1787826844.460626:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4400/0xb3f83dad1b7e0837 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 14 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460632:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4400/0xb3f83dad1b7e0837 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 14 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460642:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4000/0xb3f83dad1b7e083e lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 13 type: IBT flags: 0x0 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460647:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4000/0xb3f83dad1b7e083e lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 13 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460652:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4000/0xb3f83dad1b7e083e lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 13 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460657:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4000/0xb3f83dad1b7e083e lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 13 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460663:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4000/0xb3f83dad1b7e083e lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 13 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460667:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9856056d4000 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:2.0:1787826844.460670:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4000/0xb3f83dad1b7e083e lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 13 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460675:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4000/0xb3f83dad1b7e083e lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 13 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460681:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4000/0xb3f83dad1b7e083e lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 13 type: IBT flags: 0x20e400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460686:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1787826844.460689:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4000/0xb3f83dad1b7e083e lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 13 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460713:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4000/0xb3f83dad1b7e083e lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 13 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460725:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4200/0xb3f83dad1b7e0845 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 12 type: IBT flags: 0x0 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460732:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4200/0xb3f83dad1b7e0845 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 12 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460738:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4200/0xb3f83dad1b7e0845 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 12 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460744:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4200/0xb3f83dad1b7e0845 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 12 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460749:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4200/0xb3f83dad1b7e0845 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 12 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460754:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9856056d4200 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:2.0:1787826844.460758:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4200/0xb3f83dad1b7e0845 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 12 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460763:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4200/0xb3f83dad1b7e0845 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 12 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460769:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4200/0xb3f83dad1b7e0845 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 12 type: IBT flags: 0x20e400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460774:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1787826844.460778:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4200/0xb3f83dad1b7e0845 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 12 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460783:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4200/0xb3f83dad1b7e0845 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 12 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460794:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5400/0xb3f83dad1b7e084c lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 11 type: IBT flags: 0x0 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460799:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5400/0xb3f83dad1b7e084c lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 11 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460805:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5400/0xb3f83dad1b7e084c lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 11 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460811:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5400/0xb3f83dad1b7e084c lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 11 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460819:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5400/0xb3f83dad1b7e084c lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 11 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460825:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9856056d5400 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:2.0:1787826844.460828:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5400/0xb3f83dad1b7e084c lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 11 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460834:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5400/0xb3f83dad1b7e084c lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 11 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460840:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5400/0xb3f83dad1b7e084c lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 11 type: IBT flags: 0x20e400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460847:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1787826844.460852:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5400/0xb3f83dad1b7e084c lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 11 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460858:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5400/0xb3f83dad1b7e084c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 11 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460870:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4e00/0xb3f83dad1b7e0853 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 10 type: IBT flags: 0x0 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460877:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4e00/0xb3f83dad1b7e0853 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 10 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460882:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4e00/0xb3f83dad1b7e0853 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 10 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460888:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4e00/0xb3f83dad1b7e0853 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 10 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826844.460893:0:79202:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610d3c000 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:79202:1874669225570176:192.168.203.114@tcp:101:rm.0 00010000:00010000:2.0:1787826844.460896:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4e00/0xb3f83dad1b7e0853 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 10 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460903:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9856056d4e00 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:2.0:1787826844.460907:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4e00/0xb3f83dad1b7e0853 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 10 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.460909:0:79202:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e0899 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 11 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1b4f expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460915:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4e00/0xb3f83dad1b7e0853 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 11 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.460927:0:79202:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e0899 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 11 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1b4f expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460929:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4e00/0xb3f83dad1b7e0853 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 11 type: IBT flags: 0x20e400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460946:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787826844.460950:0:79202:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e0899 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 10 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1b4f expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.460963:0:79202:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e0899 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 10 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1b4f expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460964:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4e00/0xb3f83dad1b7e0853 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 11 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460981:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4e00/0xb3f83dad1b7e0853 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 10 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.460984:0:79202:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e0899 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 11 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1b4f expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.460994:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5e00/0xb3f83dad1b7e085a lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 9 type: IBT flags: 0x0 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.460995:0:79202:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e0899 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 10 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1b4f expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.461014:0:79202:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e0899 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 9 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1b4f expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.461035:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5e00/0xb3f83dad1b7e085a lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 9 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826844.461040:0:79202:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff985610d3c000 x1874669225570176/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826860 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00010000:00010000:2.0:1787826844.461047:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5e00/0xb3f83dad1b7e085a lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 9 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.461065:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5e00/0xb3f83dad1b7e085a lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 9 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00010000:1.0:1787826844.461085:0:79202:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e0899 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 10 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1b4f expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.461086:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5e00/0xb3f83dad1b7e085a lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 10 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.461102:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9856056d5e00 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:2.0:1787826844.461106:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5e00/0xb3f83dad1b7e085a lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 9 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.461109:0:79202:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8589934599/1 (0 0) 00010000:00010000:2.0:1787826844.461118:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5e00/0xb3f83dad1b7e085a lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 9 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00000080:00200000:1.0:1787826844.461119:0:79202:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00010000:00010000:2.0:1787826844.461131:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5e00/0xb3f83dad1b7e085a lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 9 type: IBT flags: 0x20e400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.461137:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1787826844.461141:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5e00/0xb3f83dad1b7e085a lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 9 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00000080:00200000:1.0:1787826844.461144:0:79202:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00010000:00010000:2.0:1787826844.461150:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5e00/0xb3f83dad1b7e085a lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 9 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00000080:00010000:1.0:1787826844.461153:0:79202:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e0899 00000080:00010000:1.0:1787826844.461161:0:79202:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826844.461166:0:79202:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0899 from it ffffab3ac0dd3b58 00010000:00010000:2.0:1787826844.461169:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5600/0xb3f83dad1b7e0861 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 8 type: IBT flags: 0x0 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.461171:0:79202:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e0899 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 9 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1b4f expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.461190:0:79202:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e0899 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 8 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1b4f expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826844.461191:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5600/0xb3f83dad1b7e0861 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 9 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.461208:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5600/0xb3f83dad1b7e0861 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 8 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.461214:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5600/0xb3f83dad1b7e0861 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 8 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00000080:00200000:1.0:1787826844.461220:0:79202:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00010000:00010000:2.0:1787826844.461221:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5600/0xb3f83dad1b7e0861 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 8 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.461230:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9856056d5600 already unused, calling callback (ffffffffc1d08990) 00000080:00200000:1.0:1787826844.461232:0:79202:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f33b.replay-single--6, flags=8 00010000:00010000:2.0:1787826844.461234:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5600/0xb3f83dad1b7e0861 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 8 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.461247:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5600/0xb3f83dad1b7e0861 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 8 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.461250:0:79202:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050400/0xb3f83dad1b7e049b lrc: 3/1,0 mode: PR/PR res: [0x200016b60:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a18fc expref: -99 pid: 79201 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1787826844.461255:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5600/0xb3f83dad1b7e0861 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 8 type: IBT flags: 0x20e400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00000080:00010000:1.0:1787826844.461266:0:79202:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e049b from it ffffab3ac0dd3af8 00010000:00010000:2.0:1787826844.461267:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787826844.461274:0:79202:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050400/0xb3f83dad1b7e049b lrc: 3/1,0 mode: PR/PR res: [0x200016b60:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a18fc expref: -99 pid: 79201 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1787826844.461275:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5600/0xb3f83dad1b7e0861 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 8 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.461286:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5600/0xb3f83dad1b7e0861 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 8 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.461301:0:79202:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050400/0xb3f83dad1b7e049b lrc: 2/0,0 mode: PR/PR res: [0x200016b60:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a18fc expref: -99 pid: 79201 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:2.0:1787826844.461308:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4600/0xb3f83dad1b7e0868 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 7 type: IBT flags: 0x0 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.461326:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4600/0xb3f83dad1b7e0868 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 7 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.461335:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4600/0xb3f83dad1b7e0868 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 7 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00000080:00200008:1.0:1787826844.461340:0:79202:0:(file.c:6313:ll_getattr_dentry()) START file f33b.replay-single--6:[0x200016b60:0x7:0x0](ffff98561158f448), request_mask 2047, flags 0, foreign 0 00010000:00010000:2.0:1787826844.461341:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4600/0xb3f83dad1b7e0868 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 7 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00000080:00200000:1.0:1787826844.461351:0:79202:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200016b60:0x7:0x0](ffff98561158f448),name=f33b.replay-single--6 00010000:00010000:2.0:1787826844.461353:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4600/0xb3f83dad1b7e0868 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 7 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.461360:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9856056d4600 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:2.0:1787826844.461363:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4600/0xb3f83dad1b7e0868 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 7 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.461373:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4600/0xb3f83dad1b7e0868 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 7 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00010000:1.0:1787826844.461378:0:79202:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200016b60:0x7:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826844.461380:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4600/0xb3f83dad1b7e0868 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 7 type: IBT flags: 0x20e400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.461390:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787826844.461391:0:79202:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050400/0xb3f83dad1b7e049b lrc: 2/1,0 mode: PR/PR res: [0x200016b60:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a18fc expref: -99 pid: 79201 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:2.0:1787826844.461394:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4600/0xb3f83dad1b7e0868 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 7 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.461409:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4600/0xb3f83dad1b7e0868 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 7 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.461412:0:79202:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050400/0xb3f83dad1b7e049b lrc: 2/1,0 mode: PR/PR res: [0x200016b60:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a18fc expref: -99 pid: 79201 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:2.0:1787826844.461421:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5000/0xb3f83dad1b7e086f lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 6 type: IBT flags: 0x0 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00000080:00200000:1.0:1787826844.461422:0:79202:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826844.461432:0:79202:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200016b60:0x7:0x0](ffff98561158f448) 00010000:00010000:2.0:1787826844.461434:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5000/0xb3f83dad1b7e086f lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 6 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00000080:00010000:1.0:1787826844.461438:0:79202:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200016b60:0x7:0x0](ffff98561158f448) for lock 0xb3f83dad1b7e049b 00010000:00010000:2.0:1787826844.461447:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5000/0xb3f83dad1b7e086f lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 6 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00000080:00010000:1.0:1787826844.461448:0:79202:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e049b from it ffffab3ac0dd3cb8 00010000:00010000:1.0:1787826844.461455:0:79202:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050400/0xb3f83dad1b7e049b lrc: 3/1,0 mode: PR/PR res: [0x200016b60:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a18fc expref: -99 pid: 79201 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:2.0:1787826844.461458:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5000/0xb3f83dad1b7e086f lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 6 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.461468:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5000/0xb3f83dad1b7e086f lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 6 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.461472:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9856056d5000 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:2.0:1787826844.461475:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5000/0xb3f83dad1b7e086f lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 6 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.461482:0:79202:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050400/0xb3f83dad1b7e049b lrc: 2/0,0 mode: PR/PR res: [0x200016b60:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a18fc expref: -99 pid: 79201 timeout: 0 lvb_type: 3 lru_score: 4 lru_type: 0 00010000:00010000:2.0:1787826844.461499:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5000/0xb3f83dad1b7e086f lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 6 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.461505:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5000/0xb3f83dad1b7e086f lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 6 type: IBT flags: 0x20e400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00000080:00200000:1.0:1787826844.461509:0:79202:0:(vvp_io.c:1885:vvp_io_init()) [0x200016b60:0x7:0x0] ignore/verify layout 0/1, layout version 0 restore needed 0 00010000:00010000:2.0:1787826844.461509:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1787826844.461515:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5000/0xb3f83dad1b7e086f lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 6 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00010000:1.0:1787826844.461518:0:79202:0:(glimpse.c:73:cl_glimpse_lock()) Glimpsing inode [0x200016b60:0x7:0x0] 00010000:00010000:2.0:1787826844.461524:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5000/0xb3f83dad1b7e086f lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 6 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787826844.461533:0:79202:0:(lustre_osc.h:166:osc_enq2ldlm_flags()) flags: a 00010000:00010000:2.0:1787826844.461535:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4c00/0xb3f83dad1b7e0876 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x0 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787826844.461537:0:79202:0:(osc_lock.c:1320:osc_lock_init()) ### lock ffff98560560c830, osc lock ffff985602f8f380, flags 0x400001002 00010000:00010000:2.0:1787826844.461545:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4c00/0xb3f83dad1b7e0876 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.461551:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4c00/0xb3f83dad1b7e0876 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.461553:0:79202:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0000-osc-ffff985602bc9800 lock: 00000000b37c3983/0xb3f83dad1b7e04fd lrc: 2/1,0 mode: PR/PR res: [0x280000401:0x605:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a19dc expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826844.461566:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4c00/0xb3f83dad1b7e0876 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.461583:0:79202:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 18446744073709551615) ns: lustre-OST0000-osc-ffff985602bc9800 lock: 00000000b37c3983/0xb3f83dad1b7e04fd lrc: 2/1,0 mode: PR/PR res: [0x280000401:0x605:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a19dc expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826844.461584:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4c00/0xb3f83dad1b7e0876 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.461603:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9856056d4c00 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:2.0:1787826844.461608:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4c00/0xb3f83dad1b7e0876 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 5 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787826844.461614:0:79202:0:(osc_lock.c:192:osc_lock_lvb_update()) ### acquired size 0, setting rss=0; kms=0, end=18446744073709551615 ns: lustre-OST0000-osc-ffff985602bc9800 lock: 00000000b37c3983/0xb3f83dad1b7e04fd lrc: 4/1,0 mode: PR/PR res: [0x280000401:0x605:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a19dc expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826844.461617:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4c00/0xb3f83dad1b7e0876 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 5 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.461634:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4c00/0xb3f83dad1b7e0876 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 5 type: IBT flags: 0x20e400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00200000:1.0:1787826844.461638:0:79202:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00010000:00010000:2.0:1787826844.461641:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787826844.461645:0:79202:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff985602bc9800 lock: 00000000b37c3983/0xb3f83dad1b7e04fd lrc: 5/1,0 mode: PR/PR res: [0x280000401:0x605:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x5aa6d504d91a19dc expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826844.461646:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4c00/0xb3f83dad1b7e0876 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 5 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.461662:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4c00/0xb3f83dad1b7e0876 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 5 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.461669:0:79202:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0000-osc-ffff985602bc9800 lock: 00000000b37c3983/0xb3f83dad1b7e04fd lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x605:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x5aa6d504d91a19dc expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1787826844.461675:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5c00/0xb3f83dad1b7e087d lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x0 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00200000:1.0:1787826844.461685:0:79202:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00010000:00010000:2.0:1787826844.461685:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5c00/0xb3f83dad1b7e087d lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00000080:00200000:1.0:1787826844.461725:0:79202:0:(file.c:1775:ll_merge_attr_nolock()) [0x200016b60:0x7:0x0] updating i_size 0 i_blocks 0 00010000:00010000:2.0:1787826844.461726:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5c00/0xb3f83dad1b7e087d lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00010000:1.0:1787826844.461734:0:79202:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff985605499638 (ffff985610fe17e8/0) at cl_lock_release():245 00010000:00010000:2.0:1787826844.461735:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5c00/0xb3f83dad1b7e087d lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00010000:1.0:1787826844.461742:0:79202:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff985605499638 (ffff985610fe17e8/0) at cl_lock_cancel():142 00010000:00010000:2.0:1787826844.461745:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5c00/0xb3f83dad1b7e087d lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00010000:1.0:1787826844.461747:0:79202:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff98560560c830 (ffff985610fe1488/1) at cl_lock_cancel():142 00010000:00010000:2.0:1787826844.461750:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9856056d5c00 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:2.0:1787826844.461753:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5c00/0xb3f83dad1b7e087d lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 4 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00010000:1.0:1787826844.461755:0:79202:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff985605499638 (ffff985610fe17e8/0) at cl_lock_fini():72 00010000:00010000:2.0:1787826844.461762:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5c00/0xb3f83dad1b7e087d lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 4 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.461766:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5c00/0xb3f83dad1b7e087d lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 4 type: IBT flags: 0x20e400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.461769:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1787826844.461771:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5c00/0xb3f83dad1b7e087d lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00010000:1.0:1787826844.461775:0:79202:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff98560560c830 (ffff985610fe17e8/1) at cl_lock_fini():72 00010000:00010000:2.0:1787826844.461777:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5c00/0xb3f83dad1b7e087d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00000080:00200000:1.0:1787826844.461789:0:79202:0:(vvp_io.c:304:vvp_io_fini()) [0x200016b60:0x7:0x0] ignore/verify layout 0/1, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00010000:00010000:2.0:1787826844.461790:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4a00/0xb3f83dad1b7e0884 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x0 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.461800:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4a00/0xb3f83dad1b7e0884 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.461804:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4a00/0xb3f83dad1b7e0884 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.461808:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4a00/0xb3f83dad1b7e0884 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.461811:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4a00/0xb3f83dad1b7e0884 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.461814:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9856056d4a00 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:2.0:1787826844.461815:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4a00/0xb3f83dad1b7e0884 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 3 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00000080:00200000:1.0:1787826844.461820:0:79202:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00010000:00010000:2.0:1787826844.461821:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4a00/0xb3f83dad1b7e0884 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 3 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.461828:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4a00/0xb3f83dad1b7e0884 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 3 type: IBT flags: 0x20e400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00010000:1.0:1787826844.461831:0:79202:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826844.461831:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1787826844.461834:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4a00/0xb3f83dad1b7e0884 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826844.461840:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4a00/0xb3f83dad1b7e0884 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.461843:0:79202:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e0899 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1b4f expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826844.461856:0:79202:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e0899 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1b4f expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000100:00100000:2.0:1787826844.461856:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669225569536:192.168.203.114@tcp:63:ll_sa.0 00000080:00200000:1.0:1787826844.461868:0:79202:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826844.461874:0:79202:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826844.461876:0:79202:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e0899 00000080:00010000:1.0:1787826844.461881:0:79202:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0899 from it ffffab3ac0dd3c10 00010000:00010000:1.0:1787826844.461885:0:79202:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e0899 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1b4f expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826844.461893:0:79202:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e0899 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1b4f expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826844.461902:0:79202:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787826844.461908:0:79202:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f33b.replay-single--6, flags=0 00000080:00200000:1.0:1787826844.461912:0:79202:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826844.461919:0:79202:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826844.461925:0:79202:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e0899 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1b4f expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826844.461934:0:79202:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e0899 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1b4f expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826844.461942:0:79202:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826844.461946:0:79202:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826844.461949:0:79202:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e0899 00000080:00010000:1.0:1787826844.461953:0:79202:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0899 from it ffffab3ac0dd3d68 00010000:00010000:1.0:1787826844.461957:0:79202:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e0899 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1b4f expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826844.461964:0:79202:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e0899 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1b4f expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787826844.461972:0:79202:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787826844.461978:0:79202:0:(namei.c:2418:ll_unlink()) VFS Op:name=f33b.replay-single--6, dir=[0x200000007:0x1:0x0](ffff98561158e3c8) 00010000:00010000:1.0:1787826844.462002:0:79202:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e0899 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x5aa6d504d91a1b4f expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787826844.462024:0:79202:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050400/0xb3f83dad1b7e049b lrc: 2/0,0 mode: PR/PR res: [0x200016b60:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x5aa6d504d91a18fc expref: -99 pid: 79201 timeout: 0 lvb_type: 3 lru_score: 4 lru_type: 0 00000080:00200000:1.0:1787826844.462035:0:79202:0:(vvp_object.c:117:vvp_conf_set()) [0x200016b60:0x7:0x0]: losing layout lock 00010000:00010000:1.0:1787826844.462061:0:79202:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050400/0xb3f83dad1b7e049b lrc: 1/0,0 mode: --/PR res: [0x200016b60:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x5aa6d504d91a18fc expref: -99 pid: 79201 timeout: 0 lvb_type: 3 lru_score: 4 lru_type: 0 00010000:00010000:1.0:1787826844.462082:0:79202:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e0899 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a1b4f expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787826844.462089:0:79202:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 2 locks packed 00010000:00010000:1.0:1787826844.462092:0:79202:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050400/0xb3f83dad1b7e049b lrc: 0/0,0 mode: --/PR res: [0x200016b60:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x5aa6d504d91a18fc expref: -99 pid: 79201 timeout: 0 lvb_type: 3 lru_score: 4 lru_type: 0 00010000:00010000:1.0:1787826844.462108:0:79202:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e0899 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a1b4f expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000020:00100000:1.0:1787826844.462127:0:79202:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787826844.462136:0:79202:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610d3c000 x1874669225570432/t0(0) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826844.462159:0:79202:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610d3c000 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:79202:1874669225570432:192.168.203.114@tcp:36:rm.0 00000100:00100000:1.0:1787826844.462223:0:79202:0:(client.c:2715:ptlrpc_set_wait()) set ffff98560957d240 going to sleep for 11 seconds 00000100:00100000:1.0:1787826844.464608:0:79202:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610d3c000 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:79202:1874669225570432:192.168.203.114@tcp:36:rm.0 00000080:00200000:1.0:1787826844.464649:0:79202:0:(vvp_io.c:1885:vvp_io_init()) [0x200016b60:0x7:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826844.464660:0:79202:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826844.464685:0:79202:0:(vvp_io.c:304:vvp_io_fini()) [0x200016b60:0x7:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826844.464720:0:79202:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200016b60:0x7:0x0](ffff98561158f448) 00000008:00010000:1.0:1787826844.464747:0:79202:0:(osc_object.c:202:osc_object_ast_clear()) ### update lock size 0 blocks 0 [cma]time: 0 0 0 by oinfo size 0 blocks 0 [cma]time 0 0 0 ns: lustre-OST0000-osc-ffff985602bc9800 lock: 00000000b37c3983/0xb3f83dad1b7e04fd lrc: 1/0,0 mode: PR/PR res: [0x280000401:0x605:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x5aa6d504d91a19dc expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826844.464838:0:79202:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826844.464855:0:79202:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826844.464913:0:79202:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e08a0 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 79202 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.464920:0:79202:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e08a0 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 79202 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.464928:0:79202:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e08a0 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 79202 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826844.464943:0:79202:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610db6940 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:79202:1874669225570560:192.168.203.114@tcp:101:rm.0 00000100:00100000:1.0:1787826844.465000:0:79202:0:(client.c:2715:ptlrpc_set_wait()) set ffff98560957d480 going to sleep for 11 seconds 00000100:00100000:1.0:1787826844.467383:0:79202:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610db6940 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:79202:1874669225570560:192.168.203.114@tcp:101:rm.0 00010000:00010000:1.0:1787826844.467404:0:79202:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e08a0 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1b79 expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.467414:0:79202:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e08a0 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1b79 expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.467424:0:79202:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e08a0 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1b79 expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.467433:0:79202:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e08a0 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1b79 expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.467443:0:79202:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e08a0 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1b79 expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.467450:0:79202:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e08a0 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1b79 expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.467457:0:79202:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e08a0 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1b79 expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826844.467468:0:79202:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff985610db6940 x1874669225570560/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826860 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826844.467479:0:79202:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e08a0 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1b79 expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.467489:0:79202:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:1.0:1787826844.467494:0:79202:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:1.0:1787826844.467516:0:79202:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:1.0:1787826844.467521:0:79202:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e08a0 00000080:00010000:1.0:1787826844.467526:0:79202:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826844.467529:0:79202:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e08a0 from it ffffab3ac0dd3b58 00010000:00010000:1.0:1787826844.467532:0:79202:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e08a0 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1b79 expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.467540:0:79202:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e08a0 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1b79 expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826844.467558:0:79202:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787826844.467567:0:79202:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f33b.replay-single--7, flags=8 00010000:00010000:1.0:1787826844.467581:0:79202:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e04a9 lrc: 3/1,0 mode: PR/PR res: [0x200016b60:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a192d expref: -99 pid: 79201 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00000080:00010000:1.0:1787826844.467588:0:79202:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e04a9 from it ffffab3ac0dd3af8 00010000:00010000:1.0:1787826844.467591:0:79202:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e04a9 lrc: 3/1,0 mode: PR/PR res: [0x200016b60:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a192d expref: -99 pid: 79201 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826844.467598:0:79202:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e04a9 lrc: 2/0,0 mode: PR/PR res: [0x200016b60:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a192d expref: -99 pid: 79201 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200008:1.0:1787826844.467616:0:79202:0:(file.c:6313:ll_getattr_dentry()) START file f33b.replay-single--7:[0x200016b60:0x8:0x0](ffff98561166a1c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:1.0:1787826844.467621:0:79202:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200016b60:0x8:0x0](ffff98561166a1c8),name=f33b.replay-single--7 00000002:00010000:1.0:1787826844.467634:0:79202:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200016b60:0x8:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826844.467641:0:79202:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e04a9 lrc: 2/1,0 mode: PR/PR res: [0x200016b60:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a192d expref: -99 pid: 79201 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787826844.467656:0:79202:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e04a9 lrc: 2/1,0 mode: PR/PR res: [0x200016b60:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a192d expref: -99 pid: 79201 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787826844.467663:0:79202:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826844.467667:0:79202:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200016b60:0x8:0x0](ffff98561166a1c8) 00000080:00010000:1.0:1787826844.467669:0:79202:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200016b60:0x8:0x0](ffff98561166a1c8) for lock 0xb3f83dad1b7e04a9 00000080:00010000:1.0:1787826844.467672:0:79202:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e04a9 from it ffffab3ac0dd3cb8 00010000:00010000:1.0:1787826844.467676:0:79202:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e04a9 lrc: 3/1,0 mode: PR/PR res: [0x200016b60:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a192d expref: -99 pid: 79201 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787826844.467683:0:79202:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e04a9 lrc: 2/0,0 mode: PR/PR res: [0x200016b60:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a192d expref: -99 pid: 79201 timeout: 0 lvb_type: 3 lru_score: 4 lru_type: 0 00000080:00200000:1.0:1787826844.467720:0:79202:0:(vvp_io.c:1885:vvp_io_init()) [0x200016b60:0x8:0x0] ignore/verify layout 0/1, layout version 0 restore needed 0 00000001:00010000:1.0:1787826844.467731:0:79202:0:(glimpse.c:73:cl_glimpse_lock()) Glimpsing inode [0x200016b60:0x8:0x0] 00000008:00010000:1.0:1787826844.467748:0:79202:0:(lustre_osc.h:166:osc_enq2ldlm_flags()) flags: a 00000008:00010000:1.0:1787826844.467751:0:79202:0:(osc_lock.c:1320:osc_lock_init()) ### lock ffff98560560c830, osc lock ffff985602f8f380, flags 0x400001002 00010000:00010000:1.0:1787826844.467766:0:79202:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0001-osc-ffff985602bc9800 lock: 000000008d58046f/0xb3f83dad1b7e0504 lrc: 2/1,0 mode: PR/PR res: [0x2c0000401:0x605:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a19d5 expref: -99 pid: 79203 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826844.467776:0:79202:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 18446744073709551615) ns: lustre-OST0001-osc-ffff985602bc9800 lock: 000000008d58046f/0xb3f83dad1b7e0504 lrc: 2/1,0 mode: PR/PR res: [0x2c0000401:0x605:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a19d5 expref: -99 pid: 79203 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000008:00010000:1.0:1787826844.467789:0:79202:0:(osc_lock.c:192:osc_lock_lvb_update()) ### acquired size 0, setting rss=0; kms=0, end=18446744073709551615 ns: lustre-OST0001-osc-ffff985602bc9800 lock: 000000008d58046f/0xb3f83dad1b7e0504 lrc: 4/1,0 mode: PR/PR res: [0x2c0000401:0x605:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a19d5 expref: -99 pid: 79203 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000020:00200000:1.0:1787826844.467805:0:79202:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00010000:00010000:1.0:1787826844.467809:0:79202:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0001-osc-ffff985602bc9800 lock: 000000008d58046f/0xb3f83dad1b7e0504 lrc: 5/1,0 mode: PR/PR res: [0x2c0000401:0x605:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x5aa6d504d91a19d5 expref: -99 pid: 79203 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826844.467817:0:79202:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0001-osc-ffff985602bc9800 lock: 000000008d58046f/0xb3f83dad1b7e0504 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x605:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x5aa6d504d91a19d5 expref: -99 pid: 79203 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000020:00200000:1.0:1787826844.467826:0:79202:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1787826844.467835:0:79202:0:(file.c:1775:ll_merge_attr_nolock()) [0x200016b60:0x8:0x0] updating i_size 0 i_blocks 0 00000020:00010000:1.0:1787826844.467838:0:79202:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff985605499638 (ffff985610fe17e8/0) at cl_lock_release():245 00000020:00010000:1.0:1787826844.467841:0:79202:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff985605499638 (ffff985610fe17e8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1787826844.467843:0:79202:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff98560560c830 (ffff985610fe1488/1) at cl_lock_cancel():142 00000020:00010000:1.0:1787826844.467847:0:79202:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff985605499638 (ffff985610fe17e8/0) at cl_lock_fini():72 00000020:00010000:1.0:1787826844.467850:0:79202:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff98560560c830 (ffff985610fe17e8/1) at cl_lock_fini():72 00000080:00200000:1.0:1787826844.467858:0:79202:0:(vvp_io.c:304:vvp_io_fini()) [0x200016b60:0x8:0x0] ignore/verify layout 0/1, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826844.467884:0:79202:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826844.467891:0:79202:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826844.467897:0:79202:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e08a0 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1b79 expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826844.467905:0:79202:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e08a0 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1b79 expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826844.467911:0:79202:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826844.467915:0:79202:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826844.467918:0:79202:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e08a0 00000080:00010000:1.0:1787826844.467921:0:79202:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e08a0 from it ffffab3ac0dd3c10 00010000:00010000:1.0:1787826844.467924:0:79202:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e08a0 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1b79 expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826844.467931:0:79202:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e08a0 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1b79 expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826844.467938:0:79202:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787826844.467944:0:79202:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f33b.replay-single--7, flags=0 00000080:00200000:1.0:1787826844.467948:0:79202:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826844.467967:0:79202:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826844.467972:0:79202:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e08a0 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1b79 expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826844.467979:0:79202:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e08a0 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1b79 expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826844.467985:0:79202:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826844.467988:0:79202:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826844.467991:0:79202:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e08a0 00000080:00010000:1.0:1787826844.467994:0:79202:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e08a0 from it ffffab3ac0dd3d68 00010000:00010000:1.0:1787826844.467997:0:79202:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e08a0 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1b79 expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826844.468004:0:79202:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e08a0 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1b79 expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787826844.468015:0:79202:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787826844.468024:0:79202:0:(namei.c:2418:ll_unlink()) VFS Op:name=f33b.replay-single--7, dir=[0x200000007:0x1:0x0](ffff98561158e3c8) 00010000:00010000:1.0:1787826844.468043:0:79202:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e08a0 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x5aa6d504d91a1b79 expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787826844.468064:0:79202:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e04a9 lrc: 2/0,0 mode: PR/PR res: [0x200016b60:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x5aa6d504d91a192d expref: -99 pid: 79201 timeout: 0 lvb_type: 3 lru_score: 4 lru_type: 0 00000080:00200000:1.0:1787826844.468085:0:79202:0:(vvp_object.c:117:vvp_conf_set()) [0x200016b60:0x8:0x0]: losing layout lock 00010000:00010000:1.0:1787826844.468115:0:79202:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e04a9 lrc: 1/0,0 mode: --/PR res: [0x200016b60:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x5aa6d504d91a192d expref: -99 pid: 79201 timeout: 0 lvb_type: 3 lru_score: 4 lru_type: 0 00010000:00010000:1.0:1787826844.468123:0:79202:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e08a0 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a1b79 expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787826844.468128:0:79202:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 2 locks packed 00010000:00010000:1.0:1787826844.468131:0:79202:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e04a9 lrc: 0/0,0 mode: --/PR res: [0x200016b60:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x5aa6d504d91a192d expref: -99 pid: 79201 timeout: 0 lvb_type: 3 lru_score: 4 lru_type: 0 00010000:00010000:1.0:1787826844.468146:0:79202:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051a00/0xb3f83dad1b7e08a0 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a1b79 expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000020:00100000:1.0:1787826844.468162:0:79202:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787826844.468168:0:79202:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610db6940 x1874669225570816/t0(0) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826844.468186:0:79202:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610db6940 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:79202:1874669225570816:192.168.203.114@tcp:36:rm.0 00000100:00100000:1.0:1787826844.468282:0:79202:0:(client.c:2715:ptlrpc_set_wait()) set ffff98560957d480 going to sleep for 11 seconds 00010000:00010000:1.0:1787826844.468794:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826844.470950:0:79202:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610db6940 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:79202:1874669225570816:192.168.203.114@tcp:36:rm.0 00000080:00200000:1.0:1787826844.470999:0:79202:0:(vvp_io.c:1885:vvp_io_init()) [0x200016b60:0x8:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826844.471012:0:79202:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826844.471285:0:79202:0:(vvp_io.c:304:vvp_io_fini()) [0x200016b60:0x8:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826844.471294:0:79202:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200016b60:0x8:0x0](ffff98561166a1c8) 00000008:00010000:1.0:1787826844.471324:0:79202:0:(osc_object.c:202:osc_object_ast_clear()) ### update lock size 0 blocks 0 [cma]time: 0 0 0 by oinfo size 0 blocks 0 [cma]time 0 0 0 ns: lustre-OST0001-osc-ffff985602bc9800 lock: 000000008d58046f/0xb3f83dad1b7e0504 lrc: 1/0,0 mode: PR/PR res: [0x2c0000401:0x605:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x5aa6d504d91a19d5 expref: -99 pid: 79203 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826844.471395:0:79202:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826844.471415:0:79202:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826844.471459:0:79202:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e08a7 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 79202 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.471478:0:79202:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e08a7 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 79202 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.471487:0:79202:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e08a7 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 79202 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826844.471505:0:79202:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610db6580 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:79202:1874669225570944:192.168.203.114@tcp:101:rm.0 00000100:00100000:1.0:1787826844.471562:0:79202:0:(client.c:2715:ptlrpc_set_wait()) set ffff98560957d480 going to sleep for 11 seconds 00000100:00100000:1.0:1787826844.472979:0:79202:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610db6580 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:79202:1874669225570944:192.168.203.114@tcp:101:rm.0 00010000:00010000:1.0:1787826844.472991:0:79202:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e08a7 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1ba3 expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.473002:0:79202:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e08a7 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1ba3 expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.473009:0:79202:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e08a7 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1ba3 expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.473016:0:79202:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e08a7 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1ba3 expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.473023:0:79202:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e08a7 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1ba3 expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.473028:0:79202:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e08a7 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1ba3 expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.473033:0:79202:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e08a7 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1ba3 expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826844.473049:0:79202:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff985610db6580 x1874669225570944/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826860 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826844.473059:0:79202:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e08a7 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1ba3 expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.473075:0:79202:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:1.0:1787826844.473081:0:79202:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:1.0:1787826844.473101:0:79202:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:1.0:1787826844.473108:0:79202:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e08a7 00000080:00010000:1.0:1787826844.473113:0:79202:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826844.473116:0:79202:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e08a7 from it ffffab3ac0dd3b58 00010000:00010000:1.0:1787826844.473119:0:79202:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e08a7 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1ba3 expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.473129:0:79202:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e08a7 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1ba3 expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826844.473148:0:79202:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787826844.473156:0:79202:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f33b.replay-single--8, flags=8 00010000:00010000:1.0:1787826844.473169:0:79202:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1a00/0xb3f83dad1b7e04b7 lrc: 3/1,0 mode: PR/PR res: [0x200016b60:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a195e expref: -99 pid: 79201 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00000080:00010000:1.0:1787826844.473177:0:79202:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e04b7 from it ffffab3ac0dd3af8 00010000:00010000:1.0:1787826844.473180:0:79202:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1a00/0xb3f83dad1b7e04b7 lrc: 3/1,0 mode: PR/PR res: [0x200016b60:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a195e expref: -99 pid: 79201 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826844.473186:0:79202:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1a00/0xb3f83dad1b7e04b7 lrc: 2/0,0 mode: PR/PR res: [0x200016b60:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a195e expref: -99 pid: 79201 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200008:1.0:1787826844.473199:0:79202:0:(file.c:6313:ll_getattr_dentry()) START file f33b.replay-single--8:[0x200016b60:0x9:0x0](ffff98561166ec08), request_mask 2047, flags 0, foreign 0 00000080:00200000:1.0:1787826844.473202:0:79202:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200016b60:0x9:0x0](ffff98561166ec08),name=f33b.replay-single--8 00000002:00010000:1.0:1787826844.473212:0:79202:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200016b60:0x9:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826844.473218:0:79202:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1a00/0xb3f83dad1b7e04b7 lrc: 2/1,0 mode: PR/PR res: [0x200016b60:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a195e expref: -99 pid: 79201 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787826844.473224:0:79202:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1a00/0xb3f83dad1b7e04b7 lrc: 2/1,0 mode: PR/PR res: [0x200016b60:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a195e expref: -99 pid: 79201 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787826844.473228:0:79202:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826844.473231:0:79202:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200016b60:0x9:0x0](ffff98561166ec08) 00000080:00010000:1.0:1787826844.473233:0:79202:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200016b60:0x9:0x0](ffff98561166ec08) for lock 0xb3f83dad1b7e04b7 00000080:00010000:1.0:1787826844.473235:0:79202:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e04b7 from it ffffab3ac0dd3cb8 00010000:00010000:1.0:1787826844.473237:0:79202:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1a00/0xb3f83dad1b7e04b7 lrc: 3/1,0 mode: PR/PR res: [0x200016b60:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a195e expref: -99 pid: 79201 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787826844.473241:0:79202:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1a00/0xb3f83dad1b7e04b7 lrc: 2/0,0 mode: PR/PR res: [0x200016b60:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a195e expref: -99 pid: 79201 timeout: 0 lvb_type: 3 lru_score: 4 lru_type: 0 00000080:00200000:1.0:1787826844.473252:0:79202:0:(vvp_io.c:1885:vvp_io_init()) [0x200016b60:0x9:0x0] ignore/verify layout 0/1, layout version 0 restore needed 0 00000001:00010000:1.0:1787826844.473257:0:79202:0:(glimpse.c:73:cl_glimpse_lock()) Glimpsing inode [0x200016b60:0x9:0x0] 00000008:00010000:1.0:1787826844.473268:0:79202:0:(lustre_osc.h:166:osc_enq2ldlm_flags()) flags: a 00000008:00010000:1.0:1787826844.473271:0:79202:0:(osc_lock.c:1320:osc_lock_init()) ### lock ffff98560560c9b0, osc lock ffff985602f8f380, flags 0x400001002 00010000:00010000:1.0:1787826844.473282:0:79202:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0000-osc-ffff985602bc9800 lock: 000000005fe2ea3b/0xb3f83dad1b7e050b lrc: 2/1,0 mode: PR/PR res: [0x280000401:0x606:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a19e3 expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826844.473288:0:79202:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 18446744073709551615) ns: lustre-OST0000-osc-ffff985602bc9800 lock: 000000005fe2ea3b/0xb3f83dad1b7e050b lrc: 2/1,0 mode: PR/PR res: [0x280000401:0x606:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a19e3 expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000008:00010000:1.0:1787826844.473298:0:79202:0:(osc_lock.c:192:osc_lock_lvb_update()) ### acquired size 0, setting rss=0; kms=0, end=18446744073709551615 ns: lustre-OST0000-osc-ffff985602bc9800 lock: 000000005fe2ea3b/0xb3f83dad1b7e050b lrc: 4/1,0 mode: PR/PR res: [0x280000401:0x606:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a19e3 expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000020:00200000:1.0:1787826844.473306:0:79202:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00010000:00010000:1.0:1787826844.473308:0:79202:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff985602bc9800 lock: 000000005fe2ea3b/0xb3f83dad1b7e050b lrc: 5/1,0 mode: PR/PR res: [0x280000401:0x606:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x5aa6d504d91a19e3 expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826844.473317:0:79202:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0000-osc-ffff985602bc9800 lock: 000000005fe2ea3b/0xb3f83dad1b7e050b lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x606:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x5aa6d504d91a19e3 expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000020:00200000:1.0:1787826844.473323:0:79202:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1787826844.473330:0:79202:0:(file.c:1775:ll_merge_attr_nolock()) [0x200016b60:0x9:0x0] updating i_size 0 i_blocks 0 00000020:00010000:1.0:1787826844.473333:0:79202:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff985605499638 (ffff985610fe17e8/0) at cl_lock_release():245 00000020:00010000:1.0:1787826844.473335:0:79202:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff985605499638 (ffff985610fe17e8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1787826844.473336:0:79202:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff98560560c9b0 (ffff985610fe1488/1) at cl_lock_cancel():142 00000020:00010000:1.0:1787826844.473339:0:79202:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff985605499638 (ffff985610fe17e8/0) at cl_lock_fini():72 00000020:00010000:1.0:1787826844.473341:0:79202:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff98560560c9b0 (ffff985610fe17e8/1) at cl_lock_fini():72 00000080:00200000:1.0:1787826844.473347:0:79202:0:(vvp_io.c:304:vvp_io_fini()) [0x200016b60:0x9:0x0] ignore/verify layout 0/1, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826844.473363:0:79202:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826844.473368:0:79202:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826844.473372:0:79202:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e08a7 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1ba3 expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826844.473377:0:79202:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e08a7 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1ba3 expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826844.473382:0:79202:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826844.473384:0:79202:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826844.473385:0:79202:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e08a7 00000080:00010000:1.0:1787826844.473388:0:79202:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e08a7 from it ffffab3ac0dd3c10 00010000:00010000:1.0:1787826844.473390:0:79202:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e08a7 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1ba3 expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826844.473398:0:79202:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e08a7 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1ba3 expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826844.473403:0:79202:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787826844.473407:0:79202:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f33b.replay-single--8, flags=0 00000080:00200000:1.0:1787826844.473410:0:79202:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826844.473415:0:79202:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826844.473421:0:79202:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e08a7 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1ba3 expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826844.473428:0:79202:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e08a7 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1ba3 expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826844.473434:0:79202:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826844.473438:0:79202:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826844.473441:0:79202:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e08a7 00000080:00010000:1.0:1787826844.473444:0:79202:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e08a7 from it ffffab3ac0dd3d68 00010000:00010000:1.0:1787826844.473447:0:79202:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e08a7 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1ba3 expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826844.473454:0:79202:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e08a7 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1ba3 expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787826844.473461:0:79202:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787826844.473475:0:79202:0:(namei.c:2418:ll_unlink()) VFS Op:name=f33b.replay-single--8, dir=[0x200000007:0x1:0x0](ffff98561158e3c8) 00010000:00010000:1.0:1787826844.473498:0:79202:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e08a7 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x5aa6d504d91a1ba3 expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787826844.473520:0:79202:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1a00/0xb3f83dad1b7e04b7 lrc: 2/0,0 mode: PR/PR res: [0x200016b60:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x5aa6d504d91a195e expref: -99 pid: 79201 timeout: 0 lvb_type: 3 lru_score: 4 lru_type: 0 00000080:00200000:1.0:1787826844.473530:0:79202:0:(vvp_object.c:117:vvp_conf_set()) [0x200016b60:0x9:0x0]: losing layout lock 00010000:00010000:1.0:1787826844.473556:0:79202:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1a00/0xb3f83dad1b7e04b7 lrc: 1/0,0 mode: --/PR res: [0x200016b60:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x5aa6d504d91a195e expref: -99 pid: 79201 timeout: 0 lvb_type: 3 lru_score: 4 lru_type: 0 00010000:00010000:1.0:1787826844.473564:0:79202:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e08a7 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a1ba3 expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787826844.473570:0:79202:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 2 locks packed 00010000:00010000:1.0:1787826844.473572:0:79202:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1a00/0xb3f83dad1b7e04b7 lrc: 0/0,0 mode: --/PR res: [0x200016b60:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x5aa6d504d91a195e expref: -99 pid: 79201 timeout: 0 lvb_type: 3 lru_score: 4 lru_type: 0 00010000:00010000:1.0:1787826844.473586:0:79202:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050c00/0xb3f83dad1b7e08a7 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a1ba3 expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000020:00100000:1.0:1787826844.473603:0:79202:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787826844.473610:0:79202:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610db6580 x1874669225571200/t0(0) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826844.473623:0:79202:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610db6580 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:79202:1874669225571200:192.168.203.114@tcp:36:rm.0 00000100:00100000:1.0:1787826844.473658:0:79202:0:(client.c:2715:ptlrpc_set_wait()) set ffff98560957d480 going to sleep for 11 seconds 00000100:00100000:1.0:1787826844.476743:0:79202:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610db6580 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:79202:1874669225571200:192.168.203.114@tcp:36:rm.0 00000080:00200000:1.0:1787826844.476791:0:79202:0:(vvp_io.c:1885:vvp_io_init()) [0x200016b60:0x9:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826844.476816:0:79202:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826844.476842:0:79202:0:(vvp_io.c:304:vvp_io_fini()) [0x200016b60:0x9:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826844.476850:0:79202:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200016b60:0x9:0x0](ffff98561166ec08) 00000008:00010000:1.0:1787826844.476881:0:79202:0:(osc_object.c:202:osc_object_ast_clear()) ### update lock size 0 blocks 0 [cma]time: 0 0 0 by oinfo size 0 blocks 0 [cma]time 0 0 0 ns: lustre-OST0000-osc-ffff985602bc9800 lock: 000000005fe2ea3b/0xb3f83dad1b7e050b lrc: 1/0,0 mode: PR/PR res: [0x280000401:0x606:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x5aa6d504d91a19e3 expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826844.476951:0:79202:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826844.476966:0:79202:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826844.477004:0:79202:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050400/0xb3f83dad1b7e08ae lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 79202 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.477010:0:79202:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050400/0xb3f83dad1b7e08ae lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 79202 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.477016:0:79202:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050400/0xb3f83dad1b7e08ae lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 79202 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826844.477029:0:79202:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610db4f00 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:79202:1874669225571328:192.168.203.114@tcp:101:rm.0 00000100:00100000:1.0:1787826844.477102:0:79202:0:(client.c:2715:ptlrpc_set_wait()) set ffff98560957d480 going to sleep for 11 seconds 00000100:00100000:1.0:1787826844.478753:0:79202:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610db4f00 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:79202:1874669225571328:192.168.203.114@tcp:101:rm.0 00010000:00010000:1.0:1787826844.478769:0:79202:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050400/0xb3f83dad1b7e08ae lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1bcd expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.478779:0:79202:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050400/0xb3f83dad1b7e08ae lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1bcd expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.478790:0:79202:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050400/0xb3f83dad1b7e08ae lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1bcd expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.478802:0:79202:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050400/0xb3f83dad1b7e08ae lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1bcd expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.478823:0:79202:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050400/0xb3f83dad1b7e08ae lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1bcd expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.478832:0:79202:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050400/0xb3f83dad1b7e08ae lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1bcd expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.478840:0:79202:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050400/0xb3f83dad1b7e08ae lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1bcd expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826844.478852:0:79202:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff985610db4f00 x1874669225571328/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826860 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826844.478865:0:79202:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050400/0xb3f83dad1b7e08ae lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1bcd expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.478876:0:79202:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:1.0:1787826844.478881:0:79202:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:1.0:1787826844.478906:0:79202:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:1.0:1787826844.478912:0:79202:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e08ae 00000080:00010000:1.0:1787826844.478916:0:79202:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826844.478918:0:79202:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e08ae from it ffffab3ac0dd3b58 00010000:00010000:1.0:1787826844.478921:0:79202:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050400/0xb3f83dad1b7e08ae lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1bcd expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826844.478927:0:79202:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050400/0xb3f83dad1b7e08ae lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1bcd expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826844.478945:0:79202:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787826844.478952:0:79202:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f33b.replay-single--9, flags=8 00010000:00010000:1.0:1787826844.478963:0:79202:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0a00/0xb3f83dad1b7e04c5 lrc: 3/1,0 mode: PR/PR res: [0x200016b60:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a198f expref: -99 pid: 79201 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00000080:00010000:1.0:1787826844.478968:0:79202:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e04c5 from it ffffab3ac0dd3af8 00010000:00010000:1.0:1787826844.478977:0:79202:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0a00/0xb3f83dad1b7e04c5 lrc: 3/1,0 mode: PR/PR res: [0x200016b60:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a198f expref: -99 pid: 79201 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826844.478988:0:79202:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0a00/0xb3f83dad1b7e04c5 lrc: 2/0,0 mode: PR/PR res: [0x200016b60:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a198f expref: -99 pid: 79201 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200008:1.0:1787826844.479007:0:79202:0:(file.c:6313:ll_getattr_dentry()) START file f33b.replay-single--9:[0x200016b60:0xa:0x0](ffff98561166aa08), request_mask 2047, flags 0, foreign 0 00000080:00200000:1.0:1787826844.479013:0:79202:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200016b60:0xa:0x0](ffff98561166aa08),name=f33b.replay-single--9 00000002:00010000:1.0:1787826844.479032:0:79202:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200016b60:0xa:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826844.479041:0:79202:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0a00/0xb3f83dad1b7e04c5 lrc: 2/1,0 mode: PR/PR res: [0x200016b60:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a198f expref: -99 pid: 79201 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787826844.479050:0:79202:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0a00/0xb3f83dad1b7e04c5 lrc: 2/1,0 mode: PR/PR res: [0x200016b60:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a198f expref: -99 pid: 79201 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787826844.479057:0:79202:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826844.479061:0:79202:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200016b60:0xa:0x0](ffff98561166aa08) 00000080:00010000:1.0:1787826844.479064:0:79202:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200016b60:0xa:0x0](ffff98561166aa08) for lock 0xb3f83dad1b7e04c5 00000080:00010000:1.0:1787826844.479078:0:79202:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e04c5 from it ffffab3ac0dd3cb8 00010000:00010000:1.0:1787826844.479081:0:79202:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0a00/0xb3f83dad1b7e04c5 lrc: 3/1,0 mode: PR/PR res: [0x200016b60:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a198f expref: -99 pid: 79201 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787826844.479090:0:79202:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0a00/0xb3f83dad1b7e04c5 lrc: 2/0,0 mode: PR/PR res: [0x200016b60:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a198f expref: -99 pid: 79201 timeout: 0 lvb_type: 3 lru_score: 4 lru_type: 0 00000080:00200000:1.0:1787826844.479107:0:79202:0:(vvp_io.c:1885:vvp_io_init()) [0x200016b60:0xa:0x0] ignore/verify layout 0/1, layout version 0 restore needed 0 00000001:00010000:1.0:1787826844.479115:0:79202:0:(glimpse.c:73:cl_glimpse_lock()) Glimpsing inode [0x200016b60:0xa:0x0] 00000008:00010000:1.0:1787826844.479130:0:79202:0:(lustre_osc.h:166:osc_enq2ldlm_flags()) flags: a 00000008:00010000:1.0:1787826844.479134:0:79202:0:(osc_lock.c:1320:osc_lock_init()) ### lock ffff98560560c9b0, osc lock ffff985602f8f380, flags 0x400001002 00010000:00010000:1.0:1787826844.479149:0:79202:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0001-osc-ffff985602bc9800 lock: 000000009edc46ab/0xb3f83dad1b7e053c lrc: 2/1,0 mode: PR/PR res: [0x2c0000401:0x606:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a1a14 expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826844.479157:0:79202:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 18446744073709551615) ns: lustre-OST0001-osc-ffff985602bc9800 lock: 000000009edc46ab/0xb3f83dad1b7e053c lrc: 2/1,0 mode: PR/PR res: [0x2c0000401:0x606:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a1a14 expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000008:00010000:1.0:1787826844.479168:0:79202:0:(osc_lock.c:192:osc_lock_lvb_update()) ### acquired size 0, setting rss=0; kms=0, end=18446744073709551615 ns: lustre-OST0001-osc-ffff985602bc9800 lock: 000000009edc46ab/0xb3f83dad1b7e053c lrc: 4/1,0 mode: PR/PR res: [0x2c0000401:0x606:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a1a14 expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000020:00200000:1.0:1787826844.479176:0:79202:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00010000:00010000:1.0:1787826844.479178:0:79202:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0001-osc-ffff985602bc9800 lock: 000000009edc46ab/0xb3f83dad1b7e053c lrc: 5/1,0 mode: PR/PR res: [0x2c0000401:0x606:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x5aa6d504d91a1a14 expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826844.479182:0:79202:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0001-osc-ffff985602bc9800 lock: 000000009edc46ab/0xb3f83dad1b7e053c lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x606:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x5aa6d504d91a1a14 expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000020:00200000:1.0:1787826844.479189:0:79202:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1787826844.479198:0:79202:0:(file.c:1775:ll_merge_attr_nolock()) [0x200016b60:0xa:0x0] updating i_size 0 i_blocks 0 00000020:00010000:1.0:1787826844.479202:0:79202:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff985605499638 (ffff985610fe17e8/0) at cl_lock_release():245 00000020:00010000:1.0:1787826844.479204:0:79202:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff985605499638 (ffff985610fe17e8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1787826844.479206:0:79202:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff98560560c9b0 (ffff985610fe1488/1) at cl_lock_cancel():142 00000020:00010000:1.0:1787826844.479210:0:79202:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff985605499638 (ffff985610fe17e8/0) at cl_lock_fini():72 00000020:00010000:1.0:1787826844.479212:0:79202:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff98560560c9b0 (ffff985610fe17e8/1) at cl_lock_fini():72 00000080:00200000:1.0:1787826844.479220:0:79202:0:(vvp_io.c:304:vvp_io_fini()) [0x200016b60:0xa:0x0] ignore/verify layout 0/1, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826844.479245:0:79202:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826844.479253:0:79202:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826844.479260:0:79202:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050400/0xb3f83dad1b7e08ae lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1bcd expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826844.479267:0:79202:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050400/0xb3f83dad1b7e08ae lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1bcd expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826844.479272:0:79202:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826844.479276:0:79202:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826844.479278:0:79202:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e08ae 00000080:00010000:1.0:1787826844.479282:0:79202:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e08ae from it ffffab3ac0dd3c10 00010000:00010000:1.0:1787826844.479285:0:79202:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050400/0xb3f83dad1b7e08ae lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1bcd expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826844.479291:0:79202:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050400/0xb3f83dad1b7e08ae lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1bcd expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826844.479299:0:79202:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787826844.479305:0:79202:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f33b.replay-single--9, flags=0 00000080:00200000:1.0:1787826844.479309:0:79202:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826844.479315:0:79202:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826844.479321:0:79202:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050400/0xb3f83dad1b7e08ae lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1bcd expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826844.479327:0:79202:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050400/0xb3f83dad1b7e08ae lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1bcd expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826844.479333:0:79202:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826844.479337:0:79202:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826844.479339:0:79202:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e08ae 00000080:00010000:1.0:1787826844.479342:0:79202:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e08ae from it ffffab3ac0dd3d68 00010000:00010000:1.0:1787826844.479344:0:79202:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050400/0xb3f83dad1b7e08ae lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1bcd expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826844.479350:0:79202:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050400/0xb3f83dad1b7e08ae lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1bcd expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787826844.479357:0:79202:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787826844.479361:0:79202:0:(namei.c:2418:ll_unlink()) VFS Op:name=f33b.replay-single--9, dir=[0x200000007:0x1:0x0](ffff98561158e3c8) 00010000:00010000:1.0:1787826844.479385:0:79202:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050400/0xb3f83dad1b7e08ae lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x5aa6d504d91a1bcd expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787826844.479407:0:79202:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0a00/0xb3f83dad1b7e04c5 lrc: 2/0,0 mode: PR/PR res: [0x200016b60:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x5aa6d504d91a198f expref: -99 pid: 79201 timeout: 0 lvb_type: 3 lru_score: 4 lru_type: 0 00000080:00200000:1.0:1787826844.479429:0:79202:0:(vvp_object.c:117:vvp_conf_set()) [0x200016b60:0xa:0x0]: losing layout lock 00010000:00010000:1.0:1787826844.479455:0:79202:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0a00/0xb3f83dad1b7e04c5 lrc: 1/0,0 mode: --/PR res: [0x200016b60:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x5aa6d504d91a198f expref: -99 pid: 79201 timeout: 0 lvb_type: 3 lru_score: 4 lru_type: 0 00010000:00010000:1.0:1787826844.479463:0:79202:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050400/0xb3f83dad1b7e08ae lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a1bcd expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787826844.479469:0:79202:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 2 locks packed 00010000:00010000:1.0:1787826844.479471:0:79202:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0a00/0xb3f83dad1b7e04c5 lrc: 0/0,0 mode: --/PR res: [0x200016b60:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x5aa6d504d91a198f expref: -99 pid: 79201 timeout: 0 lvb_type: 3 lru_score: 4 lru_type: 0 00010000:00010000:1.0:1787826844.479489:0:79202:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050400/0xb3f83dad1b7e08ae lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a1bcd expref: -99 pid: 79202 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000020:00100000:1.0:1787826844.479509:0:79202:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787826844.479515:0:79202:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610db4f00 x1874669225571584/t0(0) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826844.479542:0:79202:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610db4f00 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:79202:1874669225571584:192.168.203.114@tcp:36:rm.0 00000100:00100000:1.0:1787826844.479594:0:79202:0:(client.c:2715:ptlrpc_set_wait()) set ffff98560957d480 going to sleep for 11 seconds 00000100:00100000:1.0:1787826844.483846:0:79202:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610db4f00 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:79202:1874669225571584:192.168.203.114@tcp:36:rm.0 00000080:00200000:1.0:1787826844.483896:0:79202:0:(vvp_io.c:1885:vvp_io_init()) [0x200016b60:0xa:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826844.483910:0:79202:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826844.483937:0:79202:0:(vvp_io.c:304:vvp_io_fini()) [0x200016b60:0xa:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826844.483944:0:79202:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200016b60:0xa:0x0](ffff98561166aa08) 00000008:00010000:1.0:1787826844.483974:0:79202:0:(osc_object.c:202:osc_object_ast_clear()) ### update lock size 0 blocks 0 [cma]time: 0 0 0 by oinfo size 0 blocks 0 [cma]time 0 0 0 ns: lustre-OST0001-osc-ffff985602bc9800 lock: 000000009edc46ab/0xb3f83dad1b7e053c lrc: 1/0,0 mode: PR/PR res: [0x2c0000401:0x606:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x5aa6d504d91a1a14 expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1787826846.518127:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:2.0:1787826846.754518:0:79767:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == replay-single test 34: abort recovery before client does replay (test mds_cleanup_orphans) ========================================================== 06:34:06 (1787826846) 00000080:00200000:1.0:1787826846.789429:0:79773:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826846.789460:0:79773:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826846.789520:0:79773:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e08b5 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 79773 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826846.789535:0:79773:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e08b5 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 79773 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826846.789545:0:79773:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e08b5 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 79773 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826846.789569:0:79773:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610db43c0 pname:cluuid:pid:xid:nid:opc:job multiop:7b4e8535-9db7-485d-abf6-bda966b63320:79773:1874669225571712:192.168.203.114@tcp:101:multiop.0 00000100:00100000:1.0:1787826846.789653:0:79773:0:(client.c:2715:ptlrpc_set_wait()) set ffff985610f13480 going to sleep for 11 seconds 00000100:00100000:1.0:1787826846.791540:0:79773:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610db43c0 pname:cluuid:pid:xid:nid:opc:job multiop:7b4e8535-9db7-485d-abf6-bda966b63320:79773:1874669225571712:192.168.203.114@tcp:101:multiop.0 00010000:00010000:1.0:1787826846.791561:0:79773:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e08b5 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1bf7 expref: -99 pid: 79773 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826846.791573:0:79773:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e08b5 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1bf7 expref: -99 pid: 79773 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826846.791586:0:79773:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e08b5 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1bf7 expref: -99 pid: 79773 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826846.791597:0:79773:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e08b5 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1bf7 expref: -99 pid: 79773 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826846.791609:0:79773:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e08b5 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1bf7 expref: -99 pid: 79773 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826846.791617:0:79773:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e08b5 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1bf7 expref: -99 pid: 79773 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826846.791631:0:79773:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e08b5 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1bf7 expref: -99 pid: 79773 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826846.791644:0:79773:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff985610db43c0 x1874669225571712/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826862 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'multiop.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826846.791658:0:79773:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e08b5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1bf7 expref: -99 pid: 79773 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826846.791671:0:79773:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:1.0:1787826846.791678:0:79773:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:1.0:1787826846.791724:0:79773:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:1.0:1787826846.791730:0:79773:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e08b5 00000080:00010000:1.0:1787826846.791736:0:79773:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826846.791739:0:79773:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e08b5 from it ffffab3ac0c97bf8 00010000:00010000:1.0:1787826846.791743:0:79773:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e08b5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1bf7 expref: -99 pid: 79773 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826846.791752:0:79773:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e08b5 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1bf7 expref: -99 pid: 79773 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826846.791774:0:79773:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787826846.791790:0:79773:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826846.791803:0:79773:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826846.791816:0:79773:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e08b5 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1bf7 expref: -99 pid: 79773 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826846.791832:0:79773:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e08b5 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1bf7 expref: -99 pid: 79773 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826846.791839:0:79773:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826846.791844:0:79773:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826846.791846:0:79773:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e08b5 00000080:00010000:1.0:1787826846.791850:0:79773:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e08b5 from it ffffab3ac0c97b38 00010000:00010000:1.0:1787826846.791853:0:79773:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e08b5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1bf7 expref: -99 pid: 79773 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826846.791861:0:79773:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e08b5 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1bf7 expref: -99 pid: 79773 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826846.791870:0:79773:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787826846.791875:0:79773:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f34.replay-single, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), file ffff985614f86f00, open_flags 8042, mode 81a4 opened 0 00000080:00200000:1.0:1787826846.791884:0:79773:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f34.replay-single, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=open|creat 00000002:00010000:1.0:1787826846.791915:0:79773:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f34.replay-single,[0x200016b60:0xb:0x0]) in obj [0x200000007:0x1:0x0], intent: open|creat flags 0100103 00010000:00010000:1.0:1787826846.791926:0:79773:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e08b5 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x5aa6d504d91a1bf7 expref: -99 pid: 79773 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826846.791965:0:79773:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e08b5 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a1bf7 expref: -99 pid: 79773 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826846.791973:0:79773:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826846.791976:0:79773:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e08b5 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a1bf7 expref: -99 pid: 79773 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826846.792012:0:79773:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e08bc lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 79773 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826846.792020:0:79773:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e08bc lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 79773 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:1.0:1787826846.792040:0:79773:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:1.0:1787826846.792047:0:79773:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610db43c0 x1874669225571968/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:1.0:1787826846.792067:0:79773:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e08bc lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 79773 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826846.792084:0:79773:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610db43c0 pname:cluuid:pid:xid:nid:opc:job multiop:7b4e8535-9db7-485d-abf6-bda966b63320:79773:1874669225571968:192.168.203.114@tcp:101:multiop.0 00000100:00100000:1.0:1787826846.792196:0:79773:0:(client.c:2715:ptlrpc_set_wait()) set ffff985610f13480 going to sleep for 11 seconds 00000100:00100000:1.0:1787826846.794623:0:79773:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610db43c0 pname:cluuid:pid:xid:nid:opc:job multiop:7b4e8535-9db7-485d-abf6-bda966b63320:79773:1874669225571968:192.168.203.114@tcp:101:multiop.0 00010000:00010000:1.0:1787826846.794641:0:79773:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e08bc lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1c13 expref: -99 pid: 79773 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826846.794663:0:79773:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e08bc lrc: 4/0,1 mode: --/PR res: [0x200016b60:0xb:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1c13 expref: -99 pid: 79773 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826846.794671:0:79773:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e08bc lrc: 4/0,1 mode: --/PR res: [0x200016b60:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1c13 expref: -99 pid: 79773 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826846.794679:0:79773:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e08bc lrc: 4/0,1 mode: PR/PR res: [0x200016b60:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1c13 expref: -99 pid: 79773 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826846.795036:0:79773:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e08bc lrc: 4/0,1 mode: PR/PR res: [0x200016b60:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1c13 expref: -99 pid: 79773 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826846.795062:0:79773:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e08bc lrc: 5/1,1 mode: PR/PR res: [0x200016b60:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1c13 expref: -99 pid: 79773 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826846.795072:0:79773:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e08bc lrc: 5/1,1 mode: PR/PR res: [0x200016b60:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1c13 expref: -99 pid: 79773 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826846.795079:0:79773:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e08bc lrc: 4/1,0 mode: PR/PR res: [0x200016b60:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1c13 expref: -99 pid: 79773 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826846.795091:0:79773:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff985610db43c0 x1874669225571968/t197568495647(197568495647) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/664 e 0 to 0 dl 1787826862 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'multiop.0' uid:0 gid:0 projid:0 00000002:00100000:1.0:1787826846.795104:0:79773:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff985610db43c0 x1874669225571968/t197568495647(197568495647) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/664 e 0 to 0 dl 1787826862 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'multiop.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826846.795121:0:79773:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e08bc lrc: 3/1,0 mode: PR/PR res: [0x200016b60:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1c13 expref: -99 pid: 79773 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:1.0:1787826846.795134:0:79773:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e08bc lrc: 3/1,0 mode: PR/PR res: [0x200016b60:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1c13 expref: -99 pid: 79773 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826846.795143:0:79773:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8590027616/11 (0 0) 00000080:00200000:1.0:1787826846.795149:0:79773:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:1.0:1787826846.795160:0:79773:0:(namei.c:1628:ll_create_it()) VFS Op:name=f34.replay-single, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=open|creat 00000080:00200000:1.0:1787826846.795185:0:79773:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200016b60:0xb:0x0](ffff98561158f448) 00000080:00200000:1.0:1787826846.795257:0:79773:0:(file.c:6854:ll_layout_conf()) [0x200016b60:0xb:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:1.0:1787826846.795262:0:79773:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x200016b60:0xb:0x0](ffff98561158f448) 00000080:00010000:1.0:1787826846.795266:0:79773:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200016b60:0xb:0x0](ffff98561158f448) for lock 0xb3f83dad1b7e08bc 00000080:00010000:1.0:1787826846.795275:0:79773:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200016b60:0xb:0x0](ffff98561158f448) 00000080:00200008:1.0:1787826846.795286:0:79773:0:(file.c:1090:ll_file_open()) START file f34.replay-single:[0x200016b60:0xb:0x0](ffff98561158f448), flags 100102 00000002:00100000:1.0:1787826846.795306:0:79773:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff985610db43c0 x1874669225571968/t197568495647(197568495647) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826862 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'multiop.0' uid:0 gid:0 projid:0 00000080:00010000:1.0:1787826846.795334:0:79773:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e08bc from it ffff985610e3fea0 00010000:00010000:1.0:1787826846.795339:0:79773:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e08bc lrc: 3/1,0 mode: PR/PR res: [0x200016b60:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a1c13 expref: -99 pid: 79773 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826846.795348:0:79773:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e08bc lrc: 2/0,0 mode: PR/PR res: [0x200016b60:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a1c13 expref: -99 pid: 79773 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826846.795367:0:79773:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200016b60:0xb:0x0](ffff98561158f448), inode mode 81a4 mask 40 00000080:00200000:1.0:1787826846.804877:0:79778:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826846.804916:0:79778:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826846.805007:0:79778:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e08c3 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 79778 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826846.805020:0:79778:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e08c3 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 79778 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826846.805032:0:79778:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e08c3 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 79778 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826846.805079:0:79778:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610db7480 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:79778:1874669225572096:192.168.203.114@tcp:101:rm.0 00000100:00100000:1.0:1787826846.805198:0:79778:0:(client.c:2715:ptlrpc_set_wait()) set ffff985610f13780 going to sleep for 11 seconds 00000100:00100000:1.0:1787826846.807323:0:79778:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610db7480 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:79778:1874669225572096:192.168.203.114@tcp:101:rm.0 00010000:00010000:1.0:1787826846.807347:0:79778:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e08c3 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1c28 expref: -99 pid: 79778 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826846.807357:0:79778:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e08c3 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1c28 expref: -99 pid: 79778 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826846.807368:0:79778:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e08c3 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1c28 expref: -99 pid: 79778 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826846.807378:0:79778:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e08c3 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1c28 expref: -99 pid: 79778 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826846.807388:0:79778:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e08c3 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1c28 expref: -99 pid: 79778 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826846.807394:0:79778:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e08c3 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1c28 expref: -99 pid: 79778 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826846.807399:0:79778:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e08c3 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1c28 expref: -99 pid: 79778 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826846.807407:0:79778:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff985610db7480 x1874669225572096/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826862 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826846.807417:0:79778:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e08c3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1c28 expref: -99 pid: 79778 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826846.807431:0:79778:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:1.0:1787826846.807437:0:79778:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:1.0:1787826846.807468:0:79778:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:1.0:1787826846.807475:0:79778:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e08c3 00000080:00010000:1.0:1787826846.807483:0:79778:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826846.807486:0:79778:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e08c3 from it ffffab3ac151bb58 00010000:00010000:1.0:1787826846.807491:0:79778:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e08c3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1c28 expref: -99 pid: 79778 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826846.807500:0:79778:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e08c3 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1c28 expref: -99 pid: 79778 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826846.807526:0:79778:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787826846.807535:0:79778:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f34.replay-single, flags=8 00000080:00200008:1.0:1787826846.807559:0:79778:0:(file.c:6313:ll_getattr_dentry()) START file f34.replay-single:[0x200016b60:0xb:0x0](ffff98561158f448), request_mask 2047, flags 0, foreign 0 00000080:00200000:1.0:1787826846.807564:0:79778:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200016b60:0xb:0x0](ffff98561158f448),name=f34.replay-single 00000002:00010000:1.0:1787826846.807582:0:79778:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200016b60:0xb:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826846.807590:0:79778:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e08bc lrc: 2/1,0 mode: PR/PR res: [0x200016b60:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a1c13 expref: -99 pid: 79773 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826846.807598:0:79778:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e08bc lrc: 2/1,0 mode: PR/PR res: [0x200016b60:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a1c13 expref: -99 pid: 79773 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826846.807603:0:79778:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826846.807606:0:79778:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200016b60:0xb:0x0](ffff98561158f448) 00000080:00010000:1.0:1787826846.807608:0:79778:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200016b60:0xb:0x0](ffff98561158f448) for lock 0xb3f83dad1b7e08bc 00000080:00010000:1.0:1787826846.807612:0:79778:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e08bc from it ffffab3ac151bcb8 00010000:00010000:1.0:1787826846.807614:0:79778:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e08bc lrc: 3/1,0 mode: PR/PR res: [0x200016b60:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a1c13 expref: -99 pid: 79773 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826846.807619:0:79778:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e08bc lrc: 2/0,0 mode: PR/PR res: [0x200016b60:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a1c13 expref: -99 pid: 79773 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826846.807638:0:79778:0:(vvp_io.c:1885:vvp_io_init()) [0x200016b60:0xb:0x0] ignore/verify layout 0/1, layout version 0 restore needed 0 00000001:00010000:1.0:1787826846.807649:0:79778:0:(glimpse.c:73:cl_glimpse_lock()) Glimpsing inode [0x200016b60:0xb:0x0] 00000008:00010000:1.0:1787826846.807667:0:79778:0:(lustre_osc.h:166:osc_enq2ldlm_flags()) flags: a 00000008:00010000:1.0:1787826846.807669:0:79778:0:(osc_lock.c:1320:osc_lock_init()) ### lock ffff98560560c230, osc lock ffff985602f8f380, flags 0x400001002 00010000:00010000:1.0:1787826846.807738:0:79778:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0000-osc-ffff985602bc9800 lock: 00000000eb029571/0xb3f83dad1b7e08ca lrc: 3/1,0 mode: --/PR res: [0x280000401:0x607:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 79778 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826846.807749:0:79778:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x400001000 ns: lustre-OST0000-osc-ffff985602bc9800 lock: 00000000eb029571/0xb3f83dad1b7e08ca lrc: 3/1,0 mode: --/PR res: [0x280000401:0x607:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 79778 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000020:00200000:1.0:1787826846.807785:0:79778:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1787826846.809178:0:79778:0:(file.c:1775:ll_merge_attr_nolock()) [0x200016b60:0xb:0x0] updating i_size 0 i_blocks 0 00000020:00010000:1.0:1787826846.809187:0:79778:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff985605499638 (ffff985610fe17e8/0) at cl_lock_release():245 00000020:00010000:1.0:1787826846.809192:0:79778:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff985605499638 (ffff985610fe17e8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1787826846.809196:0:79778:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff98560560c230 (ffff985610fe1488/1) at cl_lock_cancel():142 00000020:00010000:1.0:1787826846.809203:0:79778:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff985605499638 (ffff985610fe17e8/0) at cl_lock_fini():72 00000020:00010000:1.0:1787826846.809206:0:79778:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff98560560c230 (ffff985610fe17e8/1) at cl_lock_fini():72 00000080:00200000:1.0:1787826846.809223:0:79778:0:(vvp_io.c:304:vvp_io_fini()) [0x200016b60:0xb:0x0] ignore/verify layout 0/1, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826846.809270:0:79778:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826846.809289:0:79778:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826846.809301:0:79778:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e08c3 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1c28 expref: -99 pid: 79778 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826846.809315:0:79778:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e08c3 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1c28 expref: -99 pid: 79778 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826846.809322:0:79778:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826846.809329:0:79778:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826846.809333:0:79778:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e08c3 00000080:00010000:1.0:1787826846.809339:0:79778:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e08c3 from it ffffab3ac151bc10 00010000:00010000:1.0:1787826846.809343:0:79778:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e08c3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1c28 expref: -99 pid: 79778 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826846.809352:0:79778:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e08c3 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1c28 expref: -99 pid: 79778 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826846.809377:0:79778:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787826846.809387:0:79778:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f34.replay-single, flags=0 00000080:00200000:1.0:1787826846.809394:0:79778:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826846.809403:0:79778:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826846.809409:0:79778:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e08c3 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1c28 expref: -99 pid: 79778 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826846.809418:0:79778:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e08c3 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1c28 expref: -99 pid: 79778 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826846.809425:0:79778:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826846.809429:0:79778:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826846.809432:0:79778:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e08c3 00000080:00010000:1.0:1787826846.809436:0:79778:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e08c3 from it ffffab3ac151bd68 00010000:00010000:1.0:1787826846.809450:0:79778:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e08c3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1c28 expref: -99 pid: 79778 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826846.809458:0:79778:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e08c3 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1c28 expref: -99 pid: 79778 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787826846.809469:0:79778:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787826846.809476:0:79778:0:(namei.c:2418:ll_unlink()) VFS Op:name=f34.replay-single, dir=[0x200000007:0x1:0x0](ffff98561158e3c8) 00010000:00010000:1.0:1787826846.809511:0:79778:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e08c3 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x5aa6d504d91a1c28 expref: -99 pid: 79778 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787826846.809540:0:79778:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e08bc lrc: 2/0,0 mode: PR/PR res: [0x200016b60:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x5aa6d504d91a1c13 expref: -99 pid: 79773 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826846.809551:0:79778:0:(vvp_object.c:117:vvp_conf_set()) [0x200016b60:0xb:0x0]: losing layout lock 00010000:00010000:1.0:1787826846.809594:0:79778:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e08bc lrc: 1/0,0 mode: --/PR res: [0x200016b60:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x5aa6d504d91a1c13 expref: -99 pid: 79773 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826846.809603:0:79778:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e08c3 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a1c28 expref: -99 pid: 79778 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787826846.809609:0:79778:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 2 locks packed 00010000:00010000:1.0:1787826846.809613:0:79778:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642051400/0xb3f83dad1b7e08bc lrc: 0/0,0 mode: --/PR res: [0x200016b60:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x5aa6d504d91a1c13 expref: -99 pid: 79773 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826846.809641:0:79778:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e08c3 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a1c28 expref: -99 pid: 79778 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000020:00100000:1.0:1787826846.809667:0:79778:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787826846.809677:0:79778:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610db5a40 x1874669225572480/t0(0) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826846.809722:0:79778:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610db5a40 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:79778:1874669225572480:192.168.203.114@tcp:36:rm.0 00000100:00100000:1.0:1787826846.809799:0:79778:0:(client.c:2715:ptlrpc_set_wait()) set ffff985610f13780 going to sleep for 11 seconds 00000100:00100000:1.0:1787826846.811631:0:79778:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610db5a40 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:79778:1874669225572480:192.168.203.114@tcp:36:rm.0 00000100:00100000:3.0:1787826846.965303:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2385:1874669225572608:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787826846.965314:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669225572736:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787826846.965394:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee1e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669225572864:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787826846.966674:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee1e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669225572864:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787826846.966746:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669225572736:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787826846.966768:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2385:1874669225572608:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787826847.123482:0:2377:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.114@tcp (source: 12345-192.168.203.114@tcp) 00000100:00100000:2.0:1787826847.123518:0:2377:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.114@tcp (source: 12345-192.168.203.114@tcp) 00000100:00100000:3.0:1787826847.123543:0:33343:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874669235508480 00000100:00100000:3.0:1787826847.123552:0:33343:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874669235508480 00000100:00100000:2.0:1787826847.123557:0:15156:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874669235508608 00000100:00100000:2.0:1787826847.123561:0:15156:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874669235508608 00000100:00100000:2.0:1787826847.123592:0:15156:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.114@tcp, seq: 246 00000100:00100000:3.0:1787826847.123601:0:33343:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.114@tcp, seq: 247 00000100:00100000:2.0:1787826847.123602:0:15156:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff985602ee1e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb01_001:LOV_OSC_UUID+4:8406:x1874669235508608:12345-192.168.203.114@tcp:104: 00000100:00100000:3.0:1787826847.123615:0:33343:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff985602ee2940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb01_002:LOV_OSC_UUID+4:8407:x1874669235508480:12345-192.168.203.114@tcp:104: 00010000:00010000:2.0:1787826847.123618:0:15156:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-OST0001-osc-ffff985602bc9800 lock: 0000000024db9e92/0xb3f83dad1b7e04da lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x602:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x420000010000 nid: local remote: 0x5aa6d504d91a19ab expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826847.123626:0:33343:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-OST0000-osc-ffff985602bc9800 lock: 000000006563a0df/0xb3f83dad1b7e04d3 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x602:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x420000010000 nid: local remote: 0x5aa6d504d91a19b9 expref: -99 pid: 79202 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826847.123686:0:15156:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to priority blp list, 2 blwis in pool 00000100:00100000:2.0:1787826847.123718:0:15156:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff985602ee1e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb01_001:LOV_OSC_UUID+4:8406:x1874669235508608:12345-192.168.203.114@tcp:104: Request processed in 93us (174us total) trans 0 rc 0/0 00010000:00010000:3.0:1787826847.123723:0:33343:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to priority blp list, 2 blwis in pool 00000100:00100000:2.0:1787826847.123726:0:15156:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.114@tcp, seq: 246 00000100:00100000:3.0:1787826847.123734:0:33343:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff985602ee2940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb01_002:LOV_OSC_UUID+4:8407:x1874669235508480:12345-192.168.203.114@tcp:104: Request processed in 121us (259us total) trans 0 rc 0/0 00010000:00010000:1.0:1787826847.123743:0:15163:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00000100:00100000:3.0:1787826847.123743:0:33343:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.114@tcp, seq: 247 00010000:00010000:1.0:1787826847.123757:0:15163:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-OST0001-osc-ffff985602bc9800 lock: 0000000024db9e92/0xb3f83dad1b7e04da lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x602:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x420000010000 nid: local remote: 0x5aa6d504d91a19ab expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826847.123776:0:15163:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985610dc1600 already unused, calling callback (ffffffffc13b01e0) 00010000:00010000:1.0:1787826847.123785:0:15163:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0001-osc-ffff985602bc9800 lock: 0000000024db9e92/0xb3f83dad1b7e04da lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x602:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x428400010000 nid: local remote: 0x5aa6d504d91a19ab expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826847.123847:0:15163:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-OST0001-osc-ffff985602bc9800 lock: 0000000024db9e92/0xb3f83dad1b7e04da lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x602:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x4c29400010020 nid: local remote: 0x5aa6d504d91a19ab expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826847.123856:0:15163:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826847.123882:0:15163:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-OST0001-osc-ffff985602bc9800 lock: 0000000024db9e92/0xb3f83dad1b7e04da lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x602:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x4c29400010020 nid: local remote: 0x5aa6d504d91a19ab expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826847.123895:0:15163:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0001-osc-ffff985602bc9800 lock: 0000000024db9e92/0xb3f83dad1b7e04da lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x602:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x4c29400010020 nid: local remote: 0x5aa6d504d91a19ab expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826847.123945:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826847.123950:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826847.123956:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826847.123976:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610db70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669225572992:192.168.203.114@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1787826847.125257:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610db70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669225572992:192.168.203.114@tcp:103:ldlm_bl.0 00000080:00200000:3.0:1787826847.177273:0:79798:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:3.0:1787826847.177302:0:79798:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826847.177320:0:79798:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0800/0xb3f83dad1b7e08d1 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1c60 expref: -99 pid: 79791 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:3.0:1787826847.177336:0:79798:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0800/0xb3f83dad1b7e08d1 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1c60 expref: -99 pid: 79791 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:3.0:1787826847.177345:0:79798:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826847.177352:0:79798:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:3.0:1787826847.177356:0:79798:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e08d1 00000080:00010000:3.0:1787826847.177363:0:79798:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e08d1 from it ffffab3ac167bbd0 00010000:00010000:3.0:1787826847.177367:0:79798:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0800/0xb3f83dad1b7e08d1 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1c60 expref: -99 pid: 79791 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:3.0:1787826847.177378:0:79798:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0800/0xb3f83dad1b7e08d1 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1c60 expref: -99 pid: 79791 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00000080:00200000:3.0:1787826847.177394:0:79798:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:3.0:1787826847.177406:0:79798:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=fsa-oleg314-client.virtnet, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), flags=1536 00000080:00200000:3.0:1787826847.177410:0:79798:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:3.0:1787826847.177416:0:79798:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826847.177422:0:79798:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0800/0xb3f83dad1b7e08d1 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1c60 expref: -99 pid: 79791 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00010000:00010000:3.0:1787826847.177430:0:79798:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0800/0xb3f83dad1b7e08d1 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1c60 expref: -99 pid: 79791 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00000080:00200000:3.0:1787826847.177436:0:79798:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826847.177440:0:79798:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:3.0:1787826847.177443:0:79798:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e08d1 00000080:00010000:3.0:1787826847.177446:0:79798:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e08d1 from it ffffab3ac167bcd0 00010000:00010000:3.0:1787826847.177449:0:79798:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0800/0xb3f83dad1b7e08d1 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1c60 expref: -99 pid: 79791 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00010000:00010000:3.0:1787826847.177456:0:79798:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0800/0xb3f83dad1b7e08d1 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1c60 expref: -99 pid: 79791 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00000080:00200000:3.0:1787826847.177464:0:79798:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 3 00000080:00200000:3.0:1787826847.177469:0:79798:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:3.0:1787826847.177474:0:79798:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826847.177480:0:79798:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0800/0xb3f83dad1b7e08d1 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1c60 expref: -99 pid: 79791 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00010000:00010000:3.0:1787826847.177487:0:79798:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0800/0xb3f83dad1b7e08d1 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1c60 expref: -99 pid: 79791 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00000080:00200000:3.0:1787826847.177493:0:79798:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826847.177496:0:79798:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:3.0:1787826847.177499:0:79798:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e08d1 00000080:00010000:3.0:1787826847.177502:0:79798:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e08d1 from it ffffab3ac167bdb0 00010000:00010000:3.0:1787826847.177505:0:79798:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0800/0xb3f83dad1b7e08d1 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1c60 expref: -99 pid: 79791 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00010000:00010000:3.0:1787826847.177512:0:79798:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0800/0xb3f83dad1b7e08d1 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1c60 expref: -99 pid: 79791 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00000080:00200000:3.0:1787826847.177519:0:79798:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 3 00000080:00200000:3.0:1787826847.177523:0:79798:0:(namei.c:2129:ll_create_nd()) VFS Op:name=fsa-oleg314-client.virtnet, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), flags=33188, excl=1 00000080:00200000:3.0:1787826847.177527:0:79798:0:(namei.c:2084:ll_mknod()) VFS Op:name=fsa-oleg314-client.virtnet, dir=[0x200000007:0x1:0x0](ffff98561158e3c8) mode 100644 dev 0 00010000:00010000:3.0:1787826847.177549:0:79798:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0800/0xb3f83dad1b7e08d1 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x5aa6d504d91a1c60 expref: -99 pid: 79791 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00010000:00010000:3.0:1787826847.177594:0:79798:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0800/0xb3f83dad1b7e08d1 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a1c60 expref: -99 pid: 79791 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00010000:00010000:3.0:1787826847.177600:0:79798:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826847.177603:0:79798:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0800/0xb3f83dad1b7e08d1 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a1c60 expref: -99 pid: 79791 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00000020:00100000:3.0:1787826847.177624:0:79798:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:3.0:1787826847.177637:0:79798:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610e4de00 x1874669225574016/t0(0) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 528/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826847.177659:0:79798:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e4de00 pname:cluuid:pid:xid:nid:opc:job mcreate:7b4e8535-9db7-485d-abf6-bda966b63320:79798:1874669225574016:192.168.203.114@tcp:36:mcreate.0 00000100:00100000:3.0:1787826847.177761:0:79798:0:(client.c:2715:ptlrpc_set_wait()) set ffff985611f30480 going to sleep for 11 seconds 00000100:00100000:3.0:1787826847.179774:0:79798:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e4de00 pname:cluuid:pid:xid:nid:opc:job mcreate:7b4e8535-9db7-485d-abf6-bda966b63320:79798:1874669225574016:192.168.203.114@tcp:36:mcreate.0 00000080:00200000:3.0:1787826847.179822:0:79798:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200016b60:0xc:0x0](ffff98561011e3c8) 00000080:00200000:3.0:1787826847.179857:0:79798:0:(namei.c:2137:ll_create_nd()) VFS Op:name=fsa-oleg314-client.virtnet, unhashed 1 00000080:00200000:2.0:1787826847.186477:0:79801:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:2.0:1787826847.186505:0:79801:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826847.186568:0:79801:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5200/0xb3f83dad1b7e08d8 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 79801 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826847.186579:0:79801:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5200/0xb3f83dad1b7e08d8 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 79801 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826847.186589:0:79801:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5200/0xb3f83dad1b7e08d8 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 79801 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826847.186615:0:79801:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee0f00 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:79801:1874669225574144:192.168.203.114@tcp:101:rm.0 00000100:00100000:2.0:1787826847.186714:0:79801:0:(client.c:2715:ptlrpc_set_wait()) set ffff985611e5e3c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826847.188644:0:79801:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee0f00 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:79801:1874669225574144:192.168.203.114@tcp:101:rm.0 00010000:00010000:1.0:1787826847.188662:0:79801:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5200/0xb3f83dad1b7e08d8 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1c7c expref: -99 pid: 79801 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826847.188673:0:79801:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5200/0xb3f83dad1b7e08d8 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1c7c expref: -99 pid: 79801 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826847.188685:0:79801:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5200/0xb3f83dad1b7e08d8 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1c7c expref: -99 pid: 79801 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826847.188711:0:79801:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5200/0xb3f83dad1b7e08d8 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1c7c expref: -99 pid: 79801 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826847.188724:0:79801:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5200/0xb3f83dad1b7e08d8 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1c7c expref: -99 pid: 79801 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826847.188732:0:79801:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5200/0xb3f83dad1b7e08d8 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1c7c expref: -99 pid: 79801 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826847.188739:0:79801:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5200/0xb3f83dad1b7e08d8 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1c7c expref: -99 pid: 79801 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826847.188751:0:79801:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff985602ee0f00 x1874669225574144/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826863 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826847.188769:0:79801:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5200/0xb3f83dad1b7e08d8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1c7c expref: -99 pid: 79801 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826847.188780:0:79801:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:1.0:1787826847.188786:0:79801:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:1.0:1787826847.188815:0:79801:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:1.0:1787826847.188821:0:79801:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e08d8 00000080:00010000:1.0:1787826847.188826:0:79801:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826847.188830:0:79801:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e08d8 from it ffffab3ac1c4fb58 00010000:00010000:1.0:1787826847.188833:0:79801:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5200/0xb3f83dad1b7e08d8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1c7c expref: -99 pid: 79801 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826847.188841:0:79801:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5200/0xb3f83dad1b7e08d8 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1c7c expref: -99 pid: 79801 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826847.188861:0:79801:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787826847.188877:0:79801:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=fsa-oleg314-client.virtnet, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), flags=8 00000080:00200000:1.0:1787826847.188881:0:79801:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=fsa-oleg314-client.virtnet, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=getattr 00000002:00010000:1.0:1787826847.188897:0:79801:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: fsa-oleg314-client.virtnet,[0x0:0x0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 01 00010000:00010000:1.0:1787826847.188925:0:79801:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e08df lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 79801 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826847.188932:0:79801:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e08df lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 79801 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826847.188939:0:79801:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e08df lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 79801 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826847.188959:0:79801:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610db70c0 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:79801:1874669225574272:192.168.203.114@tcp:101:rm.0 00000100:00100000:1.0:1787826847.189008:0:79801:0:(client.c:2715:ptlrpc_set_wait()) set ffff985610f13780 going to sleep for 11 seconds 00000100:00100000:1.0:1787826847.190876:0:79801:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610db70c0 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:79801:1874669225574272:192.168.203.114@tcp:101:rm.0 00010000:00010000:1.0:1787826847.190895:0:79801:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e08df lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1c98 expref: -99 pid: 79801 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826847.190921:0:79801:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e08df lrc: 4/1,0 mode: --/PR res: [0x200016b60:0xc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1c98 expref: -99 pid: 79801 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826847.190930:0:79801:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e08df lrc: 4/1,0 mode: --/PR res: [0x200016b60:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1c98 expref: -99 pid: 79801 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826847.190939:0:79801:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e08df lrc: 4/1,0 mode: PR/PR res: [0x200016b60:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1c98 expref: -99 pid: 79801 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826847.190948:0:79801:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e08df lrc: 4/1,0 mode: PR/PR res: [0x200016b60:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1c98 expref: -99 pid: 79801 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826847.190959:0:79801:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e08df lrc: 5/2,0 mode: PR/PR res: [0x200016b60:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1c98 expref: -99 pid: 79801 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826847.190967:0:79801:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e08df lrc: 5/2,0 mode: PR/PR res: [0x200016b60:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1c98 expref: -99 pid: 79801 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826847.190974:0:79801:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e08df lrc: 4/1,0 mode: PR/PR res: [0x200016b60:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1c98 expref: -99 pid: 79801 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826847.190991:0:79801:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff985610db70c0 x1874669225574272/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 600/584 e 0 to 0 dl 1787826863 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826847.191004:0:79801:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e08df lrc: 3/1,0 mode: PR/PR res: [0x200016b60:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1c98 expref: -99 pid: 79801 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826847.191012:0:79801:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8590027616/12 (0 0) 00000080:00200000:1.0:1787826847.191017:0:79801:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:1.0:1787826847.191033:0:79801:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200016b60:0xc:0x0], updating i_size 0 00000080:00200000:1.0:1787826847.191035:0:79801:0:(namei.c:147:ll_iget()) got inode: [0x200016b60:0xc:0x0](ffff98561011e3c8): rc = 0 00000080:00200000:1.0:1787826847.191061:0:79801:0:(file.c:6854:ll_layout_conf()) [0x200016b60:0xc:0x0]: layout version change: 4294967294 -> 4294967295 00000080:00010000:1.0:1787826847.191068:0:79801:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200016b60:0xc:0x0](ffff98561011e3c8) for lock 0xb3f83dad1b7e08df 00000080:00010000:1.0:1787826847.191076:0:79801:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200016b60:0xc:0x0](ffff98561011e3c8) 00000080:00010000:1.0:1787826847.191080:0:79801:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e08df from it ffffab3ac1c4fb20 00010000:00010000:1.0:1787826847.191083:0:79801:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e08df lrc: 3/1,0 mode: PR/PR res: [0x200016b60:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a1c98 expref: -99 pid: 79801 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826847.191092:0:79801:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e08df lrc: 2/0,0 mode: PR/PR res: [0x200016b60:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a1c98 expref: -99 pid: 79801 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:1.0:1787826847.191122:0:79801:0:(file.c:6313:ll_getattr_dentry()) START file fsa-oleg314-client.virtnet:[0x200016b60:0xc:0x0](ffff98561011e3c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:1.0:1787826847.191129:0:79801:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200016b60:0xc:0x0](ffff98561011e3c8),name=fsa-oleg314-client.virtnet 00000002:00010000:1.0:1787826847.191145:0:79801:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200016b60:0xc:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826847.191153:0:79801:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e08df lrc: 2/1,0 mode: PR/PR res: [0x200016b60:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a1c98 expref: -99 pid: 79801 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826847.191163:0:79801:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e08df lrc: 2/1,0 mode: PR/PR res: [0x200016b60:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a1c98 expref: -99 pid: 79801 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826847.191170:0:79801:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826847.191174:0:79801:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200016b60:0xc:0x0](ffff98561011e3c8) 00000080:00010000:1.0:1787826847.191177:0:79801:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200016b60:0xc:0x0](ffff98561011e3c8) for lock 0xb3f83dad1b7e08df 00000080:00010000:1.0:1787826847.191181:0:79801:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e08df from it ffffab3ac1c4fcb8 00010000:00010000:1.0:1787826847.191184:0:79801:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e08df lrc: 3/1,0 mode: PR/PR res: [0x200016b60:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a1c98 expref: -99 pid: 79801 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826847.191190:0:79801:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e08df lrc: 2/0,0 mode: PR/PR res: [0x200016b60:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a1c98 expref: -99 pid: 79801 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826847.191236:0:79801:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826847.191243:0:79801:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826847.191250:0:79801:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5200/0xb3f83dad1b7e08d8 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1c7c expref: -99 pid: 79801 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826847.191257:0:79801:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5200/0xb3f83dad1b7e08d8 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1c7c expref: -99 pid: 79801 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826847.191264:0:79801:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826847.191267:0:79801:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826847.191270:0:79801:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e08d8 00000080:00010000:1.0:1787826847.191273:0:79801:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e08d8 from it ffffab3ac1c4fc10 00010000:00010000:1.0:1787826847.191276:0:79801:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5200/0xb3f83dad1b7e08d8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1c7c expref: -99 pid: 79801 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826847.191283:0:79801:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5200/0xb3f83dad1b7e08d8 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1c7c expref: -99 pid: 79801 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826847.191292:0:79801:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787826847.191299:0:79801:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=fsa-oleg314-client.virtnet, flags=0 00000080:00200000:1.0:1787826847.191303:0:79801:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826847.191308:0:79801:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826847.191322:0:79801:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5200/0xb3f83dad1b7e08d8 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1c7c expref: -99 pid: 79801 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826847.191331:0:79801:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5200/0xb3f83dad1b7e08d8 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1c7c expref: -99 pid: 79801 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826847.191337:0:79801:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826847.191341:0:79801:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826847.191343:0:79801:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e08d8 00000080:00010000:1.0:1787826847.191346:0:79801:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e08d8 from it ffffab3ac1c4fd68 00010000:00010000:1.0:1787826847.191349:0:79801:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5200/0xb3f83dad1b7e08d8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a1c7c expref: -99 pid: 79801 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826847.191356:0:79801:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5200/0xb3f83dad1b7e08d8 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a1c7c expref: -99 pid: 79801 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787826847.191363:0:79801:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787826847.191367:0:79801:0:(namei.c:2418:ll_unlink()) VFS Op:name=fsa-oleg314-client.virtnet, dir=[0x200000007:0x1:0x0](ffff98561158e3c8) 00010000:00010000:1.0:1787826847.191386:0:79801:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5200/0xb3f83dad1b7e08d8 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x5aa6d504d91a1c7c expref: -99 pid: 79801 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787826847.191407:0:79801:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e08df lrc: 2/0,0 mode: PR/PR res: [0x200016b60:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x5aa6d504d91a1c98 expref: -99 pid: 79801 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826847.191416:0:79801:0:(vvp_object.c:117:vvp_conf_set()) [0x200016b60:0xc:0x0]: losing layout lock 00010000:00010000:1.0:1787826847.191439:0:79801:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e08df lrc: 1/0,0 mode: --/PR res: [0x200016b60:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x5aa6d504d91a1c98 expref: -99 pid: 79801 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826847.191447:0:79801:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5200/0xb3f83dad1b7e08d8 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a1c7c expref: -99 pid: 79801 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787826847.191452:0:79801:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 2 locks packed 00010000:00010000:1.0:1787826847.191455:0:79801:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050200/0xb3f83dad1b7e08df lrc: 0/0,0 mode: --/PR res: [0x200016b60:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x5aa6d504d91a1c98 expref: -99 pid: 79801 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826847.191466:0:79801:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5200/0xb3f83dad1b7e08d8 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a1c7c expref: -99 pid: 79801 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000020:00100000:1.0:1787826847.191483:0:79801:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787826847.191490:0:79801:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610db70c0 x1874669225574528/t0(0) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826847.191509:0:79801:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610db70c0 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:79801:1874669225574528:192.168.203.114@tcp:36:rm.0 00000100:00100000:1.0:1787826847.191572:0:79801:0:(client.c:2715:ptlrpc_set_wait()) set ffff985610f13780 going to sleep for 11 seconds 00000100:00100000:1.0:1787826847.194199:0:79801:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610db70c0 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:79801:1874669225574528:192.168.203.114@tcp:36:rm.0 00000080:00200000:1.0:1787826847.194275:0:79801:0:(vvp_io.c:1885:vvp_io_init()) [0x200016b60:0xc:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826847.194287:0:79801:0:(vvp_io.c:304:vvp_io_fini()) [0x200016b60:0xc:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826847.194296:0:79801:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200016b60:0xc:0x0](ffff98561011e3c8) 00010000:00010000:2.0:1787826848.564855:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:3.0:1787826849.582602:0:79866:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: local REPLAY BARRIER on lustre-MDT0000 00010000:00010000:3.0:1787826849.588861:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826850.233396:0:2377:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.114@tcp (source: 12345-192.168.203.114@tcp) 00000100:00100000:3.0:1787826850.233434:0:2377:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.114@tcp (source: 12345-192.168.203.114@tcp) 00000100:00100000:2.0:1787826850.233496:0:15156:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874669235512704 00000100:00100000:3.0:1787826850.233511:0:2377:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.114@tcp (source: 12345-192.168.203.114@tcp) 00000100:00100000:2.0:1787826850.233512:0:15156:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874669235512704 00000100:00100000:3.0:1787826850.233537:0:2377:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.114@tcp (source: 12345-192.168.203.114@tcp) 00000100:00100000:3.0:1787826850.233558:0:2377:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.114@tcp (source: 12345-192.168.203.114@tcp) 00000100:00100000:3.0:1787826850.233587:0:2377:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.114@tcp (source: 12345-192.168.203.114@tcp) 00000100:00100000:2.0:1787826850.233593:0:15156:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:3.0:1787826850.233624:0:2377:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.114@tcp (source: 12345-192.168.203.114@tcp) 00000100:00100000:2.0:1787826850.233626:0:15156:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874669235512832 00000100:00100000:2.0:1787826850.233633:0:15156:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874669235512832 00000100:00100000:3.0:1787826850.233661:0:2377:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.114@tcp (source: 12345-192.168.203.114@tcp) 00000100:00100000:1.0:1787826850.233662:0:15154:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874669235513344 00000100:00100000:2.0:1787826850.233675:0:15156:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874669235512960 00000100:00100000:1.0:1787826850.233676:0:15154:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874669235513344 00000100:00100000:2.0:1787826850.233684:0:15156:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874669235512960 00000100:00100000:3.0:1787826850.233754:0:15155:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874669235513088 00000100:00100000:3.0:1787826850.233761:0:15155:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874669235513088 00000100:00100000:2.0:1787826850.233780:0:15156:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874669235513600 00000100:00100000:2.0:1787826850.233787:0:15156:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874669235513600 00000100:00100000:3.0:1787826850.233819:0:15155:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.114@tcp, seq: 248 00000100:00100000:3.0:1787826850.233830:0:15155:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff985610e4cf00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb01_000:LOV_OSC_UUID+7:43894:x1874669235512704:12345-192.168.203.114@tcp:104: 00000100:00100000:2.0:1787826850.233840:0:15156:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.114@tcp, seq: 249 00010000:00010000:3.0:1787826850.233844:0:15155:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-OST0001-osc-ffff985602bc9800 lock: 0000000024907bc3/0xb3f83dad1b7e04e8 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x603:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x420000010000 nid: local remote: 0x5aa6d504d91a19c0 expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000100:00100000:2.0:1787826850.233867:0:15156:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff985610e4d680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb01_001:LOV_OSC_UUID+7:17106:x1874669235512832:12345-192.168.203.114@tcp:104: 00010000:00010000:2.0:1787826850.233889:0:15156:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-OST0000-osc-ffff985602bc9800 lock: 000000002c3b7699/0xb3f83dad1b7e04ef lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x604:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x420000010000 nid: local remote: 0x5aa6d504d91a19c7 expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826850.233955:0:15155:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to priority blp list, 3 blwis in pool 00000100:00100000:3.0:1787826850.233962:0:15155:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff985610e4cf00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb01_000:LOV_OSC_UUID+7:43894:x1874669235512704:12345-192.168.203.114@tcp:104: Request processed in 133us (574us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826850.233972:0:15155:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.114@tcp, seq: 248 00010000:00010000:2.0:1787826850.233979:0:15156:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to priority blp list, 3 blwis in pool 00000100:00100000:2.0:1787826850.233993:0:15156:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff985610e4d680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb01_001:LOV_OSC_UUID+7:17106:x1874669235512832:12345-192.168.203.114@tcp:104: Request processed in 137us (554us total) trans 0 rc 0/0 00010000:00010000:1.0:1787826850.234007:0:33342:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00000100:00100000:3.0:1787826850.234007:0:15155:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.114@tcp, seq: 250 00000100:00100000:3.0:1787826850.234019:0:15155:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff985610e4c3c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb01_000:LOV_OSC_UUID+6:43895:x1874669235512960:12345-192.168.203.114@tcp:104: 00010000:00010000:1.0:1787826850.234023:0:33342:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-OST0001-osc-ffff985602bc9800 lock: 0000000024907bc3/0xb3f83dad1b7e04e8 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x603:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x420000010000 nid: local remote: 0x5aa6d504d91a19c0 expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000100:00100000:2.0:1787826850.234037:0:15156:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.114@tcp, seq: 249 00010000:00010000:3.0:1787826850.234045:0:15155:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-OST0001-osc-ffff985602bc9800 lock: 000000009edc46ab/0xb3f83dad1b7e053c lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x606:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x420000010000 nid: local remote: 0x5aa6d504d91a1a14 expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826850.234058:0:33342:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985610dc1000 already unused, calling callback (ffffffffc13b01e0) 00000100:00100000:2.0:1787826850.234068:0:15156:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.114@tcp, seq: 251 00010000:00010000:1.0:1787826850.234071:0:33342:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0001-osc-ffff985602bc9800 lock: 0000000024907bc3/0xb3f83dad1b7e04e8 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x603:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x428400010000 nid: local remote: 0x5aa6d504d91a19c0 expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000100:00100000:2.0:1787826850.234097:0:15156:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9856088c2850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb01_001:LOV_OSC_UUID+5:43899:x1874669235513088:12345-192.168.203.114@tcp:104: 00010000:00010000:2.0:1787826850.234131:0:15156:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-OST0000-osc-ffff985602bc9800 lock: 00000000a32e421a/0xb3f83dad1b7e04e1 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x603:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x420000010000 nid: local remote: 0x5aa6d504d91a19b2 expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826850.234155:0:15155:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to priority blp list, 3 blwis in pool 00000100:00100000:3.0:1787826850.234162:0:15155:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff985610e4c3c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb01_000:LOV_OSC_UUID+6:43895:x1874669235512960:12345-192.168.203.114@tcp:104: Request processed in 145us (651us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826850.234170:0:15155:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.114@tcp, seq: 250 00010000:00010000:2.0:1787826850.234174:0:15156:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/4 locks to priority blp list, 4 blwis in pool 00010000:00010000:1.0:1787826850.234177:0:33342:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-OST0001-osc-ffff985602bc9800 lock: 0000000024907bc3/0xb3f83dad1b7e04e8 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x603:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x4c29400010020 nid: local remote: 0x5aa6d504d91a19c0 expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000100:00100000:2.0:1787826850.234192:0:15156:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9856088c2850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb01_001:LOV_OSC_UUID+5:43899:x1874669235513088:12345-192.168.203.114@tcp:104: Request processed in 102us (647us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826850.234198:0:15155:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.114@tcp, seq: 252 00000100:00100000:3.0:1787826850.234218:0:15155:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff985610e4cb40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb01_000:LOV_OSC_UUID+5:43929:x1874669235513600:12345-192.168.203.114@tcp:104: 00010000:00010000:1.0:1787826850.234221:0:33342:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:2.0:1787826850.234234:0:15156:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.114@tcp, seq: 251 00010000:00010000:3.0:1787826850.234242:0:15155:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-OST0000-osc-ffff985602bc9800 lock: 000000005fe2ea3b/0xb3f83dad1b7e050b lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x606:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x420000010000 nid: local remote: 0x5aa6d504d91a19e3 expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826850.234263:0:33342:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-OST0001-osc-ffff985602bc9800 lock: 0000000024907bc3/0xb3f83dad1b7e04e8 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x603:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x4c29400010020 nid: local remote: 0x5aa6d504d91a19c0 expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826850.234277:0:33342:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0001-osc-ffff985602bc9800 lock: 0000000024907bc3/0xb3f83dad1b7e04e8 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x603:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x4c29400010020 nid: local remote: 0x5aa6d504d91a19c0 expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826850.234287:0:15155:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/6 locks to priority blp list, 6 blwis in pool 00000100:00100000:3.0:1787826850.234305:0:15155:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff985610e4cb40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb01_000:LOV_OSC_UUID+4:43929:x1874669235513600:12345-192.168.203.114@tcp:104: Request processed in 84us (641us total) trans 0 rc 0/0 00010000:00010000:1.0:1787826850.234317:0:33342:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 5 total in blp. (5 blwis in pool) 00000100:00100000:3.0:1787826850.234322:0:15155:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.114@tcp, seq: 252 00010000:00010000:1.0:1787826850.234327:0:33342:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-OST0000-osc-ffff985602bc9800 lock: 00000000b37c3983/0xb3f83dad1b7e04fd lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x605:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x420000010000 nid: local remote: 0x5aa6d504d91a19dc expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826850.234351:0:33342:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985610dc1a00 already unused, calling callback (ffffffffc13b01e0) 00010000:00010000:3.0:1787826850.234363:0:15163:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 4 total in blp. (4 blwis in pool) 00010000:00010000:1.0:1787826850.234364:0:33342:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0000-osc-ffff985602bc9800 lock: 00000000b37c3983/0xb3f83dad1b7e04fd lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x605:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x428400010000 nid: local remote: 0x5aa6d504d91a19dc expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826850.234378:0:15163:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-OST0000-osc-ffff985602bc9800 lock: 000000002c3b7699/0xb3f83dad1b7e04ef lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x604:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x420000010000 nid: local remote: 0x5aa6d504d91a19c7 expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826850.234390:0:15163:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985610dc0c00 already unused, calling callback (ffffffffc13b01e0) 00010000:00010000:3.0:1787826850.234396:0:15163:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0000-osc-ffff985602bc9800 lock: 000000002c3b7699/0xb3f83dad1b7e04ef lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x604:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x428400010000 nid: local remote: 0x5aa6d504d91a19c7 expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826850.234409:0:33342:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-OST0000-osc-ffff985602bc9800 lock: 00000000b37c3983/0xb3f83dad1b7e04fd lrc: 2/0,0 mode: --/PR res: [0x280000401:0x605:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x4c29400010020 nid: local remote: 0x5aa6d504d91a19dc expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826850.234425:0:33342:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826850.234442:0:15163:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-OST0000-osc-ffff985602bc9800 lock: 000000002c3b7699/0xb3f83dad1b7e04ef lrc: 2/0,0 mode: --/PR res: [0x280000401:0x604:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x4c29400010020 nid: local remote: 0x5aa6d504d91a19c7 expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826850.234447:0:33342:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-OST0000-osc-ffff985602bc9800 lock: 00000000b37c3983/0xb3f83dad1b7e04fd lrc: 1/0,0 mode: --/PR res: [0x280000401:0x605:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x4c29400010020 nid: local remote: 0x5aa6d504d91a19dc expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826850.234461:0:15163:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826850.234473:0:33342:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0000-osc-ffff985602bc9800 lock: 00000000b37c3983/0xb3f83dad1b7e04fd lrc: 0/0,0 mode: --/PR res: [0x280000401:0x605:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x4c29400010020 nid: local remote: 0x5aa6d504d91a19dc expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826850.234493:0:15163:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-OST0000-osc-ffff985602bc9800 lock: 000000002c3b7699/0xb3f83dad1b7e04ef lrc: 1/0,0 mode: --/PR res: [0x280000401:0x604:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x4c29400010020 nid: local remote: 0x5aa6d504d91a19c7 expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826850.234501:0:33342:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 4 total in blp. (4 blwis in pool) 00010000:00010000:3.0:1787826850.234504:0:15163:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0000-osc-ffff985602bc9800 lock: 000000002c3b7699/0xb3f83dad1b7e04ef lrc: 0/0,0 mode: --/PR res: [0x280000401:0x604:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x4c29400010020 nid: local remote: 0x5aa6d504d91a19c7 expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826850.234511:0:33342:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-OST0001-osc-ffff985602bc9800 lock: 000000009edc46ab/0xb3f83dad1b7e053c lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x606:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x420000010000 nid: local remote: 0x5aa6d504d91a1a14 expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1787826850.234523:0:15164:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:3.0:1787826850.234557:0:15163:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1787826850.234565:0:15163:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-OST0000-osc-ffff985602bc9800 lock: 000000005fe2ea3b/0xb3f83dad1b7e050b lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x606:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x420000010000 nid: local remote: 0x5aa6d504d91a19e3 expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826850.234566:0:33342:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985610e06000 already unused, calling callback (ffffffffc13b01e0) 00010000:00010000:1.0:1787826850.234574:0:33342:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0001-osc-ffff985602bc9800 lock: 000000009edc46ab/0xb3f83dad1b7e053c lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x606:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x428400010000 nid: local remote: 0x5aa6d504d91a1a14 expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826850.234613:0:15163:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985610dc0800 already unused, calling callback (ffffffffc13b01e0) 00010000:00010000:3.0:1787826850.234619:0:15163:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0000-osc-ffff985602bc9800 lock: 000000005fe2ea3b/0xb3f83dad1b7e050b lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x606:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x428400010000 nid: local remote: 0x5aa6d504d91a19e3 expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826850.234628:0:33342:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-OST0001-osc-ffff985602bc9800 lock: 000000009edc46ab/0xb3f83dad1b7e053c lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x606:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x4c29400010020 nid: local remote: 0x5aa6d504d91a1a14 expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826850.234662:0:15163:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-OST0000-osc-ffff985602bc9800 lock: 000000005fe2ea3b/0xb3f83dad1b7e050b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x606:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x4c29400010020 nid: local remote: 0x5aa6d504d91a19e3 expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826850.234675:0:33342:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826850.234688:0:33342:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-OST0001-osc-ffff985602bc9800 lock: 000000009edc46ab/0xb3f83dad1b7e053c lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x606:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x4c29400010020 nid: local remote: 0x5aa6d504d91a1a14 expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826850.234745:0:15163:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826850.234759:0:15163:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-OST0000-osc-ffff985602bc9800 lock: 000000005fe2ea3b/0xb3f83dad1b7e050b lrc: 1/0,0 mode: --/PR res: [0x280000401:0x606:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x4c29400010020 nid: local remote: 0x5aa6d504d91a19e3 expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826850.234789:0:33342:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0001-osc-ffff985602bc9800 lock: 000000009edc46ab/0xb3f83dad1b7e053c lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x606:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x4c29400010020 nid: local remote: 0x5aa6d504d91a1a14 expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826850.234791:0:15163:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0000-osc-ffff985602bc9800 lock: 000000005fe2ea3b/0xb3f83dad1b7e050b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x606:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x4c29400010020 nid: local remote: 0x5aa6d504d91a19e3 expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1787826850.234801:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826850.234811:0:15164:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-OST0001-osc-ffff985602bc9800 lock: 00000000b69c5f27/0xb3f83dad1b7e04f6 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x604:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x420000010000 nid: local remote: 0x5aa6d504d91a19ce expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826850.234859:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826850.234868:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826850.234872:0:15164:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985610dc1c00 already unused, calling callback (ffffffffc13b01e0) 00010000:00010000:2.0:1787826850.234881:0:15164:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0001-osc-ffff985602bc9800 lock: 00000000b69c5f27/0xb3f83dad1b7e04f6 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x604:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x428400010000 nid: local remote: 0x5aa6d504d91a19ce expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826850.234885:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826850.234886:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826850.234896:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826850.234903:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826850.234917:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e4e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669225574912:192.168.203.114@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787826850.234928:0:15164:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-OST0001-osc-ffff985602bc9800 lock: 00000000b69c5f27/0xb3f83dad1b7e04f6 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x604:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x4c29400010020 nid: local remote: 0x5aa6d504d91a19ce expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000100:00100000:1.0:1787826850.234934:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610db4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669225574656:192.168.203.114@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787826850.234961:0:15164:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826850.234977:0:15164:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-OST0001-osc-ffff985602bc9800 lock: 00000000b69c5f27/0xb3f83dad1b7e04f6 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x604:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x4c29400010020 nid: local remote: 0x5aa6d504d91a19ce expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1787826850.234992:0:15164:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0001-osc-ffff985602bc9800 lock: 00000000b69c5f27/0xb3f83dad1b7e04f6 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x604:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x4c29400010020 nid: local remote: 0x5aa6d504d91a19ce expref: -99 pid: 79204 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000100:00100000:1.0:1787826850.235005:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e79e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669225575168:192.168.203.114@tcp:103:ldlm_bl.0 00000100:00100000:3.0:1787826850.235009:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669225575552:192.168.203.114@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787826850.235021:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826850.235037:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826850.235044:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826850.235058:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826850.235064:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826850.235076:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e7bc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669225575424:192.168.203.114@tcp:103:ldlm_bl.0 00000100:00100000:3.0:1787826850.235081:0:2386:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1787826850.235093:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e4e1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669225575296:192.168.203.114@tcp:103:ldlm_bl.0 00000100:00100000:3.0:1787826850.238988:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e4e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669225574912:192.168.203.114@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1787826850.239002:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610db4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669225574656:192.168.203.114@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1787826850.239022:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e79e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669225575168:192.168.203.114@tcp:103:ldlm_bl.0 00000100:00100000:3.0:1787826850.239042:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669225575552:192.168.203.114@tcp:103:ldlm_bl.0 00000100:00100000:3.0:1787826850.239070:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e4e1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669225575296:192.168.203.114@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1787826850.239077:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e7bc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669225575424:192.168.203.114@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826850.612852:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826851.636841:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826852.084815:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669225575808:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787826852.084832:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2385:1874669225575680:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787826852.084908:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669225575936:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787826852.084917:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee2d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669225576064:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787826852.084936:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669225576192:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787826852.086630:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669225575936:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787826852.086645:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669225576192:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787826852.086680:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee2d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669225576064:192.168.203.114@tcp:400:kworker.0 00000100:00080000:3.0:1787826852.086719:0:2386:0:(recover.c:205:ptlrpc_request_handle_notconn()) import lustre-MDT0000-mdc-ffff985602bc9800 of lustre-MDT0000_UUID@192.168.203.114@tcp abruptly disconnected: reconnecting 00000100:00080000:3.0:1787826852.086725:0:2386:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from FULL to DISCONN 00000100:02000400:3.0:1787826852.086767:0:2386:0:(import.c:182:ptlrpc_set_import_discon()) lustre-MDT0000-mdc-ffff985602bc9800: Connection to lustre-MDT0000 (at 192.168.203.114@tcp) was lost; in progress operations using this service will wait for recovery to complete 00000100:00080000:3.0:1787826852.086774:0:2386:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826852.086779:0:2386:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 2987 00000100:00080000:3.0:1787826852.086788:0:2386:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787826852.086800:0:2386:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787826852.086826:0:2386:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610e4e940 x1874669225576320/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826852.086939:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669225575808:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787826852.087097:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e4e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669225576320:192.168.203.114@tcp:38:ptlrpcd_01_01.0 00000100:00080000:1.0:1787826852.089339:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787826852.089364:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-19) 00000100:00100000:1.0:1787826852.089374:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e4e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669225576320:192.168.203.114@tcp:38:ptlrpcd_01_01.0 00010000:00010000:2.0:1787826852.661462:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826853.684997:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826855.732879:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826856.756893:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787826857.204830:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826857.204834:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787826857.204838:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1787826857.204842:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:3c946890-15ca-497a-b838-fe1cba61f4dd:2386:1874669225576448:192.168.203.114@tcp:400:kworker.0 00000100:00080000:2.0:1787826857.204845:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3002 00000100:00080000:2.0:1787826857.204856:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 5s 00000100:00080000:2.0:1787826857.204861:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787826857.204870:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787826857.204891:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985602ee3c00 x1874669225576576/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826857.204945:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669225576832:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787826857.204963:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee2d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669225576704:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787826857.204969:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669225576576:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787826857.205019:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669225576960:192.168.203.114@tcp:400:kworker.0 00000100:00080000:2.0:1787826857.206061:0:2386:0:(recover.c:205:ptlrpc_request_handle_notconn()) import MGC192.168.203.114@tcp of MGS@192.168.203.114@tcp abruptly disconnected: reconnecting 00000100:00080000:2.0:1787826857.206066:0:2386:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from FULL to DISCONN 00000100:02020000:2.0:1787826857.206070:0:2386:0:(import.c:186:ptlrpc_set_import_discon()) MGC192.168.203.114@tcp: Connection to MGS (at 192.168.203.114@tcp) was lost; in progress operations using this service will fail 00000100:00080000:2.0:1787826857.206074:0:2386:0:(import.c:192:ptlrpc_set_import_discon()) import MGS@192.168.203.114@tcp for MGC192.168.203.114@tcp not replayable, auto-deactivating 00000100:00080000:2.0:1787826857.206077:0:2386:0:(import.c:138:ptlrpc_deactivate_import_nolock()) setting import MGS INVALID 00000100:00100000:2.0:1787826857.206083:0:2386:0:(client.c:3589:ptlrpc_abort_inflight()) @@@ inflight req@ffff985602ee1680 x1874669225575680/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787826868 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826857.206095:0:2386:0:(client.c:3589:ptlrpc_abort_inflight()) @@@ inflight req@ffff985602ee2580 x1874669225576448/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787826873 ref 1 fl Rpc:RNQU/200/0 rc 0/-107 job:'kworker.0' uid:0 gid:0 projid:4294967295 10000000:01000000:2.0:1787826857.206105:0:2386:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808001 00000100:00080000:2.0:1787826857.206108:0:2386:0:(pinger.c:193:ptlrpc_pinger_ir_down()) IR down 10000000:01000000:2.0:1787826857.206109:0:2386:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808002 00000100:00080000:2.0:1787826857.206112:0:2386:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1787826857.206116:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee2d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669225576704:192.168.203.114@tcp:400:kworker.0 00000100:00080000:2.0:1787826857.206119:0:2386:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 2987 00000100:00100000:3.0:1787826857.206128:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2385:1874669225575680:192.168.203.114@tcp:400:kworker.0 00000100:00080000:2.0:1787826857.206130:0:2386:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787826857.206166:0:2386:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985602ee2940 x1874669225577088/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826857.206193:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:3c946890-15ca-497a-b838-fe1cba61f4dd:2386:1874669225576448:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787826857.206221:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669225577088:192.168.203.114@tcp:250:ptlrpcd_01_01.0 00000100:00100000:2.0:1787826857.206227:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669225576832:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787826857.206567:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669225576960:192.168.203.114@tcp:400:kworker.0 00000100:00080000:1.0:1787826857.207063:0:2382:0:(import.c:1132:ptlrpc_connect_interpret()) MGC192.168.203.114@tcp: connect to target with instance 0 10000000:01000000:1.0:1787826857.207070:0:2382:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808005 00000100:02000400:1.0:1787826857.207080:0:2382:0:(import.c:1284:ptlrpc_connect_interpret()) Evicted from MGS (at 192.168.203.114@tcp) after server handle changed from 0x5aa6d504d91a176d to 0x5aa6d504d91a206c 00000100:00080000:1.0:1787826857.207084:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to EVICTED 00000100:00080000:1.0:1787826857.207089:0:2382:0:(import.c:1674:ptlrpc_import_recovery_state_machine()) evicted from MGS@192.168.203.114@tcp; invalidating 00000100:00100000:1.0:1787826857.207232:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669225577088:192.168.203.114@tcp:250:ptlrpcd_01_01.0 00000100:00080000:3.0:1787826857.207252:0:80149:0:(import.c:1598:ptlrpc_invalidate_import_thread()) thread invalidate import MGC192.168.203.114@tcp to MGS@192.168.203.114@tcp 00000100:00100000:3.0:1787826857.207256:0:80149:0:(import.c:323:ptlrpc_invalidate_import()) Sleeping 20s for inflight to error out 10000000:01000000:3.0:1787826857.207259:0:80149:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808003 00010000:00010000:3.0:1787826857.207271:0:80149:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.114@tcp lock: ffff985642051800/0xb3f83dad1b7e042b lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x110e400000000 nid: local remote: 0x5aa6d504d91a179e expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:3.0:1787826857.207282:0:80149:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.114@tcp lock: ffff985642051800/0xb3f83dad1b7e042b lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x110f400000000 nid: local remote: 0x5aa6d504d91a179e expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1787826857.207346:0:80149:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:3.0:1787826857.207361:0:80149:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826857.207371:0:80149:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.114@tcp lock: ffff985642051800/0xb3f83dad1b7e042b lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x590f400000000 nid: local remote: 0x5aa6d504d91a179e expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826857.207386:0:80149:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.114@tcp lock: ffff985610dc0200/0xb3f83dad1b7e0439 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x110e400000000 nid: local remote: 0x5aa6d504d91a17ac expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:3.0:1787826857.207398:0:80149:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.114@tcp lock: ffff985610dc0200/0xb3f83dad1b7e0439 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x110f400000000 nid: local remote: 0x5aa6d504d91a17ac expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1787826857.207404:0:80149:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00080000:1.0:1787826857.207415:0:2382:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=46162) req@ffff985602ee3c00 x1874669225576576/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/416 e 0 to 0 dl 1787826872 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00010000:3.0:1787826857.207416:0:80149:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826857.207421:0:80149:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.114@tcp lock: ffff985610dc0200/0xb3f83dad1b7e0439 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x590f400000000 nid: local remote: 0x5aa6d504d91a17ac expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1787826857.207437:0:2382:0:(import.c:1132:ptlrpc_connect_interpret()) lustre-MDT0000-mdc-ffff985602bc9800: connect to target with instance 47 00010000:00010000:3.0:1787826857.207445:0:80149:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.114@tcp lock: ffff985610e06e00/0xb3f83dad1b7e0424 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x110e400000000 nid: local remote: 0x5aa6d504d91a1797 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:3.0:1787826857.207454:0:80149:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.114@tcp lock: ffff985610e06e00/0xb3f83dad1b7e0424 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x110f400000000 nid: local remote: 0x5aa6d504d91a1797 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1787826857.207456:0:2382:0:(import.c:1338:ptlrpc_connect_interpret()) @@@ lustre-MDT0000-mdc-ffff985602bc9800: evicting (reconnect/recover flags not set: 4) req@ffff985602ee3c00 x1874669225576576/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/416 e 0 to 0 dl 1787826872 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 10000000:01000000:3.0:1787826857.207470:0:80149:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x2:0x0].0x0 (lustre) 00000100:00080000:1.0:1787826857.207474:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to EVICTED 00000100:02020000:1.0:1787826857.207479:0:2382:0:(import.c:1663:ptlrpc_import_recovery_state_machine()) lustre-MDT0000-mdc-ffff985602bc9800: This client was evicted by lustre-MDT0000; in progress operations using this service will fail. 00010000:00010000:3.0:1787826857.207484:0:80149:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826857.207489:0:80149:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.114@tcp lock: ffff985610e06e00/0xb3f83dad1b7e0424 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x590f400000000 nid: local remote: 0x5aa6d504d91a1797 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826857.207504:0:80149:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.114@tcp lock: ffff985610dc1e00/0xb3f83dad1b7e0432 lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x110e400000000 nid: local remote: 0x5aa6d504d91a17a5 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:3.0:1787826857.207512:0:80149:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.114@tcp lock: ffff985610dc1e00/0xb3f83dad1b7e0432 lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x110f400000000 nid: local remote: 0x5aa6d504d91a17a5 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1787826857.207517:0:80149:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x736d61726170:0x3:0x0].0x0 (params) 00010000:00010000:3.0:1787826857.207525:0:80149:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826857.207530:0:80149:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.114@tcp lock: ffff985610dc1e00/0xb3f83dad1b7e0432 lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x590f400000000 nid: local remote: 0x5aa6d504d91a17a5 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826857.207670:0:2377:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.114@tcp (source: 12345-192.168.203.114@tcp) 00000100:00080000:3.0:1787826857.207718:0:80149:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from EVICTED to RECOVER 00000100:00100000:2.0:1787826857.207724:0:15155:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874669235529472 00000100:00080000:3.0:1787826857.207724:0:80149:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from RECOVER to FULL 10000000:01000000:3.0:1787826857.207728:0:80149:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808004 00000100:00100000:2.0:1787826857.207729:0:15155:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874669235529472 00000100:00100000:2.0:1787826857.207773:0:15155:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.114@tcp, seq: 253 00000100:00080000:3.0:1787826857.207775:0:80149:0:(pinger.c:186:ptlrpc_pinger_ir_up()) IR up 00000100:02000000:3.0:1787826857.207781:0:80149:0:(import.c:1743:ptlrpc_import_recovery_state_machine()) MGC192.168.203.114@tcp: Connection restored to 192.168.203.114@tcp (at 192.168.203.114@tcp) 00000100:00100000:2.0:1787826857.207787:0:15155:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff985602ee0b40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb01_000:LOV_OSC_UUID+4:43926:x1874669235529472:12345-192.168.203.114@tcp:104: 00010000:00010000:2.0:1787826857.207801:0:15155:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-OST0000-osc-ffff985602bc9800 lock: 00000000eb029571/0xb3f83dad1b7e08ca lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x607:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800420000010000 nid: local remote: 0x5aa6d504d91a1c2f expref: -99 pid: 79778 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826857.207861:0:15155:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:2.0:1787826857.207870:0:15155:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff985602ee0b40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb01_000:LOV_OSC_UUID+4:43926:x1874669235529472:12345-192.168.203.114@tcp:104: Request processed in 87us (201us total) trans 0 rc 0/0 00000100:00100000:2.0:1787826857.207878:0:15155:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.114@tcp, seq: 253 00000100:00080000:1.0:1787826857.212823:0:2382:0:(import.c:1674:ptlrpc_import_recovery_state_machine()) evicted from lustre-MDT0000_UUID@192.168.203.114@tcp; invalidating 00000100:00100000:1.0:1787826857.213012:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669225576576:192.168.203.114@tcp:38:kworker.0 00000100:00080000:3.0:1787826857.213038:0:80150:0:(import.c:1598:ptlrpc_invalidate_import_thread()) thread invalidate import lustre-MDT0000-mdc-ffff985602bc9800 to lustre-MDT0000_UUID@192.168.203.114@tcp 00000100:00080000:3.0:1787826857.213045:0:80150:0:(import.c:138:ptlrpc_deactivate_import_nolock()) setting import lustre-MDT0000_UUID INVALID 00000100:00100000:3.0:1787826857.213049:0:80150:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 197568495648 gen 7 00000100:00100000:3.0:1787826857.213061:0:80150:0:(client.c:3163:ptlrpc_free_committed()) @@@ free request with old gen req@ffff985610e4de00 x1874669225574016/t197568495649(197568495649) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 528/448 e 0 to 0 dl 1787826863 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826857.213084:0:80150:0:(client.c:3163:ptlrpc_free_committed()) @@@ free request with old gen req@ffff985610db70c0 x1874669225574528/t197568495650(197568495650) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787826863 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826857.213096:0:80150:0:(client.c:3213:ptlrpc_free_committed()) @@@ free stale open request req@ffff985610db43c0 x1874669225571968/t197568495647(197568495647) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826862 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00000100:00100000:3.0:1787826857.213116:0:80150:0:(import.c:323:ptlrpc_invalidate_import()) Sleeping 20s for inflight to error out 00000100:00080000:3.0:1787826857.215407:0:80150:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from EVICTED to RECOVER 00000100:00080000:3.0:1787826857.215414:0:80150:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from RECOVER to FULL 00010000:00010000:2.0:1787826857.780907:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826858.294247:0:80177:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e7b840 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:80177:1874669225577984:192.168.203.114@tcp:13:lfs.0 00020000:00000004:1.0:1787826858.294258:0:80177:0:(obd_class.h:1117:obd_statfs()) lustre-OST0001-osc-ffff985602bc9800: update ffff98560952d1c0 cache blocks 901363/958279 objects 261735/262144 00000080:00200080:1.0:1787826858.294282:0:80177:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe22344700 00020000:00000080:1.0:1787826858.294290:0:80177:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605577400 uarg=0000000000000000 00000080:00200080:1.0:1787826858.294296:0:80177:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe22344700 00020000:00000080:1.0:1787826858.294299:0:80177:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605577400 uarg=0000000000000000 00000080:00200080:1.0:1787826858.294302:0:80177:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe22344700 00020000:00000080:1.0:1787826858.294305:0:80177:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605577400 uarg=0000000000000000 00000080:00200080:1.0:1787826858.294309:0:80177:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe22344700 00020000:00000080:1.0:1787826858.294312:0:80177:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605577400 uarg=0000000000000000 00000080:00200080:1.0:1787826858.294316:0:80177:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe22344700 00020000:00000080:1.0:1787826858.294319:0:80177:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605577400 uarg=0000000000000000 00000080:00200080:1.0:1787826858.294322:0:80177:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe22344700 00020000:00000080:1.0:1787826858.294324:0:80177:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605577400 uarg=0000000000000000 00000080:00200080:1.0:1787826858.294327:0:80177:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe22344700 00020000:00000080:1.0:1787826858.294329:0:80177:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605577400 uarg=0000000000000000 00000080:00200080:1.0:1787826858.294331:0:80177:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe22344700 00020000:00000080:1.0:1787826858.294333:0:80177:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605577400 uarg=0000000000000000 00000080:00200080:1.0:1787826858.294336:0:80177:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe22344700 00020000:00000080:1.0:1787826858.294338:0:80177:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605577400 uarg=0000000000000000 00000080:00200080:1.0:1787826858.294341:0:80177:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe22344700 00020000:00000080:1.0:1787826858.294347:0:80177:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605577400 uarg=0000000000000000 00000080:00200080:1.0:1787826858.294350:0:80177:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe22344700 00020000:00000080:1.0:1787826858.294353:0:80177:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605577400 uarg=0000000000000000 00000080:00200080:1.0:1787826858.294355:0:80177:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe22344700 00020000:00000080:1.0:1787826858.294358:0:80177:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605577400 uarg=0000000000000000 00000080:00200080:1.0:1787826858.294361:0:80177:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe22344700 00020000:00000080:1.0:1787826858.294363:0:80177:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605577400 uarg=0000000000000000 00000080:00200080:1.0:1787826858.294365:0:80177:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe22344700 00020000:00000080:1.0:1787826858.294367:0:80177:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605577400 uarg=0000000000000000 00000080:00200080:1.0:1787826858.294370:0:80177:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe22344700 00020000:00000080:1.0:1787826858.294372:0:80177:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605577400 uarg=0000000000000000 00000080:00200080:1.0:1787826858.294374:0:80177:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe22344700 00020000:00000080:1.0:1787826858.294377:0:80177:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605577400 uarg=0000000000000000 00000080:00200080:1.0:1787826858.294381:0:80177:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe22344700 00020000:00000080:1.0:1787826858.294386:0:80177:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605577400 uarg=0000000000000000 00000080:00200080:1.0:1787826858.294391:0:80177:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe22344700 00020000:00000080:1.0:1787826858.294394:0:80177:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605577400 uarg=0000000000000000 00000080:00200080:1.0:1787826858.294399:0:80177:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe22344700 00020000:00000080:1.0:1787826858.294403:0:80177:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605577400 uarg=0000000000000000 00000080:00200080:1.0:1787826858.294408:0:80177:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe22344700 00020000:00000080:1.0:1787826858.294413:0:80177:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605577400 uarg=0000000000000000 00000080:00200080:1.0:1787826858.294417:0:80177:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe22344700 00020000:00000080:1.0:1787826858.294421:0:80177:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605577400 uarg=0000000000000000 00000080:00200080:1.0:1787826858.294426:0:80177:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe22344700 00020000:00000080:1.0:1787826858.294431:0:80177:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605577400 uarg=0000000000000000 00000080:00200080:1.0:1787826858.294436:0:80177:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe22344700 00020000:00000080:1.0:1787826858.294438:0:80177:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605577400 uarg=0000000000000000 00000080:00200080:1.0:1787826858.294441:0:80177:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe22344700 00020000:00000080:1.0:1787826858.294443:0:80177:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605577400 uarg=0000000000000000 00000080:00200080:1.0:1787826858.294446:0:80177:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe22344700 00020000:00000080:1.0:1787826858.294449:0:80177:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605577400 uarg=0000000000000000 00000080:00200080:1.0:1787826858.294451:0:80177:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe22344700 00020000:00000080:1.0:1787826858.294454:0:80177:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605577400 uarg=0000000000000000 00000080:00200080:1.0:1787826858.294456:0:80177:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe22344700 00020000:00000080:1.0:1787826858.294458:0:80177:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605577400 uarg=0000000000000000 00000080:00200080:1.0:1787826858.294461:0:80177:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe22344700 00020000:00000080:1.0:1787826858.294463:0:80177:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605577400 uarg=0000000000000000 00000080:00200080:1.0:1787826858.294465:0:80177:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe22344700 00020000:00000080:1.0:1787826858.294466:0:80177:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605577400 uarg=0000000000000000 00000080:00200080:1.0:1787826858.294470:0:80177:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe22344700 00020000:00000080:1.0:1787826858.294471:0:80177:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605577400 uarg=0000000000000000 00000080:00200080:1.0:1787826858.294474:0:80177:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe22344700 00020000:00000080:1.0:1787826858.294476:0:80177:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605577400 uarg=0000000000000000 00000080:00200080:1.0:1787826858.294479:0:80177:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe22344700 00020000:00000080:1.0:1787826858.294480:0:80177:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605577400 uarg=0000000000000000 00000080:00200080:1.0:1787826858.294483:0:80177:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe22344700 00020000:00000080:1.0:1787826858.294485:0:80177:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605577400 uarg=0000000000000000 00000080:00200080:1.0:1787826858.294487:0:80177:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe22344700 00020000:00000080:1.0:1787826858.294489:0:80177:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605577400 uarg=0000000000000000 00000080:00200080:1.0:1787826858.294493:0:80177:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe22344700 00020000:00000080:1.0:1787826858.294497:0:80177:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605577400 uarg=0000000000000000 00000080:00200080:1.0:1787826858.294502:0:80177:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe22344700 00020000:00000080:1.0:1787826858.294505:0:80177:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605577400 uarg=0000000000000000 00000080:00200080:1.0:1787826858.294510:0:80177:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe22344700 00020000:00000080:1.0:1787826858.294512:0:80177:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605577400 uarg=0000000000000000 00000080:00200080:1.0:1787826858.294516:0:80177:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe22344700 00020000:00000080:1.0:1787826858.294519:0:80177:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605577400 uarg=0000000000000000 00000080:00200080:1.0:1787826858.294523:0:80177:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe22344700 00020000:00000080:1.0:1787826858.294528:0:80177:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605577400 uarg=0000000000000000 00000080:00200080:1.0:1787826858.294532:0:80177:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe22344700 00020000:00000080:1.0:1787826858.294536:0:80177:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605577400 uarg=0000000000000000 00000080:00200080:1.0:1787826858.294541:0:80177:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe22344700 00020000:00000080:1.0:1787826858.294544:0:80177:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605577400 uarg=0000000000000000 00000080:00200080:1.0:1787826858.294547:0:80177:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe22344700 00020000:00000080:1.0:1787826858.294550:0:80177:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605577400 uarg=0000000000000000 00000080:00200080:1.0:1787826858.294552:0:80177:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe22344700 00020000:00000080:1.0:1787826858.294556:0:80177:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605577400 uarg=0000000000000000 00000080:00200080:1.0:1787826858.294560:0:80177:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe22344700 00020000:00000080:1.0:1787826858.294564:0:80177:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605577400 uarg=0000000000000000 00000080:00200080:1.0:1787826858.294568:0:80177:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe22344700 00020000:00000080:1.0:1787826858.294572:0:80177:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605577400 uarg=0000000000000000 00000080:00200080:1.0:1787826858.294576:0:80177:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe22344700 00020000:00000080:1.0:1787826858.294580:0:80177:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605577400 uarg=0000000000000000 00000080:00200080:1.0:1787826858.294584:0:80177:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe22344700 00020000:00000080:1.0:1787826858.294587:0:80177:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605577400 uarg=0000000000000000 00000080:00200080:1.0:1787826858.294591:0:80177:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe22344700 00020000:00000080:1.0:1787826858.294594:0:80177:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605577400 uarg=0000000000000000 00000080:00200080:1.0:1787826858.294598:0:80177:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe22344700 00020000:00000080:1.0:1787826858.294602:0:80177:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605577400 uarg=0000000000000000 00000080:00200080:1.0:1787826858.294604:0:80177:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe22344700 00020000:00000080:1.0:1787826858.294606:0:80177:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605577400 uarg=0000000000000000 00000080:00200080:1.0:1787826858.294608:0:80177:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe22344700 00020000:00000080:1.0:1787826858.294610:0:80177:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605577400 uarg=0000000000000000 00000080:00200080:1.0:1787826858.294612:0:80177:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe22344700 00020000:00000080:1.0:1787826858.294614:0:80177:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605577400 uarg=0000000000000000 00000080:00200080:1.0:1787826858.294616:0:80177:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe22344700 00020000:00000080:1.0:1787826858.294618:0:80177:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605577400 uarg=0000000000000000 00000080:00200080:1.0:1787826858.294620:0:80177:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe22344700 00020000:00000080:1.0:1787826858.294621:0:80177:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605577400 uarg=0000000000000000 00000080:00200080:1.0:1787826858.294624:0:80177:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe22344700 00020000:00000080:1.0:1787826858.294626:0:80177:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605577400 uarg=0000000000000000 00000080:00200080:1.0:1787826858.294628:0:80177:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe22344700 00020000:00000080:1.0:1787826858.294630:0:80177:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605577400 uarg=0000000000000000 00000080:00200080:1.0:1787826858.294632:0:80177:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe22344700 00020000:00000080:1.0:1787826858.294634:0:80177:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605577400 uarg=0000000000000000 00000080:00200080:1.0:1787826858.294636:0:80177:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe22344700 00020000:00000080:1.0:1787826858.294637:0:80177:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605577400 uarg=0000000000000000 00000080:00200080:1.0:1787826858.294639:0:80177:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe22344700 00020000:00000080:1.0:1787826858.294641:0:80177:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605577400 uarg=0000000000000000 00000080:00200080:1.0:1787826858.294643:0:80177:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe22344700 00020000:00000080:1.0:1787826858.294644:0:80177:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605577400 uarg=0000000000000000 00000080:00200080:1.0:1787826858.294646:0:80177:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe22344700 00020000:00000080:1.0:1787826858.294648:0:80177:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605577400 uarg=0000000000000000 00000080:00200080:1.0:1787826858.294650:0:80177:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe22344700 00020000:00000080:1.0:1787826858.294652:0:80177:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605577400 uarg=0000000000000000 00000080:00200080:1.0:1787826858.294654:0:80177:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe22344700 00020000:00000080:1.0:1787826858.294655:0:80177:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605577400 uarg=0000000000000000 00000080:00200008:1.0:1787826858.294674:0:80177:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 100000 00000100:00100000:1.0:1787826858.295877:0:15166:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e7ad00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669225578112:192.168.203.114@tcp:256:ll_cfg_requeue.0 10000000:01000000:1.0:1787826858.295988:0:15166:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.114@tcp: configuration from log 'lustre-cliir' succeeded (0). 00010000:00010000:1.0:1787826858.295996:0:15166:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.114@tcp lock: ffff985609eb0800/0xb3f83dad1b7e08ed lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x5aa6d504d91a2088 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826858.296006:0:15166:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.114@tcp lock: ffff985609eb0800/0xb3f83dad1b7e08ed lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x5aa6d504d91a2088 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787826858.296016:0:15166:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 10000000:01000000:1.0:1787826858.296020:0:15166:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff985602bc9800 from 52 10000000:01000000:1.0:1787826858.296022:0:15166:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00010000:00010000:1.0:1787826858.296048:0:15166:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.114@tcp lock: ffff985642051000/0xb3f83dad1b7e08f4 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826858.296056:0:15166:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.114@tcp lock: ffff985642051000/0xb3f83dad1b7e08f4 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826858.296065:0:15166:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.114@tcp lock: ffff985642051000/0xb3f83dad1b7e08f4 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826858.296084:0:15166:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610db4780 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669225578240:192.168.203.114@tcp:101:ll_cfg_requeue.0 00000100:00100000:1.0:1787826858.296143:0:15166:0:(client.c:2715:ptlrpc_set_wait()) set ffff985611f30b40 going to sleep for 11 seconds 00000100:00100000:2.0:1787826858.296876:0:15166:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610db4780 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669225578240:192.168.203.114@tcp:101:ll_cfg_requeue.0 00010000:00010000:2.0:1787826858.296891:0:15166:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.114@tcp lock: ffff985642051000/0xb3f83dad1b7e08f4 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x5aa6d504d91a208f expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826858.296899:0:15166:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.114@tcp lock: ffff985642051000/0xb3f83dad1b7e08f4 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x5aa6d504d91a208f expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826858.296940:0:15166:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee2580 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669225578368:192.168.203.114@tcp:501:ll_cfg_requeue.0 00000100:00100000:2.0:1787826858.296986:0:15166:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607ee2d80 going to sleep for 11 seconds 00000020:01000000:3.0:1787826858.299452:0:15166:0:(obd_config.c:2145:class_config_parse_llog()) Processed log lustre-client gen 52-51 (rc=0) 10000000:01000000:3.0:1787826858.299469:0:15166:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.114@tcp: configuration from log 'lustre-client' succeeded (0). 00010000:00010000:3.0:1787826858.299476:0:15166:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.114@tcp lock: ffff985642051000/0xb3f83dad1b7e08f4 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x5aa6d504d91a208f expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826858.299483:0:15166:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.114@tcp lock: ffff985642051000/0xb3f83dad1b7e08f4 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x5aa6d504d91a208f expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1787826858.299494:0:15166:0:(mgc_request.c:611:do_requeue()) updating log params 10000000:01000000:3.0:1787826858.299497:0:15166:0:(mgc_request.c:1802:mgc_process_log()) Process log params-ffff985602bc9800 from 7 10000000:01000000:3.0:1787826858.299499:0:15166:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for params (res 0x736d61726170) 00010000:00010000:3.0:1787826858.299520:0:15166:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.114@tcp lock: ffff985610e06400/0xb3f83dad1b7e08fb lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826858.299526:0:15166:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.114@tcp lock: ffff985610e06400/0xb3f83dad1b7e08fb lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826858.299532:0:15166:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.114@tcp lock: ffff985610e06400/0xb3f83dad1b7e08fb lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826858.299545:0:15166:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e4e1c0 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669225578624:192.168.203.114@tcp:101:ll_cfg_requeue.0 00000100:00100000:3.0:1787826858.299587:0:15166:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c70f00 going to sleep for 11 seconds 00000040:00080000:3.0:1787826858.303533:0:80196:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x2:0x0] index 64768 count 7 00000080:00200008:3.0:1787826859.569198:0:80231:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:3.0:1787826859.569211:0:80231:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:3.0:1787826859.569231:0:80231:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826859.569249:0:80231:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1000/0xb3f83dad1b7e08e6 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a2081 expref: -99 pid: 80177 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:3.0:1787826859.569264:0:80231:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1000/0xb3f83dad1b7e08e6 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a2081 expref: -99 pid: 80177 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:3.0:1787826859.569272:0:80231:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826859.569279:0:80231:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:3.0:1787826859.569282:0:80231:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e08e6 00000080:00010000:3.0:1787826859.569289:0:80231:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e08e6 from it ffffab3ac0d93cb8 00010000:00010000:3.0:1787826859.569294:0:80231:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1000/0xb3f83dad1b7e08e6 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a2081 expref: -99 pid: 80177 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:3.0:1787826859.569303:0:80231:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1000/0xb3f83dad1b7e08e6 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a2081 expref: -99 pid: 80177 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00000080:00200008:3.0:1787826859.569818:0:80231:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 0, flags 0, foreign 0 00000080:00200000:3.0:1787826859.569823:0:80231:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:3.0:1787826859.569832:0:80231:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826859.569840:0:80231:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1000/0xb3f83dad1b7e08e6 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a2081 expref: -99 pid: 80177 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00010000:00010000:3.0:1787826859.569850:0:80231:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1000/0xb3f83dad1b7e08e6 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a2081 expref: -99 pid: 80177 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00000080:00200000:3.0:1787826859.569856:0:80231:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826859.569861:0:80231:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:3.0:1787826859.569864:0:80231:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e08e6 00000080:00010000:3.0:1787826859.569868:0:80231:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e08e6 from it ffffab3ac0d93cb8 00010000:00010000:3.0:1787826859.569871:0:80231:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1000/0xb3f83dad1b7e08e6 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a2081 expref: -99 pid: 80177 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00010000:00010000:3.0:1787826859.569878:0:80231:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1000/0xb3f83dad1b7e08e6 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a2081 expref: -99 pid: 80177 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00000080:00200000:3.0:1787826859.569958:0:80231:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:3.0:1787826859.569980:0:80231:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826859.569988:0:80231:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1000/0xb3f83dad1b7e08e6 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a2081 expref: -99 pid: 80177 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00010000:00010000:3.0:1787826859.569997:0:80231:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1000/0xb3f83dad1b7e08e6 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a2081 expref: -99 pid: 80177 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00000080:00200000:3.0:1787826859.570003:0:80231:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826859.570008:0:80231:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:3.0:1787826859.570010:0:80231:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e08e6 00000080:00010000:3.0:1787826859.570013:0:80231:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e08e6 from it ffffab3ac0d93bb0 00010000:00010000:3.0:1787826859.570017:0:80231:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1000/0xb3f83dad1b7e08e6 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a2081 expref: -99 pid: 80177 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00010000:00010000:3.0:1787826859.570024:0:80231:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1000/0xb3f83dad1b7e08e6 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a2081 expref: -99 pid: 80177 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00000080:00200000:3.0:1787826859.570033:0:80231:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 44 00000080:00200008:3.0:1787826859.570040:0:80231:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 100000 00000080:00200080:3.0:1787826859.570128:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00800000:00000080:3.0:1787826859.570142:0:80231:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00800000:00000004:3.0:1787826859.570149:0:80231:0:(obd_class.h:1091:obd_statfs()) lustre-MDT0000-mdc-ffff985602bc9800: age 3009, max_age 3009 00800000:00000004:3.0:1787826859.570153:0:80231:0:(obd_class.h:1131:obd_statfs()) lustre-MDT0000-mdc-ffff985602bc9800: use ffff985606a337f0 cache blocks 317372/353529 objects 1023692/1024000 00000080:00200080:3.0:1787826859.570176:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00800000:00000080:3.0:1787826859.570182:0:80231:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00800000:00000004:3.0:1787826859.570187:0:80231:0:(obd_class.h:1091:obd_statfs()) lustre-MDT0001-mdc-ffff985602bc9800: age 3009, max_age 3009 00800000:00000004:3.0:1787826859.570189:0:80231:0:(obd_class.h:1131:obd_statfs()) lustre-MDT0001-mdc-ffff985602bc9800: use ffff98560952b7f0 cache blocks 317417/353529 objects 1023739/1024000 00000080:00200080:3.0:1787826859.570203:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00800000:00000080:3.0:1787826859.570208:0:80231:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570216:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00800000:00000080:3.0:1787826859.570220:0:80231:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570223:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00800000:00000080:3.0:1787826859.570226:0:80231:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570228:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00800000:00000080:3.0:1787826859.570230:0:80231:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570232:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00800000:00000080:3.0:1787826859.570234:0:80231:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570236:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00800000:00000080:3.0:1787826859.570238:0:80231:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570242:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00800000:00000080:3.0:1787826859.570246:0:80231:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570250:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00800000:00000080:3.0:1787826859.570253:0:80231:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570257:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00800000:00000080:3.0:1787826859.570260:0:80231:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570263:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00800000:00000080:3.0:1787826859.570268:0:80231:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570272:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00800000:00000080:3.0:1787826859.570276:0:80231:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570280:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00800000:00000080:3.0:1787826859.570283:0:80231:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570288:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00800000:00000080:3.0:1787826859.570292:0:80231:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570296:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00800000:00000080:3.0:1787826859.570301:0:80231:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570305:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00800000:00000080:3.0:1787826859.570309:0:80231:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570314:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00800000:00000080:3.0:1787826859.570318:0:80231:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570322:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00800000:00000080:3.0:1787826859.570327:0:80231:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570331:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00800000:00000080:3.0:1787826859.570335:0:80231:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570339:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00800000:00000080:3.0:1787826859.570342:0:80231:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570346:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00800000:00000080:3.0:1787826859.570349:0:80231:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570353:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00800000:00000080:3.0:1787826859.570357:0:80231:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570360:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00800000:00000080:3.0:1787826859.570363:0:80231:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570367:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00800000:00000080:3.0:1787826859.570371:0:80231:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570375:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00800000:00000080:3.0:1787826859.570379:0:80231:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570383:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00800000:00000080:3.0:1787826859.570387:0:80231:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570392:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00800000:00000080:3.0:1787826859.570395:0:80231:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570400:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00800000:00000080:3.0:1787826859.570403:0:80231:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570406:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00800000:00000080:3.0:1787826859.570410:0:80231:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570413:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00800000:00000080:3.0:1787826859.570417:0:80231:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570421:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00800000:00000080:3.0:1787826859.570424:0:80231:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570428:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00800000:00000080:3.0:1787826859.570432:0:80231:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570436:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00800000:00000080:3.0:1787826859.570440:0:80231:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570444:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00800000:00000080:3.0:1787826859.570448:0:80231:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570452:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00800000:00000080:3.0:1787826859.570455:0:80231:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570459:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00800000:00000080:3.0:1787826859.570463:0:80231:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570467:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00800000:00000080:3.0:1787826859.570470:0:80231:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570473:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00800000:00000080:3.0:1787826859.570475:0:80231:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570477:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00800000:00000080:3.0:1787826859.570479:0:80231:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570481:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00800000:00000080:3.0:1787826859.570485:0:80231:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570490:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00800000:00000080:3.0:1787826859.570494:0:80231:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570498:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00800000:00000080:3.0:1787826859.570502:0:80231:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570506:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00800000:00000080:3.0:1787826859.570510:0:80231:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570513:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00800000:00000080:3.0:1787826859.570517:0:80231:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570522:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00800000:00000080:3.0:1787826859.570526:0:80231:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570531:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00800000:00000080:3.0:1787826859.570534:0:80231:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570538:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00800000:00000080:3.0:1787826859.570541:0:80231:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570545:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00800000:00000080:3.0:1787826859.570548:0:80231:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570553:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00800000:00000080:3.0:1787826859.570557:0:80231:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570561:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00800000:00000080:3.0:1787826859.570565:0:80231:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570569:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00800000:00000080:3.0:1787826859.570572:0:80231:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570577:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00800000:00000080:3.0:1787826859.570580:0:80231:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570586:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00800000:00000080:3.0:1787826859.570590:0:80231:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570594:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00800000:00000080:3.0:1787826859.570599:0:80231:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570603:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00800000:00000080:3.0:1787826859.570606:0:80231:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570612:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00800000:00000080:3.0:1787826859.570616:0:80231:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570620:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00800000:00000080:3.0:1787826859.570623:0:80231:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570628:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00800000:00000080:3.0:1787826859.570631:0:80231:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570635:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00800000:00000080:3.0:1787826859.570639:0:80231:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570643:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00800000:00000080:3.0:1787826859.570646:0:80231:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570650:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00800000:00000080:3.0:1787826859.570654:0:80231:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570659:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00800000:00000080:3.0:1787826859.570662:0:80231:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570667:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00800000:00000080:3.0:1787826859.570671:0:80231:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570675:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00800000:00000080:3.0:1787826859.570680:0:80231:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570685:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00020000:00000080:3.0:1787826859.570711:0:80231:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00020000:00000004:3.0:1787826859.570719:0:80231:0:(obd_class.h:1091:obd_statfs()) lustre-OST0000-osc-ffff985602bc9800: age 3009, max_age 3009 00020000:00000004:3.0:1787826859.570723:0:80231:0:(obd_class.h:1131:obd_statfs()) lustre-OST0000-osc-ffff985602bc9800: use ffff985609528450 cache blocks 901357/958279 objects 261728/262144 00000080:00200080:3.0:1787826859.570747:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00020000:00000080:3.0:1787826859.570753:0:80231:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00020000:00000004:3.0:1787826859.570757:0:80231:0:(obd_class.h:1091:obd_statfs()) lustre-OST0001-osc-ffff985602bc9800: age 3009, max_age 3009 00020000:00000004:3.0:1787826859.570758:0:80231:0:(obd_class.h:1131:obd_statfs()) lustre-OST0001-osc-ffff985602bc9800: use ffff98560952d1c0 cache blocks 901363/958279 objects 261735/262144 00000080:00200080:3.0:1787826859.570769:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00020000:00000080:3.0:1787826859.570771:0:80231:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570774:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00020000:00000080:3.0:1787826859.570776:0:80231:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570779:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00020000:00000080:3.0:1787826859.570782:0:80231:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570786:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00020000:00000080:3.0:1787826859.570788:0:80231:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570791:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00020000:00000080:3.0:1787826859.570794:0:80231:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570796:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00020000:00000080:3.0:1787826859.570798:0:80231:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570800:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00020000:00000080:3.0:1787826859.570802:0:80231:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570804:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00020000:00000080:3.0:1787826859.570806:0:80231:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570810:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00020000:00000080:3.0:1787826859.570813:0:80231:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570818:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00020000:00000080:3.0:1787826859.570822:0:80231:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570827:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00020000:00000080:3.0:1787826859.570831:0:80231:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570836:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00020000:00000080:3.0:1787826859.570841:0:80231:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570846:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00020000:00000080:3.0:1787826859.570851:0:80231:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570857:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00020000:00000080:3.0:1787826859.570862:0:80231:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570868:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00020000:00000080:3.0:1787826859.570873:0:80231:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570878:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00020000:00000080:3.0:1787826859.570881:0:80231:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570884:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00020000:00000080:3.0:1787826859.570887:0:80231:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570889:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00020000:00000080:3.0:1787826859.570891:0:80231:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570893:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00020000:00000080:3.0:1787826859.570895:0:80231:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570897:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00020000:00000080:3.0:1787826859.570899:0:80231:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570902:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00020000:00000080:3.0:1787826859.570905:0:80231:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570907:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00020000:00000080:3.0:1787826859.570911:0:80231:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570913:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00020000:00000080:3.0:1787826859.570915:0:80231:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570917:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00020000:00000080:3.0:1787826859.570919:0:80231:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570921:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00020000:00000080:3.0:1787826859.570924:0:80231:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570926:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00020000:00000080:3.0:1787826859.570928:0:80231:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570931:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00020000:00000080:3.0:1787826859.570934:0:80231:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570937:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00020000:00000080:3.0:1787826859.570940:0:80231:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570943:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00020000:00000080:3.0:1787826859.570946:0:80231:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570950:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00020000:00000080:3.0:1787826859.570952:0:80231:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570955:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00020000:00000080:3.0:1787826859.570957:0:80231:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570959:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00020000:00000080:3.0:1787826859.570972:0:80231:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570977:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00020000:00000080:3.0:1787826859.570980:0:80231:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570982:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00020000:00000080:3.0:1787826859.570984:0:80231:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570987:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00020000:00000080:3.0:1787826859.570989:0:80231:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570991:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00020000:00000080:3.0:1787826859.570993:0:80231:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570995:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00020000:00000080:3.0:1787826859.570997:0:80231:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.570999:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00020000:00000080:3.0:1787826859.571001:0:80231:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.571003:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00020000:00000080:3.0:1787826859.571004:0:80231:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.571006:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00020000:00000080:3.0:1787826859.571008:0:80231:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.571010:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00020000:00000080:3.0:1787826859.571012:0:80231:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.571014:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00020000:00000080:3.0:1787826859.571015:0:80231:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.571017:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00020000:00000080:3.0:1787826859.571019:0:80231:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.571021:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00020000:00000080:3.0:1787826859.571022:0:80231:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.571024:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00020000:00000080:3.0:1787826859.571026:0:80231:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.571028:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00020000:00000080:3.0:1787826859.571029:0:80231:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.571031:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00020000:00000080:3.0:1787826859.571034:0:80231:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.571036:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00020000:00000080:3.0:1787826859.571038:0:80231:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.571040:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00020000:00000080:3.0:1787826859.571042:0:80231:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.571044:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00020000:00000080:3.0:1787826859.571045:0:80231:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.571047:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00020000:00000080:3.0:1787826859.571049:0:80231:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.571051:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00020000:00000080:3.0:1787826859.571052:0:80231:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.571054:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00020000:00000080:3.0:1787826859.571056:0:80231:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.571058:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00020000:00000080:3.0:1787826859.571059:0:80231:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.571061:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00020000:00000080:3.0:1787826859.571063:0:80231:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.571065:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00020000:00000080:3.0:1787826859.571067:0:80231:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.571068:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00020000:00000080:3.0:1787826859.571070:0:80231:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.571072:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00020000:00000080:3.0:1787826859.571074:0:80231:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.571076:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00020000:00000080:3.0:1787826859.571077:0:80231:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.571080:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00020000:00000080:3.0:1787826859.571081:0:80231:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.571084:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00020000:00000080:3.0:1787826859.571086:0:80231:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.571088:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00020000:00000080:3.0:1787826859.571089:0:80231:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200080:3.0:1787826859.571091:0:80231:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffef1d766c0 00020000:00000080:3.0:1787826859.571093:0:80231:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543d800 uarg=0000000000000000 00000080:00200008:3.0:1787826859.571099:0:80231:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 100000 00010000:00010000:3.0:1787826859.828869:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826860.853113:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826862.324886:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e783c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:3c946890-15ca-497a-b838-fe1cba61f4dd:2384:1874669225579264:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787826862.324993:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e79680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669225579520:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787826862.325031:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e7b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669225579776:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787826862.326626:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e783c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:3c946890-15ca-497a-b838-fe1cba61f4dd:2384:1874669225579264:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787826862.326649:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e79680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669225579520:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787826862.326665:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e7b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669225579776:192.168.203.114@tcp:400:kworker.0 00010000:00010000:2.0:1787826862.900864:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826864.948919:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200008:1.0:1787826865.862531:0:79773:0:(file.c:479:ll_file_release()) START file f34.replay-single:[0x200016b60:0xb:0x0](ffff98561158f448), flags 100002 00000002:00100000:1.0:1787826865.862610:0:79773:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff985610db43c0 x1874669225571968/t197568495647(197568495647) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826862 ref 1 fl Complete:RMQU/604/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00000020:00100000:1.0:1787826865.862622:0:79773:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:1.0:1787826865.862628:0:79773:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610db7480 x1874669225580032/t0(0) o35->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:1.0:1787826865.862653:0:79773:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610db7480 pname:cluuid:pid:xid:nid:opc:job multiop:7b4e8535-9db7-485d-abf6-bda966b63320:79773:1874669225580032:192.168.203.114@tcp:35:multiop.0 00000100:00100000:1.0:1787826865.862752:0:79773:0:(client.c:2715:ptlrpc_set_wait()) set ffff985611f30e40 going to sleep for 11 seconds 00000100:00100000:1.0:1787826865.865984:0:79773:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610db7480 pname:cluuid:pid:xid:nid:opc:job multiop:7b4e8535-9db7-485d-abf6-bda966b63320:79773:1874669225580032:192.168.203.114@tcp:35:multiop.0 00000002:00080000:1.0:1787826865.866005:0:79773:0:(mdc_request.c:1003:mdc_close()) @@@ Reset ESTALE = -116 req@ffff985610db7480 x1874669225580032/t0(0) o35->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:23/10 lens 392/456 e 0 to 0 dl 1787826881 ref 1 fl Complete:RQU/600/0 rc -116/-116 job:'multiop.0' uid:0 gid:0 projid:0 00000002:00100000:1.0:1787826865.866030:0:79773:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff985610db43c0 x1874669225571968/t197568495647(197568495647) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826862 ref 1 fl Complete:RMQU/604/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00000080:00200000:1.0:1787826865.866072:0:79773:0:(vvp_io.c:1885:vvp_io_init()) [0x200016b60:0xb:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826865.866086:0:79773:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826865.866114:0:79773:0:(vvp_io.c:304:vvp_io_fini()) [0x200016b60:0xb:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826865.866121:0:79773:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200016b60:0xb:0x0](ffff98561158f448) 00010000:00010000:1.0:1787826865.973820:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826866.996948:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826867.446014:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:3c946890-15ca-497a-b838-fe1cba61f4dd:2386:1874669225580288:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787826867.446100:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669225580544:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787826867.446125:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669225580800:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787826867.446149:0:2386:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:2.0:1787826867.446155:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee0f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669225580416:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787826867.446179:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669225580672:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787826867.449307:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669225580672:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787826867.449328:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:3c946890-15ca-497a-b838-fe1cba61f4dd:2386:1874669225580288:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787826867.449344:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669225580800:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787826867.450005:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee0f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669225580416:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787826867.450026:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669225580544:192.168.203.114@tcp:400:kworker.0 00010000:00010000:2.0:1787826868.020780:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:1.0:1787826868.386476:0:80933:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == replay-single test 35: test recovery from llog for unlink op ========================================================== 06:34:27 (1787826867) 00000080:00200000:3.0:1787826868.425656:0:80939:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:3.0:1787826868.425686:0:80939:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826868.425875:0:80939:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1000/0xb3f83dad1b7e08e6 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a2081 expref: -99 pid: 80177 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 1 00010000:00010000:3.0:1787826868.425966:0:80939:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1000/0xb3f83dad1b7e08e6 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a2081 expref: -99 pid: 80177 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 1 00000080:00200000:3.0:1787826868.425978:0:80939:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826868.425985:0:80939:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:3.0:1787826868.425988:0:80939:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e08e6 00000080:00010000:3.0:1787826868.425996:0:80939:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e08e6 from it ffffab3ac1e0fbf8 00010000:00010000:3.0:1787826868.426000:0:80939:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1000/0xb3f83dad1b7e08e6 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a2081 expref: -99 pid: 80177 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 1 00010000:00010000:3.0:1787826868.426009:0:80939:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1000/0xb3f83dad1b7e08e6 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a2081 expref: -99 pid: 80177 timeout: 0 lvb_type: 0 lru_score: 9 lru_type: 1 00000080:00200000:3.0:1787826868.426025:0:80939:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:3.0:1787826868.426039:0:80939:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:3.0:1787826868.426046:0:80939:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826868.426054:0:80939:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1000/0xb3f83dad1b7e08e6 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a2081 expref: -99 pid: 80177 timeout: 0 lvb_type: 0 lru_score: 9 lru_type: 1 00010000:00010000:3.0:1787826868.426062:0:80939:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1000/0xb3f83dad1b7e08e6 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a2081 expref: -99 pid: 80177 timeout: 0 lvb_type: 0 lru_score: 9 lru_type: 1 00000080:00200000:3.0:1787826868.426068:0:80939:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826868.426071:0:80939:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:3.0:1787826868.426073:0:80939:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e08e6 00000080:00010000:3.0:1787826868.426076:0:80939:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e08e6 from it ffffab3ac1e0fb38 00010000:00010000:3.0:1787826868.426078:0:80939:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1000/0xb3f83dad1b7e08e6 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a2081 expref: -99 pid: 80177 timeout: 0 lvb_type: 0 lru_score: 9 lru_type: 1 00010000:00010000:3.0:1787826868.426083:0:80939:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1000/0xb3f83dad1b7e08e6 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a2081 expref: -99 pid: 80177 timeout: 0 lvb_type: 0 lru_score: 10 lru_type: 1 00000080:00200000:3.0:1787826868.426092:0:80939:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 3 00000080:00200000:3.0:1787826868.426097:0:80939:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f35.replay-single, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), file ffff985611f19200, open_flags 8941, mode 81b6 opened 0 00000080:00200000:3.0:1787826868.426104:0:80939:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f35.replay-single, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=open|creat 00000100:00100000:3.0:1787826868.426155:0:80939:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e4f0c0 pname:cluuid:pid:xid:nid:opc:job touch:7b4e8535-9db7-485d-abf6-bda966b63320:80939:1874669225580928:192.168.203.114@tcp:700:touch.0 00000100:00100000:3.0:1787826868.426225:0:80939:0:(client.c:2715:ptlrpc_set_wait()) set ffff9856057883c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787826868.427894:0:80939:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e4f0c0 pname:cluuid:pid:xid:nid:opc:job touch:7b4e8535-9db7-485d-abf6-bda966b63320:80939:1874669225580928:192.168.203.114@tcp:700:touch.0 00000002:00010000:3.0:1787826868.427934:0:80939:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f35.replay-single,[0x200017331:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: open|creat flags 0104502 00010000:00010000:3.0:1787826868.427950:0:80939:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1000/0xb3f83dad1b7e08e6 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x5aa6d504d91a2081 expref: -99 pid: 80177 timeout: 0 lvb_type: 0 lru_score: 10 lru_type: 1 00010000:00010000:3.0:1787826868.428003:0:80939:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1000/0xb3f83dad1b7e08e6 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a2081 expref: -99 pid: 80177 timeout: 0 lvb_type: 0 lru_score: 10 lru_type: 1 00010000:00010000:3.0:1787826868.428012:0:80939:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826868.428014:0:80939:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1000/0xb3f83dad1b7e08e6 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a2081 expref: -99 pid: 80177 timeout: 0 lvb_type: 0 lru_score: 10 lru_type: 1 00010000:00010000:3.0:1787826868.428043:0:80939:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e0910 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 80939 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826868.428047:0:80939:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e0910 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 80939 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:3.0:1787826868.428055:0:80939:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:3.0:1787826868.428060:0:80939:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610e4f0c0 x1874669225581184/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:3.0:1787826868.428067:0:80939:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e0910 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 80939 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826868.428079:0:80939:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e4f0c0 pname:cluuid:pid:xid:nid:opc:job touch:7b4e8535-9db7-485d-abf6-bda966b63320:80939:1874669225581184:192.168.203.114@tcp:101:touch.0 00000100:00100000:3.0:1787826868.428182:0:80939:0:(client.c:2715:ptlrpc_set_wait()) set ffff9856057883c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787826868.431412:0:80939:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e4f0c0 pname:cluuid:pid:xid:nid:opc:job touch:7b4e8535-9db7-485d-abf6-bda966b63320:80939:1874669225581184:192.168.203.114@tcp:101:touch.0 00010000:00010000:3.0:1787826868.431445:0:80939:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e0910 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a20f1 expref: -99 pid: 80939 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826868.431483:0:80939:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e0910 lrc: 4/0,1 mode: --/PR res: [0x200017331:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a20f1 expref: -99 pid: 80939 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826868.431492:0:80939:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e0910 lrc: 4/0,1 mode: --/PR res: [0x200017331:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a20f1 expref: -99 pid: 80939 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826868.431503:0:80939:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e0910 lrc: 4/0,1 mode: PR/PR res: [0x200017331:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a20f1 expref: -99 pid: 80939 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826868.431514:0:80939:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e0910 lrc: 4/0,1 mode: PR/PR res: [0x200017331:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a20f1 expref: -99 pid: 80939 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826868.431527:0:80939:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e0910 lrc: 5/1,1 mode: PR/PR res: [0x200017331:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a20f1 expref: -99 pid: 80939 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826868.431534:0:80939:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e0910 lrc: 5/1,1 mode: PR/PR res: [0x200017331:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a20f1 expref: -99 pid: 80939 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826868.431541:0:80939:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e0910 lrc: 4/1,0 mode: PR/PR res: [0x200017331:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a20f1 expref: -99 pid: 80939 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826868.431552:0:80939:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff985610e4f0c0 x1874669225581184/t201863462913(201863462913) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/664 e 0 to 0 dl 1787826884 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'touch.0' uid:0 gid:0 projid:0 00000002:00100000:3.0:1787826868.431566:0:80939:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff985610e4f0c0 x1874669225581184/t201863462913(201863462913) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/664 e 0 to 0 dl 1787826884 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'touch.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826868.431578:0:80939:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e0910 lrc: 3/1,0 mode: PR/PR res: [0x200017331:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a20f1 expref: -99 pid: 80939 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:3.0:1787826868.431591:0:80939:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e0910 lrc: 3/1,0 mode: PR/PR res: [0x200017331:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a20f1 expref: -99 pid: 80939 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826868.431600:0:80939:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8590029617/1 (0 0) 00000080:00200000:3.0:1787826868.431606:0:80939:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:3.0:1787826868.431618:0:80939:0:(namei.c:1628:ll_create_it()) VFS Op:name=f35.replay-single, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=open|creat 00000080:00200000:3.0:1787826868.431641:0:80939:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200017331:0x1:0x0](ffff98561011aa08) 00000080:00200000:3.0:1787826868.431751:0:80939:0:(file.c:6854:ll_layout_conf()) [0x200017331:0x1:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:3.0:1787826868.431757:0:80939:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x200017331:0x1:0x0](ffff98561011aa08) 00000080:00010000:3.0:1787826868.431760:0:80939:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200017331:0x1:0x0](ffff98561011aa08) for lock 0xb3f83dad1b7e0910 00000080:00010000:3.0:1787826868.431770:0:80939:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200017331:0x1:0x0](ffff98561011aa08) 00000080:00200008:3.0:1787826868.431782:0:80939:0:(file.c:1090:ll_file_open()) START file f35.replay-single:[0x200017331:0x1:0x0](ffff98561011aa08), flags 104501 00000002:00100000:3.0:1787826868.431803:0:80939:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff985610e4f0c0 x1874669225581184/t201863462913(201863462913) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826884 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'touch.0' uid:0 gid:0 projid:0 00000080:00010000:3.0:1787826868.431834:0:80939:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0910 from it ffff985610c4af00 00010000:00010000:3.0:1787826868.431839:0:80939:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e0910 lrc: 3/1,0 mode: PR/PR res: [0x200017331:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a20f1 expref: -99 pid: 80939 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826868.431849:0:80939:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e0910 lrc: 2/0,0 mode: PR/PR res: [0x200017331:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a20f1 expref: -99 pid: 80939 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826868.431869:0:80939:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200017331:0x1:0x0](ffff98561011aa08), inode mode 81a4 mask 40 00000080:00200008:3.0:1787826868.431920:0:80939:0:(llite_lib.c:2427:ll_setattr_raw()) START file f35.replay-single:[0x200017331:0x1:0x0](ffff98561011aa08) current size 0, valid attrs 20070, mode 9856, uid -1042219440, gid -21702, new size 18446744072281848343, atime 1787826868.000000000, mtime 1787826868.000000000, ctime 1787826868.000000000 00010000:00010000:3.0:1787826868.431940:0:80939:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e0910 lrc: 2/0,0 mode: PR/PR res: [0x200017331:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x5aa6d504d91a20f1 expref: -99 pid: 80939 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826868.431966:0:80939:0:(vvp_object.c:117:vvp_conf_set()) [0x200017331:0x1:0x0]: losing layout lock 00010000:00010000:3.0:1787826868.431998:0:80939:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e0910 lrc: 1/0,0 mode: --/PR res: [0x200017331:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x5aa6d504d91a20f1 expref: -99 pid: 80939 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826868.432006:0:80939:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826868.432009:0:80939:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e0910 lrc: 0/0,0 mode: --/PR res: [0x200017331:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x5aa6d504d91a20f1 expref: -99 pid: 80939 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000020:00100000:3.0:1787826868.432029:0:80939:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:3.0:1787826868.432036:0:80939:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610e4c780 x1874669225581440/t0(0) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 488/512 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826868.432055:0:80939:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e4c780 pname:cluuid:pid:xid:nid:opc:job touch:7b4e8535-9db7-485d-abf6-bda966b63320:80939:1874669225581440:192.168.203.114@tcp:36:touch.0 00000100:00100000:3.0:1787826868.432127:0:80939:0:(client.c:2715:ptlrpc_set_wait()) set ffff985605788000 going to sleep for 11 seconds 00000100:00100000:1.0:1787826868.436066:0:80939:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e783c0 pname:cluuid:pid:xid:nid:opc:job touch:7b4e8535-9db7-485d-abf6-bda966b63320:80939:1874669225581568:192.168.203.114@tcp:101:touch.0 00010000:00010000:1.0:1787826868.436095:0:80939:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1400/0xb3f83dad1b7e0917 lrc: 4/1,0 mode: CR/CR res: [0x200017331:0x1:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a2106 expref: -99 pid: 80939 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826868.436109:0:80939:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1400/0xb3f83dad1b7e0917 lrc: 4/1,0 mode: CR/CR res: [0x200017331:0x1:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a2106 expref: -99 pid: 80939 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826868.436130:0:80939:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x400 disposition=0x1, status=0 req@ffff985610e783c0 x1874669225581568/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 376/400 e 0 to 0 dl 1787826884 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'touch.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826868.436141:0:80939:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: layout, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1400/0xb3f83dad1b7e0917 lrc: 3/1,0 mode: CR/CR res: [0x200017331:0x1:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a2106 expref: -99 pid: 80939 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00000002:00010000:1.0:1787826868.436153:0:80939:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1400/0xb3f83dad1b7e0917 lrc: 3/1,0 mode: CR/CR res: [0x200017331:0x1:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a2106 expref: -99 pid: 80939 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826868.436165:0:80939:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 16 res 8590029617/1 (0 0) 00000080:00200000:1.0:1787826868.436171:0:80939:0:(namei.c:952:ll_intent_lock()) intent lock 1024 on i1 [0x200017331:0x1:0x0] suppgids 0 0: rc 0 00000080:00010000:1.0:1787826868.436188:0:80939:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200017331:0x1:0x0](ffff98561011aa08) for lock 0xb3f83dad1b7e0917 00000080:00010000:1.0:1787826868.436194:0:80939:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0917 from it ffffab3ac1e0fa78 00010000:00010000:1.0:1787826868.436198:0:80939:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1400/0xb3f83dad1b7e0917 lrc: 3/1,0 mode: CR/CR res: [0x200017331:0x1:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a2106 expref: -99 pid: 80939 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826868.436207:0:80939:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1400/0xb3f83dad1b7e0917 lrc: 2/0,0 mode: CR/CR res: [0x200017331:0x1:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a2106 expref: -99 pid: 80939 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826868.436231:0:80939:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1400/0xb3f83dad1b7e0917 lrc: 2/1,0 mode: CR/CR res: [0x200017331:0x1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a2106 expref: -99 pid: 80939 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826868.436241:0:80939:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1400/0xb3f83dad1b7e0917 lrc: 2/1,0 mode: CR/CR res: [0x200017331:0x1:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a2106 expref: -99 pid: 80939 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00010000:1.0:1787826868.436249:0:80939:0:(file.c:6959:ll_layout_lock_set()) ### file [0x200017331:0x1:0x0](00000000180894c4) being reconfigured ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1400/0xb3f83dad1b7e0917 lrc: 3/1,0 mode: CR/CR res: [0x200017331:0x1:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a2106 expref: -99 pid: 80939 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826868.436393:0:80939:0:(file.c:6854:ll_layout_conf()) [0x200017331:0x1:0x0]: layout version change: 4294967294 -> 0 00010000:00010000:1.0:1787826868.436399:0:80939:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1400/0xb3f83dad1b7e0917 lrc: 3/1,0 mode: CR/CR res: [0x200017331:0x1:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a2106 expref: -99 pid: 80939 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826868.436406:0:80939:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1400/0xb3f83dad1b7e0917 lrc: 2/0,0 mode: CR/CR res: [0x200017331:0x1:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a2106 expref: -99 pid: 80939 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00020000:00200000:1.0:1787826868.436442:0:80939:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000100:00100000:1.0:1787826868.436522:0:2384:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1787826868.436540:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610db5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669225581696:192.168.203.114@tcp:2:touch.0 00000080:00200000:2.0:1787826868.438007:0:80939:0:(vvp_io.c:304:vvp_io_fini()) [0x200017331:0x1:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200008:2.0:1787826868.438054:0:80939:0:(llite_lib.c:2686:ll_setattr_raw()) COMPLETED file f35.replay-single:[0x200017331:0x1:0x0](ffff98561011aa08) current size 0, valid attrs 100201f0, mode 9856, uid -1042219440, gid -21702, new size 18446744072281848343, atime 1787826868.000000000, mtime 1787826868.000000000, ctime 1787826868.000000000, rc 0 00000080:00200008:2.0:1787826868.438080:0:80939:0:(file.c:479:ll_file_release()) START file f35.replay-single:[0x200017331:0x1:0x0](ffff98561011aa08), flags 104001 00000002:00100000:2.0:1787826868.438131:0:80939:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff985610e4f0c0 x1874669225581184/t201863462913(201863462913) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826884 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'touch.0' uid:0 gid:0 projid:0 00000020:00100000:2.0:1787826868.438150:0:80939:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:2.0:1787826868.438156:0:80939:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985602ee12c0 x1874669225581952/t0(0) o35->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:2.0:1787826868.438175:0:80939:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee12c0 pname:cluuid:pid:xid:nid:opc:job touch:7b4e8535-9db7-485d-abf6-bda966b63320:80939:1874669225581952:192.168.203.114@tcp:35:touch.0 00000100:00100000:2.0:1787826868.438227:0:80939:0:(client.c:2715:ptlrpc_set_wait()) set ffff985611e5e3c0 going to sleep for 11 seconds 00000080:00200000:1.0:1787826868.837625:0:80954:0:(file.c:1775:ll_merge_attr_nolock()) [0x200017331:0x1:0x0] updating i_size 0 i_blocks 0 00000020:00010000:1.0:1787826868.837631:0:80954:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff98560549b3d8 (ffff985610fe1998/0) at cl_lock_release():245 00000020:00010000:1.0:1787826868.837635:0:80954:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff98560549b3d8 (ffff985610fe1998/0) at cl_lock_cancel():142 00000020:00010000:1.0:1787826868.837638:0:80954:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff985612a3c430 (ffff985610fe1638/1) at cl_lock_cancel():142 00000020:00010000:1.0:1787826868.837643:0:80954:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff98560549b3d8 (ffff985610fe1998/0) at cl_lock_fini():72 00000020:00010000:1.0:1787826868.837647:0:80954:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff985612a3c430 (ffff985610fe1998/1) at cl_lock_fini():72 00000080:00200000:1.0:1787826868.837659:0:80954:0:(vvp_io.c:304:vvp_io_fini()) [0x200017331:0x1:0x0] ignore/verify layout 0/1, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826868.837735:0:80954:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826868.837748:0:80954:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826868.837764:0:80954:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1000/0xb3f83dad1b7e091e lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a2114 expref: -99 pid: 80954 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826868.837775:0:80954:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1000/0xb3f83dad1b7e091e lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a2114 expref: -99 pid: 80954 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826868.837782:0:80954:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826868.837789:0:80954:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826868.837793:0:80954:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e091e 00000080:00010000:1.0:1787826868.837799:0:80954:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e091e from it ffffab3ac167bc10 00010000:00010000:1.0:1787826868.837803:0:80954:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1000/0xb3f83dad1b7e091e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a2114 expref: -99 pid: 80954 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826868.837812:0:80954:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1000/0xb3f83dad1b7e091e lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a2114 expref: -99 pid: 80954 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826868.837822:0:80954:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787826868.837831:0:80954:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f35.replay-single, flags=0 00000080:00200000:1.0:1787826868.837836:0:80954:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826868.837844:0:80954:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826868.837851:0:80954:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1000/0xb3f83dad1b7e091e lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a2114 expref: -99 pid: 80954 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826868.837857:0:80954:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1000/0xb3f83dad1b7e091e lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a2114 expref: -99 pid: 80954 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826868.837861:0:80954:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826868.837864:0:80954:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826868.837867:0:80954:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e091e 00000080:00010000:1.0:1787826868.837870:0:80954:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e091e from it ffffab3ac167bd68 00010000:00010000:1.0:1787826868.837872:0:80954:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1000/0xb3f83dad1b7e091e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a2114 expref: -99 pid: 80954 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826868.837878:0:80954:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1000/0xb3f83dad1b7e091e lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a2114 expref: -99 pid: 80954 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787826868.837887:0:80954:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787826868.837891:0:80954:0:(namei.c:2418:ll_unlink()) VFS Op:name=f35.replay-single, dir=[0x200000007:0x1:0x0](ffff98561158e3c8) 00010000:00010000:1.0:1787826868.837925:0:80954:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1000/0xb3f83dad1b7e091e lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x5aa6d504d91a2114 expref: -99 pid: 80954 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787826868.837944:0:80954:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0600/0xb3f83dad1b7e0925 lrc: 2/0,0 mode: PR/PR res: [0x200017331:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x5aa6d504d91a2130 expref: -99 pid: 80954 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826868.837952:0:80954:0:(vvp_object.c:117:vvp_conf_set()) [0x200017331:0x1:0x0]: losing layout lock 00010000:00010000:1.0:1787826868.837962:0:80954:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1400/0xb3f83dad1b7e0917 lrc: 2/0,0 mode: CR/CR res: [0x200017331:0x1:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x5aa6d504d91a2106 expref: -99 pid: 80939 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826868.837968:0:80954:0:(vvp_object.c:117:vvp_conf_set()) [0x200017331:0x1:0x0]: losing layout lock 00010000:00010000:1.0:1787826868.837988:0:80954:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0600/0xb3f83dad1b7e0925 lrc: 1/0,0 mode: --/PR res: [0x200017331:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x5aa6d504d91a2130 expref: -99 pid: 80954 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826868.837995:0:80954:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1400/0xb3f83dad1b7e0917 lrc: 1/0,0 mode: --/CR res: [0x200017331:0x1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x5aa6d504d91a2106 expref: -99 pid: 80939 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826868.838002:0:80954:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1000/0xb3f83dad1b7e091e lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a2114 expref: -99 pid: 80954 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787826868.838008:0:80954:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 3 locks packed 00010000:00010000:1.0:1787826868.838011:0:80954:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0600/0xb3f83dad1b7e0925 lrc: 0/0,0 mode: --/PR res: [0x200017331:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x5aa6d504d91a2130 expref: -99 pid: 80954 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826868.838022:0:80954:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1400/0xb3f83dad1b7e0917 lrc: 0/0,0 mode: --/CR res: [0x200017331:0x1:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x5aa6d504d91a2106 expref: -99 pid: 80939 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826868.838052:0:80954:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1000/0xb3f83dad1b7e091e lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a2114 expref: -99 pid: 80954 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000020:00100000:1.0:1787826868.838068:0:80954:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787826868.838074:0:80954:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610db52c0 x1874669225582592/t0(0) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826868.838093:0:80954:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610db52c0 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:80954:1874669225582592:192.168.203.114@tcp:36:rm.0 00000100:00100000:1.0:1787826868.838152:0:80954:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607ee2cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826868.981303:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610db43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669225582848:192.168.203.114@tcp:17:kworker.0 00000100:00100000:1.0:1787826868.983279:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610db43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669225582848:192.168.203.114@tcp:17:kworker.0 00000080:00200000:1.0:1787826869.836469:0:37:0:(vvp_io.c:1885:vvp_io_init()) [0x200017331:0x1:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826869.836485:0:37:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826869.836522:0:37:0:(vvp_io.c:304:vvp_io_fini()) [0x200017331:0x1:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00010000:00010000:3.0:1787826870.069092:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826870.407012:0:2378:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.114@tcp (source: 12345-192.168.203.114@tcp) 00000100:00100000:3.0:1787826870.407109:0:33343:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874669235536256 00000100:00100000:3.0:1787826870.407120:0:33343:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874669235536256 00000100:00100000:3.0:1787826870.407173:0:33343:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.114@tcp, seq: 254 00000100:00100000:3.0:1787826870.407184:0:33343:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff985602ee3480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb01_002:LOV_OSC_UUID+4:10409:x1874669235536256:12345-192.168.203.114@tcp:104: 00010000:00010000:3.0:1787826870.407202:0:33343:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-OST0001-osc-ffff985602bc9800 lock: 0000000093e37690/0xb3f83dad1b7e092c lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x622:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800420000010000 nid: local remote: 0x5aa6d504d91a2137 expref: -99 pid: 80954 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826870.407286:0:33343:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:3.0:1787826870.407293:0:33343:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff985602ee3480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb01_002:LOV_OSC_UUID+4:10409:x1874669235536256:12345-192.168.203.114@tcp:104: Request processed in 112us (291us total) trans 0 rc 0/0 00000100:00100000:3.0:1787826870.407302:0:33343:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.114@tcp, seq: 254 00010000:00010000:1.0:1787826870.407331:0:33342:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787826870.407340:0:33342:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-OST0001-osc-ffff985602bc9800 lock: 0000000093e37690/0xb3f83dad1b7e092c lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x622:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800420000010000 nid: local remote: 0x5aa6d504d91a2137 expref: -99 pid: 80954 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826870.407350:0:33342:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985609eb1200 already unused, calling callback (ffffffffc13b01e0) 00010000:00010000:1.0:1787826870.407356:0:33342:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0001-osc-ffff985602bc9800 lock: 0000000093e37690/0xb3f83dad1b7e092c lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x622:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800428400010000 nid: local remote: 0x5aa6d504d91a2137 expref: -99 pid: 80954 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826870.407391:0:33342:0:(vvp_io.c:1885:vvp_io_init()) [0x200017331:0x1:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826870.407404:0:33342:0:(vvp_io.c:304:vvp_io_fini()) [0x200017331:0x1:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 1 00010000:00010000:1.0:1787826870.407445:0:33342:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-OST0001-osc-ffff985602bc9800 lock: 0000000093e37690/0xb3f83dad1b7e092c lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x622:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400010020 nid: local remote: 0x5aa6d504d91a2137 expref: -99 pid: 80954 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826870.407453:0:33342:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826870.407489:0:33342:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-OST0001-osc-ffff985602bc9800 lock: 0000000093e37690/0xb3f83dad1b7e092c lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x622:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400010020 nid: local remote: 0x5aa6d504d91a2137 expref: -99 pid: 80954 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826870.407502:0:33342:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0001-osc-ffff985602bc9800 lock: 0000000093e37690/0xb3f83dad1b7e092c lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x622:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400010020 nid: local remote: 0x5aa6d504d91a2137 expref: -99 pid: 80954 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826870.407534:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826870.407538:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826870.407541:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826871.092876:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826872.116904:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826872.564852:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e4c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2385:1874669225583104:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787826872.564863:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e4fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669225583232:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787826872.564945:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e4d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669225583488:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787826872.564950:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e4da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669225583360:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787826872.564977:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e4c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669225583616:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787826872.566888:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e4c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669225583616:192.168.203.114@tcp:400:kworker.0 00000100:00080000:3.0:1787826872.566964:0:2386:0:(recover.c:205:ptlrpc_request_handle_notconn()) import lustre-MDT0000-mdc-ffff985602bc9800 of lustre-MDT0000_UUID@192.168.203.114@tcp abruptly disconnected: reconnecting 00000100:00080000:3.0:1787826872.566972:0:2386:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from FULL to DISCONN 00000100:02000400:3.0:1787826872.566978:0:2386:0:(import.c:182:ptlrpc_set_import_discon()) lustre-MDT0000-mdc-ffff985602bc9800: Connection to lustre-MDT0000 (at 192.168.203.114@tcp) was lost; in progress operations using this service will wait for recovery to complete 00000100:00080000:3.0:1787826872.566986:0:2386:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826872.566991:0:2386:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3007 00000100:00080000:3.0:1787826872.567006:0:2386:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787826872.567019:0:2386:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787826872.567067:0:2386:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610e4d680 x1874669225583744/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826872.567103:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e4fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669225583232:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787826872.567126:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e4d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669225583488:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787826872.567146:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e4d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669225583744:192.168.203.114@tcp:38:ptlrpcd_01_01.0 00000100:00100000:3.0:1787826872.569409:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e4da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669225583360:192.168.203.114@tcp:400:kworker.0 00000100:00080000:1.0:1787826872.569560:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787826872.569567:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-19) 00000100:00100000:1.0:1787826872.569572:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e4d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669225583744:192.168.203.114@tcp:38:ptlrpcd_01_01.0 00010000:00010000:3.0:1787826874.164882:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826874.612854:0:78053:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050400/0xb3f83dad1b7e0933 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826874.612885:0:78053:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050400/0xb3f83dad1b7e0933 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x0 pid: 78053 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1787826874.612969:0:2384:0:(client.c:1845:ptlrpc_send_new_req()) @@@ req waiting for recovery: (FULL != DISCONN) req@ffff985610db6940 x1874669225583872/t0(0) o63->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:24/4 lens 680/4320 e 0 to 0 dl 0 ref 1 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'ll_sa.0' uid:0 gid:0 projid:4294967295 00010000:00010000:2.0:1787826874.613043:0:79203:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4e00/0xb3f83dad1b7e093a lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826874.613054:0:79203:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4e00/0xb3f83dad1b7e093a lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x0 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787826874.613114:0:2386:0:(client.c:1845:ptlrpc_send_new_req()) @@@ req waiting for recovery: (FULL != DISCONN) req@ffff985602ee3840 x1874669225584000/t0(0) o63->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:24/4 lens 680/4320 e 0 to 0 dl 0 ref 1 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'ll_sa.0' uid:0 gid:0 projid:4294967295 00010000:00010000:1.0:1787826875.188889:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826876.212895:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826877.236886:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787826877.684874:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826877.684879:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787826877.684881:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826877.684889:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3023 00000100:00080000:3.0:1787826877.684897:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 5s 00000100:00080000:3.0:1787826877.684903:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787826877.684913:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787826877.684925:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610e4e1c0 x1874669225584256/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826877.684988:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e4e1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669225584256:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787826877.685065:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e4e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2385:1874669225584128:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787826877.685108:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e4e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669225584512:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787826877.685139:0:2385:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1787826877.685144:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e4de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669225584384:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787826877.685160:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e4cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669225584640:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787826877.686518:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e4de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669225584384:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787826877.686532:0:2385:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0001-osc-ffff985602bc9800: committing for last_committed 4294967636 gen 5 00000100:00100000:3.0:1787826877.686546:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e4cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669225584640:192.168.203.114@tcp:400:kworker.0 00000100:00080000:3.0:1787826877.686553:0:2385:0:(recover.c:205:ptlrpc_request_handle_notconn()) import MGC192.168.203.114@tcp of MGS@192.168.203.114@tcp abruptly disconnected: reconnecting 00000100:00080000:3.0:1787826877.686556:0:2385:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from FULL to DISCONN 00000100:02020000:3.0:1787826877.686559:0:2385:0:(import.c:186:ptlrpc_set_import_discon()) MGC192.168.203.114@tcp: Connection to MGS (at 192.168.203.114@tcp) was lost; in progress operations using this service will fail 00000100:00080000:3.0:1787826877.686561:0:2385:0:(import.c:192:ptlrpc_set_import_discon()) import MGS@192.168.203.114@tcp for MGC192.168.203.114@tcp not replayable, auto-deactivating 00000100:00080000:3.0:1787826877.686563:0:2385:0:(import.c:138:ptlrpc_deactivate_import_nolock()) setting import MGS INVALID 00000100:00100000:3.0:1787826877.686567:0:2385:0:(client.c:3589:ptlrpc_abort_inflight()) @@@ inflight req@ffff985610e4c000 x1874669225583104/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787826888 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826877.686575:0:2385:0:(client.c:3589:ptlrpc_abort_inflight()) @@@ inflight req@ffff985610e4e580 x1874669225584128/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787826893 ref 1 fl Rpc:RNQU/200/0 rc 0/-107 job:'kworker.0' uid:0 gid:0 projid:4294967295 10000000:01000000:3.0:1787826877.686584:0:2385:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808001 00000100:00080000:3.0:1787826877.686587:0:2385:0:(pinger.c:193:ptlrpc_pinger_ir_down()) IR down 10000000:01000000:3.0:1787826877.686588:0:2385:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808002 00000100:00080000:3.0:1787826877.686590:0:2385:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826877.686594:0:2385:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3007 00000100:00080000:3.0:1787826877.686601:0:2385:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787826877.686617:0:2385:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610e4fc00 x1874669225584768/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826877.686635:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e4e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2385:1874669225584128:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787826877.686645:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e4e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669225584512:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787826877.686650:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e4c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2385:1874669225583104:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787826877.686679:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e4fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669225584768:192.168.203.114@tcp:250:ptlrpcd_01_00.0 00000100:00080000:1.0:1787826877.687654:0:2382:0:(import.c:1132:ptlrpc_connect_interpret()) MGC192.168.203.114@tcp: connect to target with instance 0 10000000:01000000:1.0:1787826877.687663:0:2382:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808005 00000100:02000400:1.0:1787826877.687668:0:2382:0:(import.c:1284:ptlrpc_connect_interpret()) Evicted from MGS (at 192.168.203.114@tcp) after server handle changed from 0x5aa6d504d91a206c to 0x5aa6d504d91a2520 00000100:00080000:1.0:1787826877.687672:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to EVICTED 00000100:00080000:1.0:1787826877.687677:0:2382:0:(import.c:1674:ptlrpc_import_recovery_state_machine()) evicted from MGS@192.168.203.114@tcp; invalidating 00000100:00100000:1.0:1787826877.687938:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e4fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669225584768:192.168.203.114@tcp:250:ptlrpcd_01_00.0 00010000:00080000:1.0:1787826877.687953:0:2382:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=46162) req@ffff985610e4e1c0 x1874669225584256/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/416 e 0 to 0 dl 1787826892 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787826877.687975:0:2382:0:(import.c:1132:ptlrpc_connect_interpret()) lustre-MDT0000-mdc-ffff985602bc9800: connect to target with instance 48 00000100:00080000:1.0:1787826877.687996:0:2382:0:(import.c:1338:ptlrpc_connect_interpret()) @@@ lustre-MDT0000-mdc-ffff985602bc9800: evicting (reconnect/recover flags not set: 4) req@ffff985610e4e1c0 x1874669225584256/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/416 e 0 to 0 dl 1787826892 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787826877.688015:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to EVICTED 00000100:02020000:1.0:1787826877.688023:0:2382:0:(import.c:1663:ptlrpc_import_recovery_state_machine()) lustre-MDT0000-mdc-ffff985602bc9800: This client was evicted by lustre-MDT0000; in progress operations using this service will fail. 00000100:00080000:1.0:1787826877.692341:0:2382:0:(import.c:1674:ptlrpc_import_recovery_state_machine()) evicted from lustre-MDT0000_UUID@192.168.203.114@tcp; invalidating 00000100:00100000:3.0:1787826877.692515:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e4e1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669225584256:192.168.203.114@tcp:38:kworker.0 00000100:00080000:1.0:1787826877.692526:0:81207:0:(import.c:1598:ptlrpc_invalidate_import_thread()) thread invalidate import lustre-MDT0000-mdc-ffff985602bc9800 to lustre-MDT0000_UUID@192.168.203.114@tcp 00000100:00080000:1.0:1787826877.692532:0:81207:0:(import.c:138:ptlrpc_deactivate_import_nolock()) setting import lustre-MDT0000_UUID INVALID 00000100:00100000:1.0:1787826877.692540:0:81207:0:(client.c:3589:ptlrpc_abort_inflight()) @@@ inflight req@ffff985610db52c0 x1874669225582592/t0(0) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/624 e 0 to 0 dl 1787826884 ref 2 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787826877.692568:0:81207:0:(client.c:3601:ptlrpc_abort_inflight()) @@@ aborting waiting req req@ffff985610db6940 x1874669225583872/t0(0) o63->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:24/4 lens 680/4320 e 0 to 0 dl 0 ref 1 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'ll_sa.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787826877.692591:0:81207:0:(client.c:3601:ptlrpc_abort_inflight()) @@@ aborting waiting req req@ffff985602ee3840 x1874669225584000/t0(0) o63->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:24/4 lens 680/4320 e 0 to 0 dl 0 ref 1 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'ll_sa.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787826877.692606:0:81207:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 197568495648 gen 8 00000100:00100000:1.0:1787826877.692612:0:81207:0:(client.c:3163:ptlrpc_free_committed()) @@@ free request with old gen req@ffff985610e4f0c0 x1874669225581184/t201863462913(201863462913) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826884 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'touch.0' uid:0 gid:0 projid:0 00000100:00100000:2.0:1787826877.692635:0:80954:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610db52c0 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:80954:1874669225582592:192.168.203.114@tcp:36:rm.0 00000100:00100000:1.0:1787826877.692640:0:81207:0:(client.c:3163:ptlrpc_free_committed()) @@@ free request with old gen req@ffff985610e4c780 x1874669225581440/t201863462914(201863462914) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 488/456 e 0 to 0 dl 1787826884 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'touch.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787826877.692662:0:81207:0:(client.c:3163:ptlrpc_free_committed()) @@@ free request with old gen req@ffff985602ee12c0 x1874669225581952/t201863462915(201863462915) o35->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:23/10 lens 392/456 e 0 to 0 dl 1787826884 ref 1 fl Complete:RQU/604/0 rc 0/0 job:'touch.0' uid:0 gid:0 projid:0 00000100:00100000:1.0:1787826877.692728:0:81207:0:(import.c:323:ptlrpc_invalidate_import()) Sleeping 20s for inflight to error out 00010000:00010000:3.0:1787826877.692743:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4e00/0xb3f83dad1b7e093a lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x0 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826877.692759:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4e00/0xb3f83dad1b7e093a lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826877.692767:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4e00/0xb3f83dad1b7e093a lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826877.692776:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4e00/0xb3f83dad1b7e093a lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826877.692780:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4e00/0xb3f83dad1b7e093a lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826877.692785:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9856056d4e00 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787826877.692792:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4e00/0xb3f83dad1b7e093a lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 4 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826877.692805:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4e00/0xb3f83dad1b7e093a lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 4 type: IBT flags: 0x206400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826877.692824:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4e00/0xb3f83dad1b7e093a lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 4 type: IBT flags: 0x20e400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826877.692854:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826877.692862:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4e00/0xb3f83dad1b7e093a lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826877.692868:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4e00/0xb3f83dad1b7e093a lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 79203 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826877.692899:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:0:1874669225584000:192.168.203.114@tcp:63:ll_sa.0 00000100:02000000:1.0:1787826877.694860:0:81207:0:(import.c:1743:ptlrpc_import_recovery_state_machine()) lustre-MDT0000-mdc-ffff985602bc9800: Connection restored to 192.168.203.114@tcp (at 192.168.203.114@tcp) 00010000:00010000:3.0:1787826879.284972:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200008:2.0:1787826879.643447:0:81286:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:2.0:1787826879.643457:0:81286:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:2.0:1787826879.643476:0:81286:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826879.643490:0:81286:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1c00/0xb3f83dad1b7e0941 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a252e expref: -99 pid: 81234 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:2.0:1787826879.643501:0:81286:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1c00/0xb3f83dad1b7e0941 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a252e expref: -99 pid: 81234 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:2.0:1787826879.643508:0:81286:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787826879.643515:0:81286:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:2.0:1787826879.643519:0:81286:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e0941 00000080:00010000:2.0:1787826879.643526:0:81286:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0941 from it ffffab3ac1f8fcb8 00010000:00010000:2.0:1787826879.643531:0:81286:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1c00/0xb3f83dad1b7e0941 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a252e expref: -99 pid: 81234 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:2.0:1787826879.643539:0:81286:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1c00/0xb3f83dad1b7e0941 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a252e expref: -99 pid: 81234 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00000080:00200008:2.0:1787826879.644078:0:81286:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 0, flags 0, foreign 0 00000080:00200000:2.0:1787826879.644081:0:81286:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:2.0:1787826879.644088:0:81286:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826879.644094:0:81286:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1c00/0xb3f83dad1b7e0941 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a252e expref: -99 pid: 81234 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00010000:00010000:2.0:1787826879.644100:0:81286:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1c00/0xb3f83dad1b7e0941 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a252e expref: -99 pid: 81234 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00000080:00200000:2.0:1787826879.644105:0:81286:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787826879.644108:0:81286:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:2.0:1787826879.644109:0:81286:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e0941 00000080:00010000:2.0:1787826879.644112:0:81286:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0941 from it ffffab3ac1f8fcb8 00010000:00010000:2.0:1787826879.644114:0:81286:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1c00/0xb3f83dad1b7e0941 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a252e expref: -99 pid: 81234 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00010000:00010000:2.0:1787826879.644119:0:81286:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1c00/0xb3f83dad1b7e0941 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a252e expref: -99 pid: 81234 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00000080:00200000:2.0:1787826879.644178:0:81286:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:2.0:1787826879.644185:0:81286:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826879.644189:0:81286:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1c00/0xb3f83dad1b7e0941 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a252e expref: -99 pid: 81234 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00010000:00010000:2.0:1787826879.644195:0:81286:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1c00/0xb3f83dad1b7e0941 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a252e expref: -99 pid: 81234 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00000080:00200000:2.0:1787826879.644199:0:81286:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787826879.644201:0:81286:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:2.0:1787826879.644203:0:81286:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e0941 00000080:00010000:2.0:1787826879.644205:0:81286:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0941 from it ffffab3ac1f8fbb0 00010000:00010000:2.0:1787826879.644208:0:81286:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1c00/0xb3f83dad1b7e0941 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a252e expref: -99 pid: 81234 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00010000:00010000:2.0:1787826879.644212:0:81286:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1c00/0xb3f83dad1b7e0941 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a252e expref: -99 pid: 81234 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 0 00000080:00200000:2.0:1787826879.644217:0:81286:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 44 00000080:00200008:2.0:1787826879.644222:0:81286:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 100000 00000080:00200080:2.0:1787826879.644340:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00800000:00000080:2.0:1787826879.644353:0:81286:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00800000:00000004:2.0:1787826879.644360:0:81286:0:(obd_class.h:1091:obd_statfs()) lustre-MDT0000-mdc-ffff985602bc9800: age 3029, max_age 3029 00800000:00000004:2.0:1787826879.644365:0:81286:0:(obd_class.h:1131:obd_statfs()) lustre-MDT0000-mdc-ffff985602bc9800: use ffff985606a337f0 cache blocks 317364/353529 objects 1023691/1024000 00000080:00200080:2.0:1787826879.644644:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00800000:00000080:2.0:1787826879.644656:0:81286:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00800000:00000004:2.0:1787826879.644662:0:81286:0:(obd_class.h:1091:obd_statfs()) lustre-MDT0001-mdc-ffff985602bc9800: age 3029, max_age 3029 00800000:00000004:2.0:1787826879.644664:0:81286:0:(obd_class.h:1131:obd_statfs()) lustre-MDT0001-mdc-ffff985602bc9800: use ffff98560952b7f0 cache blocks 317409/353529 objects 1023738/1024000 00000080:00200080:2.0:1787826879.644681:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00800000:00000080:2.0:1787826879.644688:0:81286:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.644718:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00800000:00000080:2.0:1787826879.644724:0:81286:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.644730:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00800000:00000080:2.0:1787826879.644735:0:81286:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.644740:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00800000:00000080:2.0:1787826879.644745:0:81286:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.644749:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00800000:00000080:2.0:1787826879.644753:0:81286:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.644757:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00800000:00000080:2.0:1787826879.644761:0:81286:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.644766:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00800000:00000080:2.0:1787826879.644771:0:81286:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.644776:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00800000:00000080:2.0:1787826879.644780:0:81286:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.644784:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00800000:00000080:2.0:1787826879.644788:0:81286:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.644792:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00800000:00000080:2.0:1787826879.644796:0:81286:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.644801:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00800000:00000080:2.0:1787826879.644805:0:81286:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.644809:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00800000:00000080:2.0:1787826879.644813:0:81286:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.644817:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00800000:00000080:2.0:1787826879.644831:0:81286:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.644838:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00800000:00000080:2.0:1787826879.644842:0:81286:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.644848:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00800000:00000080:2.0:1787826879.644851:0:81286:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.644854:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00800000:00000080:2.0:1787826879.644856:0:81286:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.644859:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00800000:00000080:2.0:1787826879.644861:0:81286:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.644864:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00800000:00000080:2.0:1787826879.644868:0:81286:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.644870:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00800000:00000080:2.0:1787826879.644872:0:81286:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.644874:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00800000:00000080:2.0:1787826879.644876:0:81286:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.644878:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00800000:00000080:2.0:1787826879.644880:0:81286:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.644882:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00800000:00000080:2.0:1787826879.644883:0:81286:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.644885:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00800000:00000080:2.0:1787826879.644887:0:81286:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.644889:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00800000:00000080:2.0:1787826879.644891:0:81286:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.644893:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00800000:00000080:2.0:1787826879.644894:0:81286:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.644896:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00800000:00000080:2.0:1787826879.644898:0:81286:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.644900:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00800000:00000080:2.0:1787826879.644901:0:81286:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.644903:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00800000:00000080:2.0:1787826879.644905:0:81286:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.644907:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00800000:00000080:2.0:1787826879.644908:0:81286:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.644910:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00800000:00000080:2.0:1787826879.644912:0:81286:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.644915:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00800000:00000080:2.0:1787826879.644916:0:81286:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.644919:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00800000:00000080:2.0:1787826879.644920:0:81286:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.644923:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00800000:00000080:2.0:1787826879.644924:0:81286:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.644926:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00800000:00000080:2.0:1787826879.644928:0:81286:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.644930:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00800000:00000080:2.0:1787826879.644931:0:81286:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.644933:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00800000:00000080:2.0:1787826879.644935:0:81286:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.644937:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00800000:00000080:2.0:1787826879.644939:0:81286:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.644941:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00800000:00000080:2.0:1787826879.644942:0:81286:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.644944:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00800000:00000080:2.0:1787826879.644946:0:81286:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.644948:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00800000:00000080:2.0:1787826879.644950:0:81286:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.644952:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00800000:00000080:2.0:1787826879.644953:0:81286:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.644955:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00800000:00000080:2.0:1787826879.644957:0:81286:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.644959:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00800000:00000080:2.0:1787826879.644961:0:81286:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.644963:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00800000:00000080:2.0:1787826879.644965:0:81286:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.644967:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00800000:00000080:2.0:1787826879.644969:0:81286:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.644971:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00800000:00000080:2.0:1787826879.644973:0:81286:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.644975:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00800000:00000080:2.0:1787826879.644976:0:81286:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.644978:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00800000:00000080:2.0:1787826879.644980:0:81286:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.644982:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00800000:00000080:2.0:1787826879.644983:0:81286:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.644985:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00800000:00000080:2.0:1787826879.644987:0:81286:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.644989:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00800000:00000080:2.0:1787826879.644991:0:81286:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.644993:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00800000:00000080:2.0:1787826879.644994:0:81286:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.644996:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00800000:00000080:2.0:1787826879.644998:0:81286:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.645000:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00800000:00000080:2.0:1787826879.645001:0:81286:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.645003:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00800000:00000080:2.0:1787826879.645005:0:81286:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.645007:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00800000:00000080:2.0:1787826879.645009:0:81286:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.645012:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00800000:00000080:2.0:1787826879.645013:0:81286:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.645015:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00800000:00000080:2.0:1787826879.645017:0:81286:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.645019:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00800000:00000080:2.0:1787826879.645021:0:81286:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.645023:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00800000:00000080:2.0:1787826879.645024:0:81286:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.645026:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00800000:00000080:2.0:1787826879.645028:0:81286:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.645030:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00800000:00000080:2.0:1787826879.645032:0:81286:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.645033:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00800000:00000080:2.0:1787826879.645035:0:81286:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.645037:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00020000:00000080:2.0:1787826879.645041:0:81286:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00020000:00000004:2.0:1787826879.645046:0:81286:0:(obd_class.h:1091:obd_statfs()) lustre-OST0000-osc-ffff985602bc9800: age 3029, max_age 3029 00020000:00000004:2.0:1787826879.645048:0:81286:0:(obd_class.h:1131:obd_statfs()) lustre-OST0000-osc-ffff985602bc9800: use ffff985609528450 cache blocks 901357/958279 objects 261728/262144 00000080:00200080:2.0:1787826879.645064:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00020000:00000080:2.0:1787826879.645068:0:81286:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00020000:00000004:2.0:1787826879.645070:0:81286:0:(obd_class.h:1091:obd_statfs()) lustre-OST0001-osc-ffff985602bc9800: age 3029, max_age 3029 00020000:00000004:2.0:1787826879.645071:0:81286:0:(obd_class.h:1131:obd_statfs()) lustre-OST0001-osc-ffff985602bc9800: use ffff98560952d1c0 cache blocks 901363/958279 objects 261735/262144 00000080:00200080:2.0:1787826879.645078:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00020000:00000080:2.0:1787826879.645081:0:81286:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.645084:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00020000:00000080:2.0:1787826879.645086:0:81286:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.645088:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00020000:00000080:2.0:1787826879.645090:0:81286:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.645093:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00020000:00000080:2.0:1787826879.645095:0:81286:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.645098:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00020000:00000080:2.0:1787826879.645100:0:81286:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.645102:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00020000:00000080:2.0:1787826879.645104:0:81286:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.645106:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00020000:00000080:2.0:1787826879.645108:0:81286:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.645112:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00020000:00000080:2.0:1787826879.645116:0:81286:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.645120:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00020000:00000080:2.0:1787826879.645125:0:81286:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.645129:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00020000:00000080:2.0:1787826879.645133:0:81286:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.645138:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00020000:00000080:2.0:1787826879.645142:0:81286:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.645146:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00020000:00000080:2.0:1787826879.645149:0:81286:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.645154:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00020000:00000080:2.0:1787826879.645158:0:81286:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.645163:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00020000:00000080:2.0:1787826879.645167:0:81286:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.645173:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00020000:00000080:2.0:1787826879.645177:0:81286:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.645183:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00020000:00000080:2.0:1787826879.645188:0:81286:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.645193:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00020000:00000080:2.0:1787826879.645197:0:81286:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.645201:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00020000:00000080:2.0:1787826879.645204:0:81286:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.645208:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00020000:00000080:2.0:1787826879.645212:0:81286:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.645217:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00020000:00000080:2.0:1787826879.645221:0:81286:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.645225:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00020000:00000080:2.0:1787826879.645230:0:81286:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.645235:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00020000:00000080:2.0:1787826879.645239:0:81286:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.645244:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00020000:00000080:2.0:1787826879.645248:0:81286:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.645253:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00020000:00000080:2.0:1787826879.645257:0:81286:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.645261:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00020000:00000080:2.0:1787826879.645266:0:81286:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.645271:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00020000:00000080:2.0:1787826879.645275:0:81286:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.645279:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00020000:00000080:2.0:1787826879.645283:0:81286:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.645287:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00020000:00000080:2.0:1787826879.645292:0:81286:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.645296:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00020000:00000080:2.0:1787826879.645300:0:81286:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.645304:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00020000:00000080:2.0:1787826879.645307:0:81286:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.645311:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00020000:00000080:2.0:1787826879.645314:0:81286:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.645318:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00020000:00000080:2.0:1787826879.645322:0:81286:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.645325:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00020000:00000080:2.0:1787826879.645334:0:81286:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.645339:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00020000:00000080:2.0:1787826879.645342:0:81286:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.645346:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00020000:00000080:2.0:1787826879.645350:0:81286:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.645353:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00020000:00000080:2.0:1787826879.645357:0:81286:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.645361:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00020000:00000080:2.0:1787826879.645364:0:81286:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.645368:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00020000:00000080:2.0:1787826879.645371:0:81286:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.645376:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00020000:00000080:2.0:1787826879.645379:0:81286:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.645383:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00020000:00000080:2.0:1787826879.645387:0:81286:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.645391:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00020000:00000080:2.0:1787826879.645394:0:81286:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.645398:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00020000:00000080:2.0:1787826879.645401:0:81286:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.645405:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00020000:00000080:2.0:1787826879.645409:0:81286:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.645413:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00020000:00000080:2.0:1787826879.645416:0:81286:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.645420:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00020000:00000080:2.0:1787826879.645423:0:81286:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.645427:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00020000:00000080:2.0:1787826879.645430:0:81286:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.645434:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00020000:00000080:2.0:1787826879.645438:0:81286:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.645441:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00020000:00000080:2.0:1787826879.645445:0:81286:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.645448:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00020000:00000080:2.0:1787826879.645451:0:81286:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.645455:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00020000:00000080:2.0:1787826879.645458:0:81286:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.645462:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00020000:00000080:2.0:1787826879.645465:0:81286:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.645469:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00020000:00000080:2.0:1787826879.645472:0:81286:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.645476:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00020000:00000080:2.0:1787826879.645479:0:81286:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.645483:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00020000:00000080:2.0:1787826879.645486:0:81286:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.645490:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00020000:00000080:2.0:1787826879.645494:0:81286:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.645497:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00020000:00000080:2.0:1787826879.645501:0:81286:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.645504:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00020000:00000080:2.0:1787826879.645507:0:81286:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.645511:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00020000:00000080:2.0:1787826879.645514:0:81286:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.645518:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00020000:00000080:2.0:1787826879.645521:0:81286:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.645525:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00020000:00000080:2.0:1787826879.645528:0:81286:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.645532:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00020000:00000080:2.0:1787826879.645535:0:81286:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.645539:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00020000:00000080:2.0:1787826879.645542:0:81286:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200080:2.0:1787826879.645545:0:81286:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeadefaa90 00020000:00000080:2.0:1787826879.645548:0:81286:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e800 uarg=0000000000000000 00000080:00200008:2.0:1787826879.645557:0:81286:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 100000 10000000:01000000:3.0:1787826879.732833:0:15166:0:(mgc_request.c:611:do_requeue()) updating log lustre-cliir 10000000:01000000:3.0:1787826879.732843:0:15166:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-cliir-ffff985602bc9800 from 192 10000000:01000000:3.0:1787826879.732851:0:15166:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-cliir (res 0x65727473756c) 00010000:00010000:3.0:1787826879.732919:0:15166:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.114@tcp lock: ffff985610e07600/0xb3f83dad1b7e0948 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826879.732933:0:15166:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.114@tcp lock: ffff985610e07600/0xb3f83dad1b7e0948 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826879.732946:0:15166:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.114@tcp lock: ffff985610e07600/0xb3f83dad1b7e0948 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826879.732983:0:15166:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e4e1c0 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669225585536:192.168.203.114@tcp:101:ll_cfg_requeue.0 00000100:00100000:3.0:1787826879.733118:0:15166:0:(client.c:2715:ptlrpc_set_wait()) set ffff985608840b40 going to sleep for 11 seconds 00000100:00100000:3.0:1787826879.734549:0:15166:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e4e1c0 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669225585536:192.168.203.114@tcp:101:ll_cfg_requeue.0 00010000:00010000:3.0:1787826879.734571:0:15166:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.114@tcp lock: ffff985610e07600/0xb3f83dad1b7e0948 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x5aa6d504d91a2535 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826879.734580:0:15166:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.114@tcp lock: ffff985610e07600/0xb3f83dad1b7e0948 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x5aa6d504d91a2535 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826879.734661:0:15166:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e4e1c0 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669225585664:192.168.203.114@tcp:256:ll_cfg_requeue.0 00000100:00100000:3.0:1787826879.734761:0:15166:0:(client.c:2715:ptlrpc_set_wait()) set ffff985608840b40 going to sleep for 11 seconds 00000020:01000000:3.0:1787826879.741062:0:15166:0:(obd_config.c:2145:class_config_parse_llog()) Processed log lustre-client gen 52-51 (rc=0) 10000000:01000000:3.0:1787826879.741086:0:15166:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.114@tcp: configuration from log 'lustre-client' succeeded (0). 00010000:00010000:3.0:1787826879.741099:0:15166:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.114@tcp lock: ffff985609eb1e00/0xb3f83dad1b7e094f lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x5aa6d504d91a253c expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826879.741111:0:15166:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.114@tcp lock: ffff985609eb1e00/0xb3f83dad1b7e094f lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x5aa6d504d91a253c expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1787826879.741121:0:15166:0:(mgc_request.c:611:do_requeue()) updating log params 10000000:01000000:3.0:1787826879.741125:0:15166:0:(mgc_request.c:1802:mgc_process_log()) Process log params-ffff985602bc9800 from 7 10000000:01000000:3.0:1787826879.741128:0:15166:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for params (res 0x736d61726170) 00010000:00010000:3.0:1787826879.741159:0:15166:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.114@tcp lock: ffff985610e07200/0xb3f83dad1b7e0956 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826879.741167:0:15166:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.114@tcp lock: ffff985610e07200/0xb3f83dad1b7e0956 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826879.741177:0:15166:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.114@tcp lock: ffff985610e07200/0xb3f83dad1b7e0956 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826879.741196:0:15166:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e4fc00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669225586176:192.168.203.114@tcp:101:ll_cfg_requeue.0 00000100:00100000:3.0:1787826879.741278:0:15166:0:(client.c:2715:ptlrpc_set_wait()) set ffff9856088406c0 going to sleep for 11 seconds 00000020:01000000:3.0:1787826879.745764:0:15166:0:(obd_config.c:2145:class_config_parse_llog()) Processed log params gen 7-6 (rc=0) 10000000:01000000:3.0:1787826879.745788:0:15166:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.114@tcp: configuration from log 'params' succeeded (0). 00010000:00010000:3.0:1787826879.745801:0:15166:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.114@tcp lock: ffff985610e07200/0xb3f83dad1b7e0956 lrc: 3/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x5aa6d504d91a2543 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826879.745814:0:15166:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.114@tcp lock: ffff985610e07200/0xb3f83dad1b7e0956 lrc: 2/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x5aa6d504d91a2543 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1787826879.745838:0:15166:0:(mgc_request.c:611:do_requeue()) updating log lustre-sptlrpc 10000000:01000000:3.0:1787826879.745841:0:15166:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-sptlrpc-ffff985606a30000 from 1 10000000:01000000:3.0:1787826879.745844:0:15166:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-sptlrpc (res 0x65727473756c) 00010000:00010000:3.0:1787826879.745879:0:15166:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.114@tcp lock: ffff985610e06200/0xb3f83dad1b7e095d lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826879.745889:0:15166:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.114@tcp lock: ffff985610e06200/0xb3f83dad1b7e095d lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826879.745899:0:15166:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.114@tcp lock: ffff985610e06200/0xb3f83dad1b7e095d lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826879.745921:0:15166:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e4c000 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669225586560:192.168.203.114@tcp:101:ll_cfg_requeue.0 00000100:00100000:3.0:1787826879.745991:0:15166:0:(client.c:2715:ptlrpc_set_wait()) set ffff985608840180 going to sleep for 11 seconds 00010000:00010000:1.0:1787826880.308891:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826881.332857:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826882.356900:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826882.804964:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2385:1874669225586816:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787826882.804993:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669225586944:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787826882.805072:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669225587200:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787826882.805075:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee1e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669225587072:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787826882.805099:0:2386:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1787826882.805105:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669225587328:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787826882.807110:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2385:1874669225586816:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787826882.807138:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee1e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669225587072:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787826882.807157:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669225587328:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787826882.807169:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669225586944:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787826882.807178:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669225587200:192.168.203.114@tcp:400:kworker.0 00010000:00010000:3.0:1787826884.404866:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826885.428955:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200000:1.0:1787826885.813889:0:81426:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826885.814024:0:81426:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826885.814041:0:81426:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1c00/0xb3f83dad1b7e0941 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a252e expref: -99 pid: 81234 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 0 00010000:00010000:1.0:1787826885.814056:0:81426:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1c00/0xb3f83dad1b7e0941 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a252e expref: -99 pid: 81234 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 0 00000080:00200000:1.0:1787826885.814064:0:81426:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826885.814072:0:81426:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826885.814075:0:81426:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e0941 00000080:00010000:1.0:1787826885.814083:0:81426:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0941 from it ffffab3ac1db7b58 00010000:00010000:1.0:1787826885.814087:0:81426:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1c00/0xb3f83dad1b7e0941 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a252e expref: -99 pid: 81234 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 0 00010000:00010000:1.0:1787826885.814096:0:81426:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1c00/0xb3f83dad1b7e0941 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a252e expref: -99 pid: 81234 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 1 00000080:00200000:1.0:1787826885.814109:0:81426:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787826885.814121:0:81426:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f35.replay-single, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), flags=8 00000080:00200000:1.0:1787826885.814126:0:81426:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f35.replay-single, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=getattr 00000002:00010000:1.0:1787826885.814137:0:81426:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f35.replay-single,[0x0:0x0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 01 00010000:00010000:1.0:1787826885.814172:0:81426:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050400/0xb3f83dad1b7e0964 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 81426 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826885.814178:0:81426:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050400/0xb3f83dad1b7e0964 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 81426 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826885.814184:0:81426:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050400/0xb3f83dad1b7e0964 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 81426 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826885.814204:0:81426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610db4780 pname:cluuid:pid:xid:nid:opc:job checkstat:7b4e8535-9db7-485d-abf6-bda966b63320:81426:1874669225587456:192.168.203.114@tcp:101:checkstat.0 00000100:00100000:1.0:1787826885.814275:0:81426:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607ee26c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826885.816143:0:81426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610db4780 pname:cluuid:pid:xid:nid:opc:job checkstat:7b4e8535-9db7-485d-abf6-bda966b63320:81426:1874669225587456:192.168.203.114@tcp:101:checkstat.0 00010000:00010000:1.0:1787826885.816157:0:81426:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050400/0xb3f83dad1b7e0964 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 81426 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826885.816164:0:81426:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050400/0xb3f83dad1b7e0964 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 81426 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826885.816171:0:81426:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050400/0xb3f83dad1b7e0964 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 81426 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826885.816175:0:81426:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050400/0xb3f83dad1b7e0964 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 81426 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826885.816180:0:81426:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050400/0xb3f83dad1b7e0964 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 81426 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826885.816184:0:81426:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985642050400 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:1.0:1787826885.816194:0:81426:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050400/0xb3f83dad1b7e0964 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 81426 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826885.816200:0:81426:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050400/0xb3f83dad1b7e0964 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 81426 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826885.816206:0:81426:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050400/0xb3f83dad1b7e0964 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x20e400000000 pid: 81426 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826885.816212:0:81426:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787826885.816217:0:81426:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050400/0xb3f83dad1b7e0964 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 81426 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826885.816224:0:81426:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050400/0xb3f83dad1b7e0964 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 81426 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826885.816239:0:81426:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff985610db4780 x1874669225587456/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/584 e 0 to 0 dl 1787826901 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'checkstat.0' uid:0 gid:0 projid:0 00000080:00200000:1.0:1787826885.816250:0:81426:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00010000:00010000:1.0:1787826885.816266:0:81426:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1c00/0xb3f83dad1b7e0941 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a252e expref: -99 pid: 81234 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 1 00010000:00010000:1.0:1787826885.816273:0:81426:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1c00/0xb3f83dad1b7e0941 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a252e expref: -99 pid: 81234 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 1 00000080:00010000:1.0:1787826885.816277:0:81426:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0941 from it ffffab3ac1db7930 00010000:00010000:1.0:1787826885.816280:0:81426:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1c00/0xb3f83dad1b7e0941 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a252e expref: -99 pid: 81234 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 1 00010000:00010000:1.0:1787826885.816284:0:81426:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1c00/0xb3f83dad1b7e0941 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a252e expref: -99 pid: 81234 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 1 00010000:00010000:2.0:1787826887.477781:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:2.0:1787826887.890785:0:81979:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: SKIP: replay-single test_36 skipping ALWAYS excluded test 36 00000100:00100000:1.0:1787826887.925909:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e792c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2383:1874669225587584:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787826887.926008:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e79e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669225587840:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787826887.926038:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e78f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669225588096:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787826887.927075:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e792c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2383:1874669225587584:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787826887.927097:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e79e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669225587840:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787826887.927123:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e78f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669225588096:192.168.203.114@tcp:400:kworker.0 00000001:02000400:2.0:1787826888.479942:0:82130:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == replay-single test 37: abort recovery before client does replay (test mds_cleanup_orphans for directories) ========================================================== 06:34:47 (1787826887) 00010000:00010000:1.0:1787826888.501942:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200000:1.0:1787826888.502066:0:82133:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826888.502095:0:82133:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826888.502113:0:82133:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1c00/0xb3f83dad1b7e0941 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a252e expref: -99 pid: 81234 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 1 00010000:00010000:1.0:1787826888.502137:0:82133:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1c00/0xb3f83dad1b7e0941 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a252e expref: -99 pid: 81234 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 1 00000080:00200000:1.0:1787826888.502146:0:82133:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826888.502156:0:82133:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826888.502168:0:82133:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e0941 00000080:00010000:1.0:1787826888.502176:0:82133:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0941 from it ffffab3ac1e2fb58 00010000:00010000:1.0:1787826888.502180:0:82133:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1c00/0xb3f83dad1b7e0941 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a252e expref: -99 pid: 81234 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 1 00010000:00010000:1.0:1787826888.502190:0:82133:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1c00/0xb3f83dad1b7e0941 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a252e expref: -99 pid: 81234 timeout: 0 lvb_type: 0 lru_score: 9 lru_type: 1 00000080:00200000:1.0:1787826888.502201:0:82133:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787826888.502218:0:82133:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=d37.replay-single, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), flags=13 00000080:00200000:1.0:1787826888.502222:0:82133:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=d37.replay-single, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=getattr 00000002:00010000:1.0:1787826888.502236:0:82133:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: d37.replay-single,[0x0:0x0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 01 00010000:00010000:1.0:1787826888.502291:0:82133:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e096b lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 82133 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826888.502299:0:82133:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e096b lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 82133 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826888.502309:0:82133:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e096b lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 82133 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826888.502335:0:82133:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610db4000 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:82133:1874669225588224:192.168.203.114@tcp:101:lfs.0 00000100:00100000:1.0:1787826888.502419:0:82133:0:(client.c:2715:ptlrpc_set_wait()) set ffff9856108f2f00 going to sleep for 11 seconds 00000100:00100000:1.0:1787826888.504101:0:82133:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610db4000 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:82133:1874669225588224:192.168.203.114@tcp:101:lfs.0 00010000:00010000:1.0:1787826888.504118:0:82133:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e096b lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 82133 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826888.504129:0:82133:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e096b lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 82133 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826888.504137:0:82133:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e096b lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 82133 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826888.504144:0:82133:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e096b lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 82133 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826888.504152:0:82133:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e096b lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 82133 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826888.504160:0:82133:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985642050000 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:1.0:1787826888.504165:0:82133:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e096b lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 82133 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826888.504173:0:82133:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e096b lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 82133 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826888.504180:0:82133:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e096b lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x20e400000000 pid: 82133 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826888.504188:0:82133:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787826888.504197:0:82133:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e096b lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 82133 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826888.504208:0:82133:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985642050000/0xb3f83dad1b7e096b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 82133 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826888.504225:0:82133:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff985610db4000 x1874669225588224/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/584 e 0 to 0 dl 1787826904 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:1.0:1787826888.504239:0:82133:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00010000:00010000:1.0:1787826888.504256:0:82133:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1c00/0xb3f83dad1b7e0941 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a252e expref: -99 pid: 81234 timeout: 0 lvb_type: 0 lru_score: 9 lru_type: 1 00010000:00010000:1.0:1787826888.504268:0:82133:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1c00/0xb3f83dad1b7e0941 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a252e expref: -99 pid: 81234 timeout: 0 lvb_type: 0 lru_score: 9 lru_type: 1 00000080:00010000:1.0:1787826888.504277:0:82133:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0941 from it ffffab3ac1e2f930 00010000:00010000:1.0:1787826888.504281:0:82133:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1c00/0xb3f83dad1b7e0941 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a252e expref: -99 pid: 81234 timeout: 0 lvb_type: 0 lru_score: 9 lru_type: 1 00010000:00010000:1.0:1787826888.504290:0:82133:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1c00/0xb3f83dad1b7e0941 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a252e expref: -99 pid: 81234 timeout: 0 lvb_type: 0 lru_score: 10 lru_type: 1 00000080:00200000:1.0:1787826888.504344:0:82133:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826888.504356:0:82133:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826888.504363:0:82133:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1c00/0xb3f83dad1b7e0941 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a252e expref: -99 pid: 81234 timeout: 0 lvb_type: 0 lru_score: 10 lru_type: 1 00010000:00010000:1.0:1787826888.504371:0:82133:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1c00/0xb3f83dad1b7e0941 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a252e expref: -99 pid: 81234 timeout: 0 lvb_type: 0 lru_score: 10 lru_type: 1 00000080:00200000:1.0:1787826888.504384:0:82133:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826888.504390:0:82133:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826888.504393:0:82133:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e0941 00000080:00010000:1.0:1787826888.504397:0:82133:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0941 from it ffffab3ac1e2fbe8 00010000:00010000:1.0:1787826888.504400:0:82133:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1c00/0xb3f83dad1b7e0941 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a252e expref: -99 pid: 81234 timeout: 0 lvb_type: 0 lru_score: 10 lru_type: 1 00010000:00010000:1.0:1787826888.504408:0:82133:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1c00/0xb3f83dad1b7e0941 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a252e expref: -99 pid: 81234 timeout: 0 lvb_type: 0 lru_score: 11 lru_type: 1 00000080:00200000:1.0:1787826888.504416:0:82133:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787826888.504422:0:82133:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d37.replay-single, flags=16392 00000080:00200008:1.0:1787826888.504451:0:82133:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 0, flags 0, foreign 0 00000080:00200000:1.0:1787826888.504456:0:82133:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826888.504463:0:82133:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826888.504469:0:82133:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1c00/0xb3f83dad1b7e0941 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a252e expref: -99 pid: 81234 timeout: 0 lvb_type: 0 lru_score: 11 lru_type: 1 00010000:00010000:1.0:1787826888.504478:0:82133:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1c00/0xb3f83dad1b7e0941 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a252e expref: -99 pid: 81234 timeout: 0 lvb_type: 0 lru_score: 11 lru_type: 1 00000080:00200000:1.0:1787826888.504485:0:82133:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826888.504489:0:82133:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826888.504491:0:82133:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e0941 00000080:00010000:1.0:1787826888.504496:0:82133:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0941 from it ffffab3ac1e2fcb8 00010000:00010000:1.0:1787826888.504501:0:82133:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1c00/0xb3f83dad1b7e0941 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a252e expref: -99 pid: 81234 timeout: 0 lvb_type: 0 lru_score: 11 lru_type: 1 00010000:00010000:1.0:1787826888.504509:0:82133:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1c00/0xb3f83dad1b7e0941 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a252e expref: -99 pid: 81234 timeout: 0 lvb_type: 0 lru_score: 12 lru_type: 1 00000080:00200008:1.0:1787826888.505167:0:82133:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 0, flags 0, foreign 0 00000080:00200000:1.0:1787826888.505174:0:82133:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826888.505184:0:82133:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826888.505193:0:82133:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1c00/0xb3f83dad1b7e0941 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a252e expref: -99 pid: 81234 timeout: 0 lvb_type: 0 lru_score: 12 lru_type: 1 00010000:00010000:1.0:1787826888.505203:0:82133:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1c00/0xb3f83dad1b7e0941 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a252e expref: -99 pid: 81234 timeout: 0 lvb_type: 0 lru_score: 12 lru_type: 1 00000080:00200000:1.0:1787826888.505209:0:82133:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826888.505214:0:82133:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826888.505217:0:82133:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e0941 00000080:00010000:1.0:1787826888.505222:0:82133:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0941 from it ffffab3ac1e2fcb8 00010000:00010000:1.0:1787826888.505226:0:82133:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1c00/0xb3f83dad1b7e0941 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a252e expref: -99 pid: 81234 timeout: 0 lvb_type: 0 lru_score: 12 lru_type: 1 00010000:00010000:1.0:1787826888.505234:0:82133:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1c00/0xb3f83dad1b7e0941 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a252e expref: -99 pid: 81234 timeout: 0 lvb_type: 0 lru_score: 13 lru_type: 1 00000080:00200000:1.0:1787826888.505409:0:82133:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826888.505418:0:82133:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826888.505425:0:82133:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1c00/0xb3f83dad1b7e0941 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a252e expref: -99 pid: 81234 timeout: 0 lvb_type: 0 lru_score: 13 lru_type: 1 00010000:00010000:1.0:1787826888.505438:0:82133:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1c00/0xb3f83dad1b7e0941 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a252e expref: -99 pid: 81234 timeout: 0 lvb_type: 0 lru_score: 13 lru_type: 1 00000080:00200000:1.0:1787826888.505445:0:82133:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826888.505450:0:82133:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826888.505452:0:82133:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e0941 00000080:00010000:1.0:1787826888.505456:0:82133:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0941 from it ffffab3ac1e2fbb0 00010000:00010000:1.0:1787826888.505459:0:82133:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1c00/0xb3f83dad1b7e0941 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a252e expref: -99 pid: 81234 timeout: 0 lvb_type: 0 lru_score: 13 lru_type: 1 00010000:00010000:1.0:1787826888.505467:0:82133:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1c00/0xb3f83dad1b7e0941 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a252e expref: -99 pid: 81234 timeout: 0 lvb_type: 0 lru_score: 14 lru_type: 1 00000080:00200000:1.0:1787826888.505477:0:82133:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 44 00000080:00200008:1.0:1787826888.505483:0:82133:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 300000 00000080:00200080:1.0:1787826888.505498:0:82133:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c03066f0 arg=7ffce2666d50 00000080:00200000:1.0:1787826888.505509:0:82133:0:(dir.c:451:ll_dir_setdirstripe()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) name=d37.replay-single stripe_offset=0 stripe_count=1, hash_type=0 00000100:00100000:1.0:1787826888.505554:0:82133:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610db4000 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:82133:1874669225588352:192.168.203.114@tcp:700:lfs.0 00000100:00100000:1.0:1787826888.505627:0:82133:0:(client.c:2715:ptlrpc_set_wait()) set ffff9856108f2f00 going to sleep for 11 seconds 00000002:00100000:0.0:1787826888.517091:0:82136:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff985610e78000 x1874669225588736/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826904 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787826888.517122:0:82136:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1200/0xb3f83dad1b7e0972 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a25ba expref: -99 pid: 82136 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826888.517135:0:82136:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:0.0:1787826888.517142:0:82136:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:0.0:1787826888.517174:0:82136:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:0.0:1787826888.517183:0:82136:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e0972 00000080:00010000:0.0:1787826888.517189:0:82136:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:0.0:1787826888.517194:0:82136:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0972 from it ffffab3ac1c97c20 00010000:00010000:0.0:1787826888.517198:0:82136:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1200/0xb3f83dad1b7e0972 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a25ba expref: -99 pid: 82136 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826888.517217:0:82136:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1200/0xb3f83dad1b7e0972 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a25ba expref: -99 pid: 82136 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826888.517238:0:82136:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:0.0:1787826888.517253:0:82136:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=d37.replay-single, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), flags=24 00000080:00200000:0.0:1787826888.517258:0:82136:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=d37.replay-single, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=0 00000002:00010000:0.0:1787826888.517276:0:82136:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: d37.replay-single,[0x0:0x0:0x0]) in obj [0x200000007:0x1:0x0], intent: lookup flags 0 00010000:00010000:0.0:1787826888.517309:0:82136:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1c00/0xb3f83dad1b7e0979 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 82136 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826888.517317:0:82136:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1c00/0xb3f83dad1b7e0979 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 82136 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826888.517325:0:82136:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1c00/0xb3f83dad1b7e0979 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 82136 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826888.517355:0:82136:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e78000 pname:cluuid:pid:xid:nid:opc:job rmdir:7b4e8535-9db7-485d-abf6-bda966b63320:82136:1874669225588864:192.168.203.114@tcp:101:rmdir.0 00000100:00100000:0.0:1787826888.517409:0:82136:0:(client.c:2715:ptlrpc_set_wait()) set ffff985605437b40 going to sleep for 11 seconds 00000100:00100000:0.0:1787826888.518956:0:82136:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e78000 pname:cluuid:pid:xid:nid:opc:job rmdir:7b4e8535-9db7-485d-abf6-bda966b63320:82136:1874669225588864:192.168.203.114@tcp:101:rmdir.0 00010000:00010000:0.0:1787826888.518982:0:82136:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1c00/0xb3f83dad1b7e0979 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a25d6 expref: -99 pid: 82136 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826888.519001:0:82136:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1c00/0xb3f83dad1b7e0979 lrc: 4/1,0 mode: --/PR res: [0x200017b01:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a25d6 expref: -99 pid: 82136 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826888.519008:0:82136:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1c00/0xb3f83dad1b7e0979 lrc: 4/1,0 mode: --/PR res: [0x200017b01:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a25d6 expref: -99 pid: 82136 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826888.519017:0:82136:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1c00/0xb3f83dad1b7e0979 lrc: 4/1,0 mode: PR/PR res: [0x200017b01:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a25d6 expref: -99 pid: 82136 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826888.519033:0:82136:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1c00/0xb3f83dad1b7e0979 lrc: 4/1,0 mode: PR/PR res: [0x200017b01:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a25d6 expref: -99 pid: 82136 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826888.519044:0:82136:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1c00/0xb3f83dad1b7e0979 lrc: 5/2,0 mode: PR/PR res: [0x200017b01:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a25d6 expref: -99 pid: 82136 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826888.519052:0:82136:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1c00/0xb3f83dad1b7e0979 lrc: 5/2,0 mode: PR/PR res: [0x200017b01:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a25d6 expref: -99 pid: 82136 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826888.519059:0:82136:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1c00/0xb3f83dad1b7e0979 lrc: 4/1,0 mode: PR/PR res: [0x200017b01:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a25d6 expref: -99 pid: 82136 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787826888.519076:0:82136:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x10 disposition=0xb, status=0 req@ffff985610e78000 x1874669225588864/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/688 e 0 to 0 dl 1787826904 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787826888.519089:0:82136:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1c00/0xb3f83dad1b7e0979 lrc: 3/1,0 mode: PR/PR res: [0x200017b01:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a25d6 expref: -99 pid: 82136 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826888.519098:0:82136:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8590031617/1 (0 0) 00000080:00200000:0.0:1787826888.519104:0:82136:0:(namei.c:952:ll_intent_lock()) intent lock 16 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:0.0:1787826888.519125:0:82136:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200017b01:0x1:0x0], updating i_size 4096 00000080:00200000:0.0:1787826888.519128:0:82136:0:(namei.c:147:ll_iget()) got inode: [0x200017b01:0x1:0x0](ffff98561166c2c8): rc = 0 00000080:00010000:0.0:1787826888.519133:0:82136:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200017b01:0x1:0x0](ffff98561166c2c8) for lock 0xb3f83dad1b7e0979 00000080:00010000:0.0:1787826888.519145:0:82136:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200017b01:0x1:0x0](ffff98561166c2c8) 00000080:00010000:0.0:1787826888.519149:0:82136:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0979 from it ffffab3ac1c97af0 00010000:00010000:0.0:1787826888.519153:0:82136:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1c00/0xb3f83dad1b7e0979 lrc: 3/1,0 mode: PR/PR res: [0x200017b01:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a25d6 expref: -99 pid: 82136 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826888.519162:0:82136:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1c00/0xb3f83dad1b7e0979 lrc: 2/0,0 mode: PR/PR res: [0x200017b01:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a25d6 expref: -99 pid: 82136 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826888.519187:0:82136:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200017b01:0x1:0x0](ffff98561166c2c8), inode mode 41ed mask 1 00000080:00200000:0.0:1787826888.519206:0:82136:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f37.replay-single, dir=[0x200017b01:0x1:0x0](ffff98561166c2c8), flags=0 00000080:00200000:0.0:1787826888.519212:0:82136:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f37.replay-single, dir=[0x200017b01:0x1:0x0](ffff98561166c2c8), intent=getattr 00000002:00010000:0.0:1787826888.519227:0:82136:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f37.replay-single,[0x0:0x0:0x0]) in obj [0x200017b01:0x1:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787826888.519258:0:82136:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0800/0xb3f83dad1b7e0980 lrc: 3/1,0 mode: --/CR res: [0x200017b01:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 82136 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826888.519267:0:82136:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0800/0xb3f83dad1b7e0980 lrc: 3/1,0 mode: --/CR res: [0x200017b01:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 82136 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826888.519276:0:82136:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0800/0xb3f83dad1b7e0980 lrc: 3/1,0 mode: --/CR res: [0x200017b01:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 82136 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826888.519293:0:82136:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e78000 pname:cluuid:pid:xid:nid:opc:job rmdir:7b4e8535-9db7-485d-abf6-bda966b63320:82136:1874669225588992:192.168.203.114@tcp:101:rmdir.0 00000100:00100000:0.0:1787826888.519356:0:82136:0:(client.c:2715:ptlrpc_set_wait()) set ffff985605437b40 going to sleep for 11 seconds 00000100:00100000:0.0:1787826888.520770:0:82136:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e78000 pname:cluuid:pid:xid:nid:opc:job rmdir:7b4e8535-9db7-485d-abf6-bda966b63320:82136:1874669225588992:192.168.203.114@tcp:101:rmdir.0 00010000:00010000:0.0:1787826888.520793:0:82136:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0800/0xb3f83dad1b7e0980 lrc: 4/1,0 mode: --/CR res: [0x200017b01:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 82136 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826888.520804:0:82136:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0800/0xb3f83dad1b7e0980 lrc: 4/1,0 mode: --/CR res: [0x200017b01:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 82136 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826888.520810:0:82136:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0800/0xb3f83dad1b7e0980 lrc: 4/1,0 mode: --/CR res: [0x200017b01:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 82136 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826888.520815:0:82136:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0800/0xb3f83dad1b7e0980 lrc: 3/0,0 mode: --/CR res: [0x200017b01:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 82136 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826888.520821:0:82136:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0800/0xb3f83dad1b7e0980 lrc: 4/0,0 mode: --/CR res: [0x200017b01:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 82136 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826888.520825:0:82136:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985609eb0800 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:0.0:1787826888.520829:0:82136:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0800/0xb3f83dad1b7e0980 lrc: 4/0,0 mode: --/CR res: [0x200017b01:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 82136 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826888.520834:0:82136:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0800/0xb3f83dad1b7e0980 lrc: 4/0,0 mode: --/CR res: [0x200017b01:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 82136 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826888.520838:0:82136:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0800/0xb3f83dad1b7e0980 lrc: 5/0,0 mode: --/CR res: [0x200017b01:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x20e400000000 pid: 82136 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826888.520844:0:82136:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787826888.520850:0:82136:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0800/0xb3f83dad1b7e0980 lrc: 3/0,0 mode: --/CR res: [0x200017b01:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 82136 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826888.520854:0:82136:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0800/0xb3f83dad1b7e0980 lrc: 0/0,0 mode: --/CR res: [0x200017b01:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 82136 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787826888.520867:0:82136:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff985610e78000 x1874669225588992/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/584 e 0 to 0 dl 1787826904 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'rmdir.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787826888.520878:0:82136:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200017b01:0x1:0x0] suppgids 0 -1: rc 0 00010000:00010000:0.0:1787826888.520901:0:82136:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 30 res 8590031617/1 (0 0) 00000080:00200000:0.0:1787826888.537066:0:82141:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:0.0:1787826888.537102:0:82141:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826888.537122:0:82141:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1200/0xb3f83dad1b7e0972 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a25ba expref: -99 pid: 82136 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826888.537136:0:82141:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1200/0xb3f83dad1b7e0972 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a25ba expref: -99 pid: 82136 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826888.537143:0:82141:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826888.537152:0:82141:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:0.0:1787826888.537155:0:82141:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e0972 00000080:00010000:0.0:1787826888.537163:0:82141:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0972 from it ffffab3ac16afbe0 00010000:00010000:0.0:1787826888.537168:0:82141:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1200/0xb3f83dad1b7e0972 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a25ba expref: -99 pid: 82136 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826888.537179:0:82141:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1200/0xb3f83dad1b7e0972 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a25ba expref: -99 pid: 82136 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1787826888.537201:0:82141:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:0.0:1787826888.537210:0:82141:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d37.replay-single, flags=24 00000080:00200000:0.0:1787826888.537216:0:82141:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200017b01:0x1:0x0](ffff98561166c2c8), inode mode 41ed mask 1 00000080:00200000:0.0:1787826888.537228:0:82141:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f37.replay-single, dir=[0x200017b01:0x1:0x0](ffff98561166c2c8), flags=1536 00000080:00200000:0.0:1787826888.537232:0:82141:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200017b01:0x1:0x0](ffff98561166c2c8), inode mode 41ed mask 3 00000080:00200000:0.0:1787826888.537237:0:82141:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200017b01:0x1:0x0](ffff98561166c2c8), inode mode 41ed mask 3 00000080:00200000:0.0:1787826888.537243:0:82141:0:(namei.c:2239:do_mkdir()) VFS Op:name=f37.replay-single, dir=[0x200017b01:0x1:0x0](ffff98561166c2c8) 00000020:00100000:0.0:1787826888.537313:0:82141:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:0.0:1787826888.537332:0:82141:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610e78000 x1874669225589248/t0(0) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 568/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826888.537362:0:82141:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e78000 pname:cluuid:pid:xid:nid:opc:job multiop:7b4e8535-9db7-485d-abf6-bda966b63320:82141:1874669225589248:192.168.203.114@tcp:36:multiop.0 00000100:00100000:0.0:1787826888.537466:0:82141:0:(client.c:2715:ptlrpc_set_wait()) set ffff985605437b40 going to sleep for 11 seconds 00000100:00100000:0.0:1787826888.540100:0:82141:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e78000 pname:cluuid:pid:xid:nid:opc:job multiop:7b4e8535-9db7-485d-abf6-bda966b63320:82141:1874669225589248:192.168.203.114@tcp:36:multiop.0 00000080:00200000:0.0:1787826888.540223:0:82141:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200017b01:0x2:0x0](ffff98561166cb08) 00000080:00200000:0.0:1787826888.540231:0:82141:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200017b01:0x2:0x0], updating i_size 4096 00000080:00200000:0.0:1787826888.540284:0:82141:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:0.0:1787826888.540299:0:82141:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826888.540315:0:82141:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1200/0xb3f83dad1b7e0972 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a25ba expref: -99 pid: 82136 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1787826888.540330:0:82141:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1200/0xb3f83dad1b7e0972 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a25ba expref: -99 pid: 82136 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1787826888.540338:0:82141:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826888.540343:0:82141:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:0.0:1787826888.540348:0:82141:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e0972 00000080:00010000:0.0:1787826888.540355:0:82141:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0972 from it ffffab3ac16afbf8 00010000:00010000:0.0:1787826888.540359:0:82141:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1200/0xb3f83dad1b7e0972 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a25ba expref: -99 pid: 82136 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1787826888.540368:0:82141:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1200/0xb3f83dad1b7e0972 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a25ba expref: -99 pid: 82136 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:0.0:1787826888.540382:0:82141:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:0.0:1787826888.540400:0:82141:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d37.replay-single, flags=27 00000080:00200000:0.0:1787826888.540406:0:82141:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200017b01:0x1:0x0](ffff98561166c2c8), inode mode 41ed mask 1 00000080:00200000:0.0:1787826888.540418:0:82141:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f37.replay-single, dir=[0x200017b01:0x1:0x0](ffff98561166c2c8), file ffff9856091b3e00, open_flags 18000, mode 0 opened 0 00000080:00200000:0.0:1787826888.540425:0:82141:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f37.replay-single, dir=[0x200017b01:0x1:0x0](ffff98561166c2c8), intent=open 00000002:00010000:0.0:1787826888.540442:0:82141:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f37.replay-single,[0x0:0x0:0x0]) in obj [0x200017b01:0x1:0x0], intent: open flags 0300001 00010000:00010000:0.0:1787826888.540491:0:82141:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0800/0xb3f83dad1b7e0987 lrc: 3/1,0 mode: --/CR res: [0x200017b01:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 82141 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826888.540500:0:82141:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0800/0xb3f83dad1b7e0987 lrc: 3/1,0 mode: --/CR res: [0x200017b01:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 82141 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:0.0:1787826888.540511:0:82141:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:0.0:1787826888.540520:0:82141:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610e7a580 x1874669225589504/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/131856 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:0.0:1787826888.540530:0:82141:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0800/0xb3f83dad1b7e0987 lrc: 3/1,0 mode: --/CR res: [0x200017b01:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 82141 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826888.540548:0:82141:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e7a580 pname:cluuid:pid:xid:nid:opc:job multiop:7b4e8535-9db7-485d-abf6-bda966b63320:82141:1874669225589504:192.168.203.114@tcp:101:multiop.0 00000100:00100000:0.0:1787826888.540849:0:82141:0:(client.c:2715:ptlrpc_set_wait()) set ffff985605437b40 going to sleep for 11 seconds 00000100:00100000:0.0:1787826888.542522:0:82141:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e7a580 pname:cluuid:pid:xid:nid:opc:job multiop:7b4e8535-9db7-485d-abf6-bda966b63320:82141:1874669225589504:192.168.203.114@tcp:101:multiop.0 00010000:00010000:0.0:1787826888.542544:0:82141:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0800/0xb3f83dad1b7e0987 lrc: 4/1,0 mode: --/CR res: [0x200017b01:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 82141 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826888.542559:0:82141:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0800/0xb3f83dad1b7e0987 lrc: 4/1,0 mode: --/CR res: [0x200017b01:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 82141 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826888.542568:0:82141:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0800/0xb3f83dad1b7e0987 lrc: 4/1,0 mode: --/CR res: [0x200017b01:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 82141 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826888.542576:0:82141:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0800/0xb3f83dad1b7e0987 lrc: 3/0,0 mode: --/CR res: [0x200017b01:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 82141 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826888.542590:0:82141:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0800/0xb3f83dad1b7e0987 lrc: 4/0,0 mode: --/CR res: [0x200017b01:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 82141 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826888.542599:0:82141:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985609eb0800 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:0.0:1787826888.542605:0:82141:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0800/0xb3f83dad1b7e0987 lrc: 4/0,0 mode: --/CR res: [0x200017b01:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x206400000000 pid: 82141 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826888.542614:0:82141:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0800/0xb3f83dad1b7e0987 lrc: 4/0,0 mode: --/CR res: [0x200017b01:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x206400000000 pid: 82141 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826888.542622:0:82141:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0800/0xb3f83dad1b7e0987 lrc: 5/0,0 mode: --/CR res: [0x200017b01:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x20e400000000 pid: 82141 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826888.542630:0:82141:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787826888.542638:0:82141:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0800/0xb3f83dad1b7e0987 lrc: 3/0,0 mode: --/CR res: [0x200017b01:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 82141 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826888.542644:0:82141:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0800/0xb3f83dad1b7e0987 lrc: 0/0,0 mode: --/CR res: [0x200017b01:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 82141 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787826888.542666:0:82141:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x1 disposition=0x2b, status=0 req@ffff985610e7a580 x1874669225589504/t206158430211(206158430211) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/608 e 0 to 0 dl 1787826904 ref 2 fl Complete:RPQU/604/0 rc 301/301 job:'multiop.0' uid:0 gid:0 projid:0 00000002:00100000:0.0:1787826888.542688:0:82141:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff985610e7a580 x1874669225589504/t206158430211(206158430211) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/608 e 0 to 0 dl 1787826904 ref 2 fl Complete:RPQU/604/0 rc 301/301 job:'multiop.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787826888.542730:0:82141:0:(namei.c:952:ll_intent_lock()) intent lock 1 on i1 [0x200017b01:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:0.0:1787826888.542752:0:82141:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200017b01:0x2:0x0], updating i_size 4096 00000080:00200000:0.0:1787826888.542756:0:82141:0:(namei.c:147:ll_iget()) got inode: [0x200017b01:0x2:0x0](ffff98561166cb08): rc = 0 00000080:00010000:0.0:1787826888.542773:0:82141:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200017b01:0x2:0x0](ffff98561166cb08) for lock 0x0 00000080:00200008:0.0:1787826888.542798:0:82141:0:(file.c:1090:ll_file_open()) START file f37.replay-single:[0x200017b01:0x2:0x0](ffff98561166cb08), flags 300000 00000002:00100000:0.0:1787826888.542823:0:82141:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff985610e7a580 x1874669225589504/t206158430211(206158430211) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/608 e 0 to 0 dl 1787826904 ref 2 fl Complete:RPQU/604/0 rc 301/301 job:'multiop.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787826888.542879:0:82141:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200017b01:0x2:0x0](ffff98561166cb08), inode mode 41ed mask 44 00000080:00200000:0.0:1787826888.551659:0:82146:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:0.0:1787826888.551752:0:82146:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826888.551787:0:82146:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1200/0xb3f83dad1b7e0972 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a25ba expref: -99 pid: 82136 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787826888.551806:0:82146:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1200/0xb3f83dad1b7e0972 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a25ba expref: -99 pid: 82136 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:0.0:1787826888.551817:0:82146:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826888.551828:0:82146:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:0.0:1787826888.551833:0:82146:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e0972 00000080:00010000:0.0:1787826888.551841:0:82146:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0972 from it ffffab3ac1bafc20 00010000:00010000:0.0:1787826888.551846:0:82146:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1200/0xb3f83dad1b7e0972 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a25ba expref: -99 pid: 82136 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787826888.551855:0:82146:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1200/0xb3f83dad1b7e0972 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a25ba expref: -99 pid: 82136 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00000080:00200000:0.0:1787826888.551874:0:82146:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:0.0:1787826888.551881:0:82146:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d37.replay-single, flags=24 00000080:00200000:0.0:1787826888.551889:0:82146:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200017b01:0x1:0x0](ffff98561166c2c8), inode mode 41ed mask 1 00000080:00200000:0.0:1787826888.551899:0:82146:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f37.replay-single, dir=[0x200017b01:0x1:0x0](ffff98561166c2c8), flags=0 00000080:00200000:0.0:1787826888.551904:0:82146:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f37.replay-single, dir=[0x200017b01:0x1:0x0](ffff98561166c2c8), intent=getattr 00000002:00010000:0.0:1787826888.551924:0:82146:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f37.replay-single,[0x0:0x0:0x0]) in obj [0x200017b01:0x1:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787826888.551982:0:82146:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0800/0xb3f83dad1b7e098e lrc: 3/1,0 mode: --/CR res: [0x200017b01:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 82146 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826888.551990:0:82146:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0800/0xb3f83dad1b7e098e lrc: 3/1,0 mode: --/CR res: [0x200017b01:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 82146 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826888.552003:0:82146:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0800/0xb3f83dad1b7e098e lrc: 3/1,0 mode: --/CR res: [0x200017b01:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 82146 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826888.552033:0:82146:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e7a940 pname:cluuid:pid:xid:nid:opc:job rmdir:7b4e8535-9db7-485d-abf6-bda966b63320:82146:1874669225589632:192.168.203.114@tcp:101:rmdir.0 00000100:00100000:0.0:1787826888.552152:0:82146:0:(client.c:2715:ptlrpc_set_wait()) set ffff985605437b40 going to sleep for 11 seconds 00000100:00100000:0.0:1787826888.553993:0:82146:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e7a940 pname:cluuid:pid:xid:nid:opc:job rmdir:7b4e8535-9db7-485d-abf6-bda966b63320:82146:1874669225589632:192.168.203.114@tcp:101:rmdir.0 00010000:00010000:0.0:1787826888.554014:0:82146:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0800/0xb3f83dad1b7e098e lrc: 4/1,0 mode: --/CR res: [0x200017b01:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a2638 expref: -99 pid: 82146 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826888.554030:0:82146:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0800/0xb3f83dad1b7e098e lrc: 4/1,0 mode: --/PR res: [0x200017b01:0x2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a2638 expref: -99 pid: 82146 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826888.554040:0:82146:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0800/0xb3f83dad1b7e098e lrc: 4/1,0 mode: --/PR res: [0x200017b01:0x2:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a2638 expref: -99 pid: 82146 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826888.554049:0:82146:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0800/0xb3f83dad1b7e098e lrc: 4/1,0 mode: PR/PR res: [0x200017b01:0x2:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a2638 expref: -99 pid: 82146 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826888.554057:0:82146:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0800/0xb3f83dad1b7e098e lrc: 4/1,0 mode: PR/PR res: [0x200017b01:0x2:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a2638 expref: -99 pid: 82146 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826888.554066:0:82146:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0800/0xb3f83dad1b7e098e lrc: 5/2,0 mode: PR/PR res: [0x200017b01:0x2:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a2638 expref: -99 pid: 82146 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826888.554072:0:82146:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0800/0xb3f83dad1b7e098e lrc: 5/2,0 mode: PR/PR res: [0x200017b01:0x2:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a2638 expref: -99 pid: 82146 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826888.554077:0:82146:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0800/0xb3f83dad1b7e098e lrc: 4/1,0 mode: PR/PR res: [0x200017b01:0x2:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a2638 expref: -99 pid: 82146 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787826888.554089:0:82146:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff985610e7a940 x1874669225589632/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/584 e 0 to 0 dl 1787826904 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787826888.554101:0:82146:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0800/0xb3f83dad1b7e098e lrc: 3/1,0 mode: PR/PR res: [0x200017b01:0x2:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a2638 expref: -99 pid: 82146 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826888.554114:0:82146:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8590031617/2 (0 0) 00000080:00200000:0.0:1787826888.554120:0:82146:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200017b01:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:0.0:1787826888.554144:0:82146:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200017b01:0x2:0x0], updating i_size 4096 00000080:00200000:0.0:1787826888.554148:0:82146:0:(namei.c:147:ll_iget()) got inode: [0x200017b01:0x2:0x0](ffff98561166cb08): rc = 0 00000080:00010000:0.0:1787826888.554154:0:82146:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200017b01:0x2:0x0](ffff98561166cb08) for lock 0xb3f83dad1b7e098e 00000080:00010000:0.0:1787826888.554186:0:82146:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200017b01:0x2:0x0](ffff98561166cb08) 00000080:00010000:0.0:1787826888.554194:0:82146:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e098e from it ffffab3ac1bafe08 00010000:00010000:0.0:1787826888.554199:0:82146:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0800/0xb3f83dad1b7e098e lrc: 3/1,0 mode: PR/PR res: [0x200017b01:0x2:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a2638 expref: -99 pid: 82146 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826888.554208:0:82146:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0800/0xb3f83dad1b7e098e lrc: 2/0,0 mode: PR/PR res: [0x200017b01:0x2:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a2638 expref: -99 pid: 82146 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826888.554247:0:82146:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200017b01:0x1:0x0](ffff98561166c2c8), inode mode 41ed mask 3 00000080:00200000:0.0:1787826888.554257:0:82146:0:(namei.c:2328:ll_rmdir()) VFS Op:name=f37.replay-single, dir=[0x200017b01:0x1:0x0](ffff98561166c2c8) 00010000:00010000:0.0:1787826888.554285:0:82146:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0800/0xb3f83dad1b7e098e lrc: 2/0,0 mode: PR/PR res: [0x200017b01:0x2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x5aa6d504d91a2638 expref: -99 pid: 82146 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826888.554345:0:82146:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0800/0xb3f83dad1b7e098e lrc: 1/0,0 mode: --/PR res: [0x200017b01:0x2:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a2638 expref: -99 pid: 82146 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826888.554353:0:82146:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826888.554356:0:82146:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0800/0xb3f83dad1b7e098e lrc: 0/0,0 mode: --/PR res: [0x200017b01:0x2:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a2638 expref: -99 pid: 82146 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000020:00100000:0.0:1787826888.554389:0:82146:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:0.0:1787826888.554396:0:82146:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610e7a940 x1874669225589888/t0(0) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826888.554419:0:82146:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e7a940 pname:cluuid:pid:xid:nid:opc:job rmdir:7b4e8535-9db7-485d-abf6-bda966b63320:82146:1874669225589888:192.168.203.114@tcp:36:rmdir.0 00000100:00100000:0.0:1787826888.554507:0:82146:0:(client.c:2715:ptlrpc_set_wait()) set ffff985605437b40 going to sleep for 11 seconds 00000100:00100000:0.0:1787826888.556216:0:82146:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e7a940 pname:cluuid:pid:xid:nid:opc:job rmdir:7b4e8535-9db7-485d-abf6-bda966b63320:82146:1874669225589888:192.168.203.114@tcp:36:rmdir.0 00000080:00200008:2.0:1787826888.887007:0:82156:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:2.0:1787826888.887019:0:82156:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:2.0:1787826888.887038:0:82156:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826888.887053:0:82156:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1200/0xb3f83dad1b7e0972 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a25ba expref: -99 pid: 82136 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00010000:00010000:2.0:1787826888.887064:0:82156:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1200/0xb3f83dad1b7e0972 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a25ba expref: -99 pid: 82136 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00000080:00200000:2.0:1787826888.887069:0:82156:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787826888.887075:0:82156:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:2.0:1787826888.887077:0:82156:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e0972 00000080:00010000:2.0:1787826888.887083:0:82156:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0972 from it ffffab3ac1e37cb8 00010000:00010000:2.0:1787826888.887086:0:82156:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1200/0xb3f83dad1b7e0972 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a25ba expref: -99 pid: 82136 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00010000:00010000:2.0:1787826888.887092:0:82156:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1200/0xb3f83dad1b7e0972 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a25ba expref: -99 pid: 82136 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00000080:00200008:2.0:1787826888.887504:0:82156:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 0, flags 0, foreign 0 00000080:00200000:2.0:1787826888.887509:0:82156:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:2.0:1787826888.887517:0:82156:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826888.887524:0:82156:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1200/0xb3f83dad1b7e0972 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a25ba expref: -99 pid: 82136 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00010000:00010000:2.0:1787826888.887533:0:82156:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1200/0xb3f83dad1b7e0972 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a25ba expref: -99 pid: 82136 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00000080:00200000:2.0:1787826888.887540:0:82156:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787826888.887544:0:82156:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:2.0:1787826888.887547:0:82156:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e0972 00000080:00010000:2.0:1787826888.887551:0:82156:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0972 from it ffffab3ac1e37cb8 00010000:00010000:2.0:1787826888.887554:0:82156:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1200/0xb3f83dad1b7e0972 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a25ba expref: -99 pid: 82136 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00010000:00010000:2.0:1787826888.887562:0:82156:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1200/0xb3f83dad1b7e0972 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a25ba expref: -99 pid: 82136 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 0 00000080:00200000:2.0:1787826888.887641:0:82156:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:2.0:1787826888.887647:0:82156:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826888.887654:0:82156:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1200/0xb3f83dad1b7e0972 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a25ba expref: -99 pid: 82136 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 0 00010000:00010000:2.0:1787826888.887663:0:82156:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1200/0xb3f83dad1b7e0972 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a25ba expref: -99 pid: 82136 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 0 00000080:00200000:2.0:1787826888.887670:0:82156:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787826888.887674:0:82156:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:2.0:1787826888.887677:0:82156:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e0972 00000080:00010000:2.0:1787826888.887681:0:82156:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0972 from it ffffab3ac1e37bb0 00010000:00010000:2.0:1787826888.887684:0:82156:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1200/0xb3f83dad1b7e0972 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a25ba expref: -99 pid: 82136 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 0 00010000:00010000:2.0:1787826888.887720:0:82156:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1200/0xb3f83dad1b7e0972 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a25ba expref: -99 pid: 82136 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 0 00000080:00200000:2.0:1787826888.887728:0:82156:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 44 00000080:00200008:2.0:1787826888.887733:0:82156:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 100000 00000080:00200080:2.0:1787826888.887795:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00800000:00000080:2.0:1787826888.887807:0:82156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4e400 uarg=0000000000000000 00800000:00000004:2.0:1787826888.887813:0:82156:0:(obd_class.h:1091:obd_statfs()) lustre-MDT0000-mdc-ffff985602bc9800: age 3029, max_age 3038 00000100:00100000:2.0:1787826888.887847:0:82156:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee2d00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:82156:1874669225590016:192.168.203.114@tcp:41:lfs.0 00000100:00100000:2.0:1787826888.887910:0:82156:0:(client.c:2715:ptlrpc_set_wait()) set ffff985610f11d80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826888.889075:0:82156:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 206158430212 gen 8 00000100:00100000:1.0:1787826888.889099:0:82156:0:(client.c:3174:ptlrpc_free_committed()) @@@ keeping (FL_REPLAY) req@ffff985610e7a580 x1874669225589504/t206158430211(206158430211) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/608 e 0 to 0 dl 1787826904 ref 1 fl Complete:RPQU/604/ffffffff rc 301/-1 job:'multiop.0' uid:0 gid:0 projid:0 00000100:00100000:1.0:1787826888.889115:0:82156:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee2d00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:82156:1874669225590016:192.168.203.114@tcp:41:lfs.0 00800000:00000004:1.0:1787826888.889130:0:82156:0:(obd_class.h:1117:obd_statfs()) lustre-MDT0000-mdc-ffff985602bc9800: update ffff985606a337f0 cache blocks 317362/353529 objects 1023689/1024000 00000080:00200080:1.0:1787826888.889180:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00800000:00000080:1.0:1787826888.889192:0:82156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00800000:00000004:1.0:1787826888.889198:0:82156:0:(obd_class.h:1091:obd_statfs()) lustre-MDT0001-mdc-ffff985602bc9800: age 3029, max_age 3038 00000100:00100000:1.0:1787826888.889220:0:82156:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610db7c00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:82156:1874669225590144:192.168.203.114@tcp:41:lfs.0 00000100:00100000:1.0:1787826888.889277:0:82156:0:(client.c:2715:ptlrpc_set_wait()) set ffff9856108f2840 going to sleep for 11 seconds 00000100:00100000:1.0:1787826888.890021:0:82156:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610db7c00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:82156:1874669225590144:192.168.203.114@tcp:41:lfs.0 00800000:00000004:1.0:1787826888.890033:0:82156:0:(obd_class.h:1117:obd_statfs()) lustre-MDT0001-mdc-ffff985602bc9800: update ffff98560952b7f0 cache blocks 317409/353529 objects 1023738/1024000 00000080:00200080:1.0:1787826888.890065:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00800000:00000080:1.0:1787826888.890075:0:82156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.890083:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00800000:00000080:1.0:1787826888.890088:0:82156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.890094:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00800000:00000080:1.0:1787826888.890099:0:82156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.890104:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00800000:00000080:1.0:1787826888.890108:0:82156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.890113:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00800000:00000080:1.0:1787826888.890116:0:82156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.890120:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00800000:00000080:1.0:1787826888.890124:0:82156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.890128:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00800000:00000080:1.0:1787826888.890132:0:82156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.890136:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00800000:00000080:1.0:1787826888.890140:0:82156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.890144:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00800000:00000080:1.0:1787826888.890149:0:82156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.890153:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00800000:00000080:1.0:1787826888.890156:0:82156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.890160:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00800000:00000080:1.0:1787826888.890163:0:82156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.890167:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00800000:00000080:1.0:1787826888.890171:0:82156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.890174:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00800000:00000080:1.0:1787826888.890178:0:82156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.890181:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00800000:00000080:1.0:1787826888.890184:0:82156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.890188:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00800000:00000080:1.0:1787826888.890191:0:82156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.890195:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00800000:00000080:1.0:1787826888.890198:0:82156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.890202:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00800000:00000080:1.0:1787826888.890205:0:82156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.890208:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00800000:00000080:1.0:1787826888.890211:0:82156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.890214:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00800000:00000080:1.0:1787826888.890217:0:82156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.890221:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00800000:00000080:1.0:1787826888.890224:0:82156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.890228:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00800000:00000080:1.0:1787826888.890231:0:82156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.890235:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00800000:00000080:1.0:1787826888.890238:0:82156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.890242:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00800000:00000080:1.0:1787826888.890245:0:82156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.890249:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00800000:00000080:1.0:1787826888.890253:0:82156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.890257:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00800000:00000080:1.0:1787826888.890260:0:82156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.890263:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00800000:00000080:1.0:1787826888.890266:0:82156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.890271:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00800000:00000080:1.0:1787826888.890274:0:82156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.890278:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00800000:00000080:1.0:1787826888.890281:0:82156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.890285:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00800000:00000080:1.0:1787826888.890288:0:82156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.890292:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00800000:00000080:1.0:1787826888.890295:0:82156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.890300:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00800000:00000080:1.0:1787826888.890304:0:82156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.890307:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00800000:00000080:1.0:1787826888.890311:0:82156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.890315:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00800000:00000080:1.0:1787826888.890318:0:82156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.890322:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00800000:00000080:1.0:1787826888.890326:0:82156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.890330:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00800000:00000080:1.0:1787826888.890333:0:82156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.890337:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00800000:00000080:1.0:1787826888.890340:0:82156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.890344:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00800000:00000080:1.0:1787826888.890347:0:82156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.890351:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00800000:00000080:1.0:1787826888.890354:0:82156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.890357:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00800000:00000080:1.0:1787826888.890361:0:82156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.890364:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00800000:00000080:1.0:1787826888.890367:0:82156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.890371:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00800000:00000080:1.0:1787826888.890374:0:82156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.890378:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00800000:00000080:1.0:1787826888.890382:0:82156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.890385:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00800000:00000080:1.0:1787826888.890389:0:82156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.890392:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00800000:00000080:1.0:1787826888.890396:0:82156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.890399:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00800000:00000080:1.0:1787826888.890402:0:82156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.890406:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00800000:00000080:1.0:1787826888.890409:0:82156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.890414:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00800000:00000080:1.0:1787826888.890417:0:82156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.890421:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00800000:00000080:1.0:1787826888.890424:0:82156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.890427:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00800000:00000080:1.0:1787826888.890430:0:82156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.890434:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00800000:00000080:1.0:1787826888.890438:0:82156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.890442:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00800000:00000080:1.0:1787826888.890445:0:82156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.890449:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00800000:00000080:1.0:1787826888.890452:0:82156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.890471:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00800000:00000080:1.0:1787826888.890476:0:82156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.890480:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00800000:00000080:1.0:1787826888.890483:0:82156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.890487:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00800000:00000080:1.0:1787826888.890490:0:82156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.890494:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00800000:00000080:1.0:1787826888.890497:0:82156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.890501:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00800000:00000080:1.0:1787826888.890504:0:82156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.890509:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00800000:00000080:1.0:1787826888.890512:0:82156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.890516:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00800000:00000080:1.0:1787826888.890520:0:82156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.890524:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00800000:00000080:1.0:1787826888.890528:0:82156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.890531:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00800000:00000080:1.0:1787826888.890536:0:82156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.890539:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00800000:00000080:1.0:1787826888.890542:0:82156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.890546:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00800000:00000080:1.0:1787826888.890550:0:82156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.890554:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00020000:00000080:1.0:1787826888.890560:0:82156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00020000:00000004:1.0:1787826888.890568:0:82156:0:(obd_class.h:1091:obd_statfs()) lustre-OST0000-osc-ffff985602bc9800: age 3029, max_age 3038 00000100:00100000:1.0:1787826888.890589:0:82156:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610db7c00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:82156:1874669225590272:192.168.203.114@tcp:13:lfs.0 00000100:00100000:1.0:1787826888.890636:0:82156:0:(client.c:2715:ptlrpc_set_wait()) set ffff9856108f2840 going to sleep for 11 seconds 00000100:00100000:1.0:1787826888.891812:0:82156:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610db7c00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:82156:1874669225590272:192.168.203.114@tcp:13:lfs.0 00020000:00000004:1.0:1787826888.891826:0:82156:0:(obd_class.h:1117:obd_statfs()) lustre-OST0000-osc-ffff985602bc9800: update ffff985609528450 cache blocks 901357/958279 objects 261728/262144 00000080:00200080:1.0:1787826888.891861:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00020000:00000080:1.0:1787826888.891870:0:82156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00020000:00000004:1.0:1787826888.891875:0:82156:0:(obd_class.h:1091:obd_statfs()) lustre-OST0001-osc-ffff985602bc9800: age 3029, max_age 3038 00000100:00100000:1.0:1787826888.891890:0:82156:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610db7c00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:82156:1874669225590400:192.168.203.114@tcp:13:lfs.0 00000100:00100000:1.0:1787826888.891928:0:82156:0:(client.c:2715:ptlrpc_set_wait()) set ffff9856108f2840 going to sleep for 11 seconds 00000100:00100000:1.0:1787826888.892719:0:82156:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610db7c00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:82156:1874669225590400:192.168.203.114@tcp:13:lfs.0 00020000:00000004:1.0:1787826888.892730:0:82156:0:(obd_class.h:1117:obd_statfs()) lustre-OST0001-osc-ffff985602bc9800: update ffff98560952d1c0 cache blocks 901363/958279 objects 261735/262144 00000080:00200080:1.0:1787826888.892770:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00020000:00000080:1.0:1787826888.892780:0:82156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.892788:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00020000:00000080:1.0:1787826888.892793:0:82156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.892798:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00020000:00000080:1.0:1787826888.892801:0:82156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.892803:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00020000:00000080:1.0:1787826888.892806:0:82156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.892808:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00020000:00000080:1.0:1787826888.892809:0:82156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.892811:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00020000:00000080:1.0:1787826888.892813:0:82156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.892815:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00020000:00000080:1.0:1787826888.892817:0:82156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.892818:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00020000:00000080:1.0:1787826888.892820:0:82156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.892822:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00020000:00000080:1.0:1787826888.892824:0:82156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.892826:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00020000:00000080:1.0:1787826888.892827:0:82156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.892829:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00020000:00000080:1.0:1787826888.892832:0:82156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.892835:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00020000:00000080:1.0:1787826888.892839:0:82156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.892842:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00020000:00000080:1.0:1787826888.892845:0:82156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.892850:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00020000:00000080:1.0:1787826888.892853:0:82156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.892857:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00020000:00000080:1.0:1787826888.892861:0:82156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.892865:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00020000:00000080:1.0:1787826888.892868:0:82156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.892873:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00020000:00000080:1.0:1787826888.892876:0:82156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.892880:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00020000:00000080:1.0:1787826888.892883:0:82156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.892887:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00020000:00000080:1.0:1787826888.892890:0:82156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.892894:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00020000:00000080:1.0:1787826888.892897:0:82156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.892901:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00020000:00000080:1.0:1787826888.892904:0:82156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.892908:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00020000:00000080:1.0:1787826888.892911:0:82156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.892915:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00020000:00000080:1.0:1787826888.892918:0:82156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.892921:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00020000:00000080:1.0:1787826888.892925:0:82156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.892928:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00020000:00000080:1.0:1787826888.892931:0:82156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.892934:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00020000:00000080:1.0:1787826888.892938:0:82156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.892942:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00020000:00000080:1.0:1787826888.892945:0:82156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.892948:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00020000:00000080:1.0:1787826888.892951:0:82156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.892954:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00020000:00000080:1.0:1787826888.892958:0:82156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.892962:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00020000:00000080:1.0:1787826888.892965:0:82156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.892968:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00020000:00000080:1.0:1787826888.892971:0:82156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.892975:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00020000:00000080:1.0:1787826888.892978:0:82156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.892982:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00020000:00000080:1.0:1787826888.892985:0:82156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.892989:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00020000:00000080:1.0:1787826888.892992:0:82156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.892996:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00020000:00000080:1.0:1787826888.893000:0:82156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.893003:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00020000:00000080:1.0:1787826888.893007:0:82156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.893011:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00020000:00000080:1.0:1787826888.893015:0:82156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.893018:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00020000:00000080:1.0:1787826888.893021:0:82156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.893023:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00020000:00000080:1.0:1787826888.893025:0:82156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.893028:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00020000:00000080:1.0:1787826888.893029:0:82156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.893031:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00020000:00000080:1.0:1787826888.893033:0:82156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.893035:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00020000:00000080:1.0:1787826888.893037:0:82156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.893039:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00020000:00000080:1.0:1787826888.893040:0:82156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.893042:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00020000:00000080:1.0:1787826888.893044:0:82156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.893046:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00020000:00000080:1.0:1787826888.893048:0:82156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.893050:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00020000:00000080:1.0:1787826888.893051:0:82156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.893053:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00020000:00000080:1.0:1787826888.893055:0:82156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.893057:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00020000:00000080:1.0:1787826888.893058:0:82156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.893060:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00020000:00000080:1.0:1787826888.893062:0:82156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.893064:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00020000:00000080:1.0:1787826888.893066:0:82156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.893068:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00020000:00000080:1.0:1787826888.893070:0:82156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.893072:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00020000:00000080:1.0:1787826888.893074:0:82156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.893078:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00020000:00000080:1.0:1787826888.893081:0:82156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.893084:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00020000:00000080:1.0:1787826888.893088:0:82156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.893092:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00020000:00000080:1.0:1787826888.893095:0:82156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.893099:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00020000:00000080:1.0:1787826888.893102:0:82156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.893106:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00020000:00000080:1.0:1787826888.893109:0:82156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.893112:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00020000:00000080:1.0:1787826888.893115:0:82156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.893119:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00020000:00000080:1.0:1787826888.893122:0:82156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.893137:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00020000:00000080:1.0:1787826888.893142:0:82156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.893147:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00020000:00000080:1.0:1787826888.893151:0:82156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.893155:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00020000:00000080:1.0:1787826888.893159:0:82156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200080:1.0:1787826888.893161:0:82156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc2520bab0 00020000:00000080:1.0:1787826888.893163:0:82156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985605574800 uarg=0000000000000000 00000080:00200008:1.0:1787826888.893177:0:82156:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 100000 00000080:00200000:2.0:1787826888.910324:0:82163:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:2.0:1787826888.910363:0:82163:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826888.910386:0:82163:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1200/0xb3f83dad1b7e0972 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a25ba expref: -99 pid: 82136 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 0 00010000:00010000:2.0:1787826888.910404:0:82163:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1200/0xb3f83dad1b7e0972 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a25ba expref: -99 pid: 82136 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 0 00000080:00200000:2.0:1787826888.910415:0:82163:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787826888.910423:0:82163:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:2.0:1787826888.910427:0:82163:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e0972 00000080:00010000:2.0:1787826888.910436:0:82163:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0972 from it ffffab3ac1cb7bd0 00010000:00010000:2.0:1787826888.910441:0:82163:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1200/0xb3f83dad1b7e0972 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a25ba expref: -99 pid: 82136 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 0 00010000:00010000:2.0:1787826888.910452:0:82163:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1200/0xb3f83dad1b7e0972 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a25ba expref: -99 pid: 82136 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 1 00000080:00200000:2.0:1787826888.910470:0:82163:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:2.0:1787826888.910487:0:82163:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=fsa-oleg314-client.virtnet, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), flags=1536 00000080:00200000:2.0:1787826888.910491:0:82163:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:2.0:1787826888.910497:0:82163:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826888.910504:0:82163:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1200/0xb3f83dad1b7e0972 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a25ba expref: -99 pid: 82136 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 1 00010000:00010000:2.0:1787826888.910513:0:82163:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1200/0xb3f83dad1b7e0972 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a25ba expref: -99 pid: 82136 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 1 00000080:00200000:2.0:1787826888.910520:0:82163:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787826888.910524:0:82163:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:2.0:1787826888.910527:0:82163:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e0972 00000080:00010000:2.0:1787826888.910530:0:82163:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0972 from it ffffab3ac1cb7cd0 00010000:00010000:2.0:1787826888.910534:0:82163:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1200/0xb3f83dad1b7e0972 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a25ba expref: -99 pid: 82136 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 1 00010000:00010000:2.0:1787826888.910547:0:82163:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1200/0xb3f83dad1b7e0972 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a25ba expref: -99 pid: 82136 timeout: 0 lvb_type: 0 lru_score: 9 lru_type: 1 00000080:00200000:2.0:1787826888.910555:0:82163:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 3 00000080:00200000:2.0:1787826888.910561:0:82163:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:2.0:1787826888.910568:0:82163:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826888.910573:0:82163:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1200/0xb3f83dad1b7e0972 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a25ba expref: -99 pid: 82136 timeout: 0 lvb_type: 0 lru_score: 9 lru_type: 1 00010000:00010000:2.0:1787826888.910581:0:82163:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1200/0xb3f83dad1b7e0972 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a25ba expref: -99 pid: 82136 timeout: 0 lvb_type: 0 lru_score: 9 lru_type: 1 00000080:00200000:2.0:1787826888.910587:0:82163:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787826888.910591:0:82163:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:2.0:1787826888.910594:0:82163:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e0972 00000080:00010000:2.0:1787826888.910597:0:82163:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0972 from it ffffab3ac1cb7db0 00010000:00010000:2.0:1787826888.910600:0:82163:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1200/0xb3f83dad1b7e0972 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a25ba expref: -99 pid: 82136 timeout: 0 lvb_type: 0 lru_score: 9 lru_type: 1 00010000:00010000:2.0:1787826888.910609:0:82163:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1200/0xb3f83dad1b7e0972 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a25ba expref: -99 pid: 82136 timeout: 0 lvb_type: 0 lru_score: 10 lru_type: 1 00000080:00200000:2.0:1787826888.910617:0:82163:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 3 00000080:00200000:2.0:1787826888.910622:0:82163:0:(namei.c:2129:ll_create_nd()) VFS Op:name=fsa-oleg314-client.virtnet, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), flags=33188, excl=1 00000080:00200000:2.0:1787826888.910627:0:82163:0:(namei.c:2084:ll_mknod()) VFS Op:name=fsa-oleg314-client.virtnet, dir=[0x200000007:0x1:0x0](ffff98561158e3c8) mode 100644 dev 0 00010000:00010000:2.0:1787826888.910657:0:82163:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1200/0xb3f83dad1b7e0972 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x5aa6d504d91a25ba expref: -99 pid: 82136 timeout: 0 lvb_type: 0 lru_score: 10 lru_type: 1 00010000:00010000:2.0:1787826888.910774:0:82163:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1200/0xb3f83dad1b7e0972 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a25ba expref: -99 pid: 82136 timeout: 0 lvb_type: 0 lru_score: 10 lru_type: 1 00010000:00010000:2.0:1787826888.910782:0:82163:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826888.910785:0:82163:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1200/0xb3f83dad1b7e0972 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a25ba expref: -99 pid: 82136 timeout: 0 lvb_type: 0 lru_score: 10 lru_type: 1 00000020:00100000:2.0:1787826888.910812:0:82163:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787826888.910830:0:82163:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985602ee1680 x1874669225590656/t0(0) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 528/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826888.910863:0:82163:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee1680 pname:cluuid:pid:xid:nid:opc:job mcreate:7b4e8535-9db7-485d-abf6-bda966b63320:82163:1874669225590656:192.168.203.114@tcp:36:mcreate.0 00000100:00100000:2.0:1787826888.910987:0:82163:0:(client.c:2715:ptlrpc_set_wait()) set ffff985610f11cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826888.913653:0:82163:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee1680 pname:cluuid:pid:xid:nid:opc:job mcreate:7b4e8535-9db7-485d-abf6-bda966b63320:82163:1874669225590656:192.168.203.114@tcp:36:mcreate.0 00000080:00200000:0.0:1787826888.913736:0:82163:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200017b01:0x3:0x0](ffff98561166f448) 00000080:00200000:0.0:1787826888.913790:0:82163:0:(namei.c:2137:ll_create_nd()) VFS Op:name=fsa-oleg314-client.virtnet, unhashed 1 00000080:00200000:2.0:1787826888.920207:0:82166:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:2.0:1787826888.920235:0:82166:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826888.920291:0:82166:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4000/0xb3f83dad1b7e0995 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 82166 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826888.920298:0:82166:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4000/0xb3f83dad1b7e0995 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 82166 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826888.920306:0:82166:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4000/0xb3f83dad1b7e0995 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 82166 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826888.920330:0:82166:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee2940 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:82166:1874669225590784:192.168.203.114@tcp:101:rm.0 00000100:00100000:2.0:1787826888.920405:0:82166:0:(client.c:2715:ptlrpc_set_wait()) set ffff985610f11900 going to sleep for 11 seconds 00000100:00100000:0.0:1787826888.921928:0:82166:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee2940 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:82166:1874669225590784:192.168.203.114@tcp:101:rm.0 00010000:00010000:0.0:1787826888.921948:0:82166:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4000/0xb3f83dad1b7e0995 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a2669 expref: -99 pid: 82166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826888.921956:0:82166:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4000/0xb3f83dad1b7e0995 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a2669 expref: -99 pid: 82166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826888.921965:0:82166:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4000/0xb3f83dad1b7e0995 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a2669 expref: -99 pid: 82166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826888.921972:0:82166:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4000/0xb3f83dad1b7e0995 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a2669 expref: -99 pid: 82166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826888.921979:0:82166:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4000/0xb3f83dad1b7e0995 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a2669 expref: -99 pid: 82166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826888.921985:0:82166:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4000/0xb3f83dad1b7e0995 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a2669 expref: -99 pid: 82166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826888.921989:0:82166:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4000/0xb3f83dad1b7e0995 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a2669 expref: -99 pid: 82166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787826888.921997:0:82166:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff985602ee2940 x1874669225590784/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826904 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787826888.922010:0:82166:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4000/0xb3f83dad1b7e0995 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a2669 expref: -99 pid: 82166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826888.922026:0:82166:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:0.0:1787826888.922035:0:82166:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:0.0:1787826888.922058:0:82166:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:0.0:1787826888.922064:0:82166:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e0995 00000080:00010000:0.0:1787826888.922069:0:82166:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:0.0:1787826888.922072:0:82166:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0995 from it ffffab3ac1cb7b58 00010000:00010000:0.0:1787826888.922075:0:82166:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4000/0xb3f83dad1b7e0995 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a2669 expref: -99 pid: 82166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826888.922083:0:82166:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4000/0xb3f83dad1b7e0995 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a2669 expref: -99 pid: 82166 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826888.922102:0:82166:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:0.0:1787826888.922118:0:82166:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=fsa-oleg314-client.virtnet, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), flags=8 00000080:00200000:0.0:1787826888.922121:0:82166:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=fsa-oleg314-client.virtnet, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=getattr 00000002:00010000:0.0:1787826888.922136:0:82166:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: fsa-oleg314-client.virtnet,[0x0:0x0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787826888.922160:0:82166:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0800/0xb3f83dad1b7e099c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 82166 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826888.922167:0:82166:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0800/0xb3f83dad1b7e099c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 82166 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826888.922174:0:82166:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0800/0xb3f83dad1b7e099c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 82166 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826888.922201:0:82166:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e7a1c0 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:82166:1874669225590912:192.168.203.114@tcp:101:rm.0 00000100:00100000:0.0:1787826888.922247:0:82166:0:(client.c:2715:ptlrpc_set_wait()) set ffff985605437b40 going to sleep for 11 seconds 00000100:00100000:0.0:1787826888.923732:0:82166:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e7a1c0 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:82166:1874669225590912:192.168.203.114@tcp:101:rm.0 00010000:00010000:0.0:1787826888.923751:0:82166:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0800/0xb3f83dad1b7e099c lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a2685 expref: -99 pid: 82166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826888.923780:0:82166:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0800/0xb3f83dad1b7e099c lrc: 4/1,0 mode: --/PR res: [0x200017b01:0x3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a2685 expref: -99 pid: 82166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826888.923788:0:82166:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0800/0xb3f83dad1b7e099c lrc: 4/1,0 mode: --/PR res: [0x200017b01:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a2685 expref: -99 pid: 82166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826888.923796:0:82166:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0800/0xb3f83dad1b7e099c lrc: 4/1,0 mode: PR/PR res: [0x200017b01:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a2685 expref: -99 pid: 82166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826888.923806:0:82166:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0800/0xb3f83dad1b7e099c lrc: 4/1,0 mode: PR/PR res: [0x200017b01:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a2685 expref: -99 pid: 82166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826888.923815:0:82166:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0800/0xb3f83dad1b7e099c lrc: 5/2,0 mode: PR/PR res: [0x200017b01:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a2685 expref: -99 pid: 82166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826888.923823:0:82166:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0800/0xb3f83dad1b7e099c lrc: 5/2,0 mode: PR/PR res: [0x200017b01:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a2685 expref: -99 pid: 82166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826888.923830:0:82166:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0800/0xb3f83dad1b7e099c lrc: 4/1,0 mode: PR/PR res: [0x200017b01:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a2685 expref: -99 pid: 82166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787826888.923844:0:82166:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff985610e7a1c0 x1874669225590912/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 600/584 e 0 to 0 dl 1787826904 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787826888.923856:0:82166:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0800/0xb3f83dad1b7e099c lrc: 3/1,0 mode: PR/PR res: [0x200017b01:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a2685 expref: -99 pid: 82166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826888.923866:0:82166:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8590031617/3 (0 0) 00000080:00200000:0.0:1787826888.923871:0:82166:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:0.0:1787826888.923887:0:82166:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200017b01:0x3:0x0], updating i_size 0 00000080:00200000:0.0:1787826888.923890:0:82166:0:(namei.c:147:ll_iget()) got inode: [0x200017b01:0x3:0x0](ffff98561166f448): rc = 0 00000080:00200000:0.0:1787826888.923905:0:82166:0:(file.c:6854:ll_layout_conf()) [0x200017b01:0x3:0x0]: layout version change: 4294967294 -> 4294967295 00000080:00010000:0.0:1787826888.923911:0:82166:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200017b01:0x3:0x0](ffff98561166f448) for lock 0xb3f83dad1b7e099c 00000080:00010000:0.0:1787826888.923920:0:82166:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200017b01:0x3:0x0](ffff98561166f448) 00000080:00010000:0.0:1787826888.923923:0:82166:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e099c from it ffffab3ac1cb7b20 00010000:00010000:0.0:1787826888.923926:0:82166:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0800/0xb3f83dad1b7e099c lrc: 3/1,0 mode: PR/PR res: [0x200017b01:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a2685 expref: -99 pid: 82166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826888.923934:0:82166:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0800/0xb3f83dad1b7e099c lrc: 2/0,0 mode: PR/PR res: [0x200017b01:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a2685 expref: -99 pid: 82166 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:0.0:1787826888.923961:0:82166:0:(file.c:6313:ll_getattr_dentry()) START file fsa-oleg314-client.virtnet:[0x200017b01:0x3:0x0](ffff98561166f448), request_mask 2047, flags 0, foreign 0 00000080:00200000:0.0:1787826888.923967:0:82166:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200017b01:0x3:0x0](ffff98561166f448),name=fsa-oleg314-client.virtnet 00000002:00010000:0.0:1787826888.923983:0:82166:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200017b01:0x3:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826888.923992:0:82166:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0800/0xb3f83dad1b7e099c lrc: 2/1,0 mode: PR/PR res: [0x200017b01:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a2685 expref: -99 pid: 82166 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826888.924001:0:82166:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0800/0xb3f83dad1b7e099c lrc: 2/1,0 mode: PR/PR res: [0x200017b01:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a2685 expref: -99 pid: 82166 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826888.924009:0:82166:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826888.924013:0:82166:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200017b01:0x3:0x0](ffff98561166f448) 00000080:00010000:0.0:1787826888.924016:0:82166:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200017b01:0x3:0x0](ffff98561166f448) for lock 0xb3f83dad1b7e099c 00000080:00010000:0.0:1787826888.924019:0:82166:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e099c from it ffffab3ac1cb7cb8 00010000:00010000:0.0:1787826888.924022:0:82166:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0800/0xb3f83dad1b7e099c lrc: 3/1,0 mode: PR/PR res: [0x200017b01:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91a2685 expref: -99 pid: 82166 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826888.924030:0:82166:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0800/0xb3f83dad1b7e099c lrc: 2/0,0 mode: PR/PR res: [0x200017b01:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91a2685 expref: -99 pid: 82166 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1787826888.924076:0:82166:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:0.0:1787826888.924084:0:82166:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826888.924091:0:82166:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4000/0xb3f83dad1b7e0995 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a2669 expref: -99 pid: 82166 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826888.924100:0:82166:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4000/0xb3f83dad1b7e0995 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a2669 expref: -99 pid: 82166 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826888.924107:0:82166:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826888.924111:0:82166:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:0.0:1787826888.924113:0:82166:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e0995 00000080:00010000:0.0:1787826888.924117:0:82166:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0995 from it ffffab3ac1cb7c10 00010000:00010000:0.0:1787826888.924120:0:82166:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4000/0xb3f83dad1b7e0995 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a2669 expref: -99 pid: 82166 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826888.924127:0:82166:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4000/0xb3f83dad1b7e0995 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a2669 expref: -99 pid: 82166 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1787826888.924136:0:82166:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:0.0:1787826888.924143:0:82166:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=fsa-oleg314-client.virtnet, flags=0 00000080:00200000:0.0:1787826888.924148:0:82166:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:0.0:1787826888.924154:0:82166:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826888.924160:0:82166:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4000/0xb3f83dad1b7e0995 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a2669 expref: -99 pid: 82166 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1787826888.924167:0:82166:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4000/0xb3f83dad1b7e0995 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a2669 expref: -99 pid: 82166 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1787826888.924173:0:82166:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826888.924176:0:82166:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:0.0:1787826888.924178:0:82166:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e0995 00000080:00010000:0.0:1787826888.924181:0:82166:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e0995 from it ffffab3ac1cb7d68 00010000:00010000:0.0:1787826888.924184:0:82166:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4000/0xb3f83dad1b7e0995 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a2669 expref: -99 pid: 82166 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1787826888.924191:0:82166:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4000/0xb3f83dad1b7e0995 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a2669 expref: -99 pid: 82166 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:0.0:1787826888.924198:0:82166:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 3 00000080:00200000:0.0:1787826888.924203:0:82166:0:(namei.c:2418:ll_unlink()) VFS Op:name=fsa-oleg314-client.virtnet, dir=[0x200000007:0x1:0x0](ffff98561158e3c8) 00010000:00010000:0.0:1787826888.924222:0:82166:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4000/0xb3f83dad1b7e0995 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x5aa6d504d91a2669 expref: -99 pid: 82166 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787826888.924245:0:82166:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0800/0xb3f83dad1b7e099c lrc: 2/0,0 mode: PR/PR res: [0x200017b01:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x5aa6d504d91a2685 expref: -99 pid: 82166 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1787826888.924256:0:82166:0:(vvp_object.c:117:vvp_conf_set()) [0x200017b01:0x3:0x0]: losing layout lock 00010000:00010000:0.0:1787826888.924280:0:82166:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0800/0xb3f83dad1b7e099c lrc: 1/0,0 mode: --/PR res: [0x200017b01:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x5aa6d504d91a2685 expref: -99 pid: 82166 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1787826888.924289:0:82166:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4000/0xb3f83dad1b7e0995 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a2669 expref: -99 pid: 82166 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787826888.924300:0:82166:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 2 locks packed 00010000:00010000:0.0:1787826888.924303:0:82166:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb0800/0xb3f83dad1b7e099c lrc: 0/0,0 mode: --/PR res: [0x200017b01:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x5aa6d504d91a2685 expref: -99 pid: 82166 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1787826888.924315:0:82166:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4000/0xb3f83dad1b7e0995 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91a2669 expref: -99 pid: 82166 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000020:00100000:0.0:1787826888.924330:0:82166:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:0.0:1787826888.924336:0:82166:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610e7a1c0 x1874669225591168/t0(0) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826888.924354:0:82166:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e7a1c0 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:82166:1874669225591168:192.168.203.114@tcp:36:rm.0 00000100:00100000:0.0:1787826888.924404:0:82166:0:(client.c:2715:ptlrpc_set_wait()) set ffff985605437b40 going to sleep for 11 seconds 00000100:00100000:0.0:1787826888.926868:0:82166:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e7a1c0 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:82166:1874669225591168:192.168.203.114@tcp:36:rm.0 00000080:00200000:0.0:1787826888.926919:0:82166:0:(vvp_io.c:1885:vvp_io_init()) [0x200017b01:0x3:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:0.0:1787826888.926928:0:82166:0:(vvp_io.c:304:vvp_io_fini()) [0x200017b01:0x3:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787826888.926933:0:82166:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200017b01:0x3:0x0](ffff98561166f448) 00010000:00010000:3.0:1787826889.524924:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826890.548898:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:3.0:1787826891.149661:0:82232:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: local REPLAY BARRIER on lustre-MDT0000 00010000:00010000:0.0:1787826891.572877:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826892.597042:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826893.045846:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e4cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:3c946890-15ca-497a-b838-fe1cba61f4dd:2386:1874669225591296:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787826893.045866:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e4e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669225591424:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787826893.045953:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e4e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669225591552:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787826893.045954:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e4c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669225591680:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787826893.046001:0:2385:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1787826893.046010:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e4fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669225591808:192.168.203.114@tcp:400:kworker.0 00000100:00080000:3.0:1787826893.047949:0:2385:0:(recover.c:205:ptlrpc_request_handle_notconn()) import lustre-MDT0000-mdc-ffff985602bc9800 of lustre-MDT0000_UUID@192.168.203.114@tcp abruptly disconnected: reconnecting 00000100:00080000:3.0:1787826893.047956:0:2385:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from FULL to DISCONN 00000100:02000400:3.0:1787826893.047962:0:2385:0:(import.c:182:ptlrpc_set_import_discon()) lustre-MDT0000-mdc-ffff985602bc9800: Connection to lustre-MDT0000 (at 192.168.203.114@tcp) was lost; in progress operations using this service will wait for recovery to complete 00000100:00080000:3.0:1787826893.047967:0:2385:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826893.048013:0:2385:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3028 00000100:00080000:3.0:1787826893.048026:0:2385:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787826893.048035:0:2385:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787826893.048060:0:2385:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610e4e1c0 x1874669225591936/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826893.048067:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e4e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669225591552:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787826893.048093:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e4e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669225591424:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787826893.048119:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e4c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669225591680:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787826893.048133:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e4e1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669225591936:192.168.203.114@tcp:38:ptlrpcd_01_00.0 00000100:00100000:3.0:1787826893.048147:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e4fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669225591808:192.168.203.114@tcp:400:kworker.0 00000100:00080000:1.0:1787826893.049149:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787826893.049156:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-19) 00000100:00100000:1.0:1787826893.049160:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e4e1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669225591936:192.168.203.114@tcp:38:ptlrpcd_01_00.0 00010000:00010000:1.0:1787826893.620902:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826894.644836:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826895.668855:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826896.692896:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826897.718658:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787826898.164927:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826898.164933:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787826898.164936:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787826898.164946:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3043 00000100:00080000:1.0:1787826898.164958:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 5s 00000100:00080000:1.0:1787826898.164966:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787826898.164978:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787826898.165000:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610db5a40 x1874669225592192/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826898.165063:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610db4f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669225592320:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787826898.165078:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610db5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669225592192:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787826898.165194:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610db4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:3c946890-15ca-497a-b838-fe1cba61f4dd:2384:1874669225592064:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787826898.165275:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610db52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669225592576:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787826898.165425:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610db6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669225592448:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787826898.166673:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610db4f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669225592320:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787826898.166727:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610db52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669225592576:192.168.203.114@tcp:400:kworker.0 00000100:00080000:0.0:1787826898.166952:0:2384:0:(recover.c:205:ptlrpc_request_handle_notconn()) import MGC192.168.203.114@tcp of MGS@192.168.203.114@tcp abruptly disconnected: reconnecting 00000100:00080000:0.0:1787826898.166958:0:2384:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from FULL to DISCONN 00000100:02020000:0.0:1787826898.166963:0:2384:0:(import.c:186:ptlrpc_set_import_discon()) MGC192.168.203.114@tcp: Connection to MGS (at 192.168.203.114@tcp) was lost; in progress operations using this service will fail 00000100:00080000:0.0:1787826898.166967:0:2384:0:(import.c:192:ptlrpc_set_import_discon()) import MGS@192.168.203.114@tcp for MGC192.168.203.114@tcp not replayable, auto-deactivating 00000100:00080000:0.0:1787826898.166969:0:2384:0:(import.c:138:ptlrpc_deactivate_import_nolock()) setting import MGS INVALID 00000100:00100000:0.0:1787826898.166976:0:2384:0:(client.c:3589:ptlrpc_abort_inflight()) @@@ inflight req@ffff985610e4cf00 x1874669225591296/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787826909 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826898.166998:0:2384:0:(client.c:3589:ptlrpc_abort_inflight()) @@@ inflight req@ffff985610db4780 x1874669225592064/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787826914 ref 1 fl Rpc:RNQU/200/0 rc 0/-107 job:'kworker.0' uid:0 gid:0 projid:4294967295 10000000:01000000:0.0:1787826898.167007:0:2384:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808001 00000100:00080000:0.0:1787826898.167011:0:2384:0:(pinger.c:193:ptlrpc_pinger_ir_down()) IR down 10000000:01000000:0.0:1787826898.167013:0:2384:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808002 00000100:00080000:0.0:1787826898.167016:0:2384:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787826898.167021:0:2384:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3028 00000100:00080000:0.0:1787826898.167033:0:2384:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787826898.167046:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e4cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:3c946890-15ca-497a-b838-fe1cba61f4dd:2386:1874669225591296:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787826898.167061:0:2384:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610e7b0c0 x1874669225592704/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826898.167086:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610db4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:3c946890-15ca-497a-b838-fe1cba61f4dd:2384:1874669225592064:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787826898.167207:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e7b0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669225592704:192.168.203.114@tcp:250:ptlrpcd_00_01.0 00000100:00100000:0.0:1787826898.167569:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610db6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669225592448:192.168.203.114@tcp:400:kworker.0 00000100:00080000:1.0:1787826898.168486:0:2382:0:(import.c:1132:ptlrpc_connect_interpret()) MGC192.168.203.114@tcp: connect to target with instance 0 10000000:01000000:1.0:1787826898.168496:0:2382:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808005 00000100:02000400:1.0:1787826898.168501:0:2382:0:(import.c:1284:ptlrpc_connect_interpret()) Evicted from MGS (at 192.168.203.114@tcp) after server handle changed from 0x5aa6d504d91a2520 to 0x5aa6d504d91a29aa 00000100:00080000:1.0:1787826898.168504:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to EVICTED 00000100:00080000:1.0:1787826898.168508:0:2382:0:(import.c:1674:ptlrpc_import_recovery_state_machine()) evicted from MGS@192.168.203.114@tcp; invalidating 00000100:00100000:3.0:1787826898.168662:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e7b0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669225592704:192.168.203.114@tcp:250:ptlrpcd_00_01.0 00000100:00080000:1.0:1787826898.168684:0:82498:0:(import.c:1598:ptlrpc_invalidate_import_thread()) thread invalidate import MGC192.168.203.114@tcp to MGS@192.168.203.114@tcp 00010000:00080000:3.0:1787826898.168686:0:2382:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=46162) req@ffff985610db5a40 x1874669225592192/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/416 e 0 to 0 dl 1787826913 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787826898.168718:0:82498:0:(import.c:323:ptlrpc_invalidate_import()) Sleeping 20s for inflight to error out 10000000:01000000:1.0:1787826898.168721:0:82498:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808003 00010000:00010000:1.0:1787826898.168734:0:82498:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.114@tcp lock: ffff985609eb1e00/0xb3f83dad1b7e094f lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x110e400000000 nid: local remote: 0x5aa6d504d91a253c expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787826898.168753:0:82498:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.114@tcp lock: ffff985609eb1e00/0xb3f83dad1b7e094f lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x110f400000000 nid: local remote: 0x5aa6d504d91a253c expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787826898.168759:0:82498:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00000100:00080000:3.0:1787826898.168759:0:2382:0:(import.c:1132:ptlrpc_connect_interpret()) lustre-MDT0000-mdc-ffff985602bc9800: connect to target with instance 49 00010000:00010000:1.0:1787826898.168770:0:82498:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00000100:00080000:3.0:1787826898.168779:0:2382:0:(import.c:1338:ptlrpc_connect_interpret()) @@@ lustre-MDT0000-mdc-ffff985602bc9800: evicting (reconnect/recover flags not set: 4) req@ffff985610db5a40 x1874669225592192/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/416 e 0 to 0 dl 1787826913 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00010000:1.0:1787826898.168781:0:82498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.114@tcp lock: ffff985609eb1e00/0xb3f83dad1b7e094f lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x590f400000000 nid: local remote: 0x5aa6d504d91a253c expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787826898.168801:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to EVICTED 00010000:00010000:1.0:1787826898.168806:0:82498:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.114@tcp lock: ffff985610e06200/0xb3f83dad1b7e095d lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x110e400000000 nid: local remote: 0x5aa6d504d91a254a expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:02020000:3.0:1787826898.168806:0:2382:0:(import.c:1663:ptlrpc_import_recovery_state_machine()) lustre-MDT0000-mdc-ffff985602bc9800: This client was evicted by lustre-MDT0000; in progress operations using this service will fail. 10000000:00010000:1.0:1787826898.168817:0:82498:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.114@tcp lock: ffff985610e06200/0xb3f83dad1b7e095d lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x110f400000000 nid: local remote: 0x5aa6d504d91a254a expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787826898.168823:0:82498:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:1.0:1787826898.168827:0:82498:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787826898.168830:0:82498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.114@tcp lock: ffff985610e06200/0xb3f83dad1b7e095d lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x590f400000000 nid: local remote: 0x5aa6d504d91a254a expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826898.168845:0:82498:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.114@tcp lock: ffff985610e07600/0xb3f83dad1b7e0948 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x110e400000000 nid: local remote: 0x5aa6d504d91a2535 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787826898.168850:0:82498:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.114@tcp lock: ffff985610e07600/0xb3f83dad1b7e0948 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x110f400000000 nid: local remote: 0x5aa6d504d91a2535 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787826898.168853:0:82498:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x2:0x0].0x0 (lustre) 00010000:00010000:1.0:1787826898.168855:0:82498:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787826898.168856:0:82498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.114@tcp lock: ffff985610e07600/0xb3f83dad1b7e0948 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x590f400000000 nid: local remote: 0x5aa6d504d91a2535 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826898.168863:0:82498:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.114@tcp lock: ffff985610e07200/0xb3f83dad1b7e0956 lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x110e400000000 nid: local remote: 0x5aa6d504d91a2543 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787826898.168866:0:82498:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.114@tcp lock: ffff985610e07200/0xb3f83dad1b7e0956 lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x110f400000000 nid: local remote: 0x5aa6d504d91a2543 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787826898.168869:0:82498:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x736d61726170:0x3:0x0].0x0 (params) 00010000:00010000:1.0:1787826898.168871:0:82498:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787826898.168872:0:82498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.114@tcp lock: ffff985610e07200/0xb3f83dad1b7e0956 lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x590f400000000 nid: local remote: 0x5aa6d504d91a2543 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1787826898.168881:0:82498:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from EVICTED to RECOVER 00000100:00080000:1.0:1787826898.168884:0:82498:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from RECOVER to FULL 10000000:01000000:1.0:1787826898.168885:0:82498:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808004 00000100:00080000:1.0:1787826898.168887:0:82498:0:(pinger.c:186:ptlrpc_pinger_ir_up()) IR up 00000100:02000000:1.0:1787826898.168890:0:82498:0:(import.c:1743:ptlrpc_import_recovery_state_machine()) MGC192.168.203.114@tcp: Connection restored to 192.168.203.114@tcp (at 192.168.203.114@tcp) 00000100:00080000:3.0:1787826898.177808:0:2382:0:(import.c:1674:ptlrpc_import_recovery_state_machine()) evicted from lustre-MDT0000_UUID@192.168.203.114@tcp; invalidating 00000100:00100000:1.0:1787826898.178015:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610db5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669225592192:192.168.203.114@tcp:38:kworker.0 00000100:00080000:3.0:1787826898.178026:0:82499:0:(import.c:1598:ptlrpc_invalidate_import_thread()) thread invalidate import lustre-MDT0000-mdc-ffff985602bc9800 to lustre-MDT0000_UUID@192.168.203.114@tcp 00000100:00080000:3.0:1787826898.178031:0:82499:0:(import.c:138:ptlrpc_deactivate_import_nolock()) setting import lustre-MDT0000_UUID INVALID 00000100:00100000:3.0:1787826898.178035:0:82499:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 206158430212 gen 9 00000100:00100000:3.0:1787826898.178045:0:82499:0:(client.c:3163:ptlrpc_free_committed()) @@@ free request with old gen req@ffff985602ee1680 x1874669225590656/t206158430213(206158430213) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 528/448 e 0 to 0 dl 1787826904 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826898.178061:0:82499:0:(client.c:3163:ptlrpc_free_committed()) @@@ free request with old gen req@ffff985610e7a1c0 x1874669225591168/t206158430214(206158430214) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787826904 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826898.178073:0:82499:0:(client.c:3213:ptlrpc_free_committed()) @@@ free stale open request req@ffff985610e7a580 x1874669225589504/t206158430211(206158430211) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/608 e 0 to 0 dl 1787826904 ref 1 fl Complete:RPQU/604/ffffffff rc 301/-1 job:'multiop.0' uid:0 gid:0 projid:0 00000100:00100000:3.0:1787826898.178092:0:82499:0:(import.c:323:ptlrpc_invalidate_import()) Sleeping 20s for inflight to error out 00010000:00010000:3.0:1787826898.178144:0:82499:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1c00/0xb3f83dad1b7e0979 lrc: 3/0,0 mode: PR/PR res: [0x200017b01:0x1:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x10e400000000 nid: local remote: 0x5aa6d504d91a25d6 expref: -99 pid: 82136 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826898.178167:0:82499:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826898.178177:0:82499:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985609eb1c00/0xb3f83dad1b7e0979 lrc: 0/0,0 mode: --/PR res: [0x200017b01:0x1:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x490f400000000 nid: local remote: 0x5aa6d504d91a25d6 expref: -99 pid: 82136 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000100:00080000:3.0:1787826898.180181:0:82499:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from EVICTED to RECOVER 00000100:00080000:3.0:1787826898.180188:0:82499:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from RECOVER to FULL 00000100:02000000:1.0:1787826898.180300:0:82499:0:(import.c:1743:ptlrpc_import_recovery_state_machine()) lustre-MDT0000-mdc-ffff985602bc9800: Connection restored to 192.168.203.114@tcp (at 192.168.203.114@tcp) 00010000:00010000:1.0:1787826898.740965:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:0.0:1787826899.572783:0:15166:0:(mgc_request.c:611:do_requeue()) updating log lustre-cliir 10000000:01000000:0.0:1787826899.572799:0:15166:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-cliir-ffff985602bc9800 from 196 10000000:01000000:0.0:1787826899.572806:0:15166:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-cliir (res 0x65727473756c) 00010000:00010000:0.0:1787826899.572872:0:15166:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.114@tcp lock: ffff985609eb1a00/0xb3f83dad1b7e09a3 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826899.572883:0:15166:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.114@tcp lock: ffff985609eb1a00/0xb3f83dad1b7e09a3 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826899.572892:0:15166:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.114@tcp lock: ffff985609eb1a00/0xb3f83dad1b7e09a3 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826899.572919:0:15166:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e78b40 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669225592832:192.168.203.114@tcp:101:ll_cfg_requeue.0 00000100:00100000:0.0:1787826899.573048:0:15166:0:(client.c:2715:ptlrpc_set_wait()) set ffff985606a6eb40 going to sleep for 11 seconds 00000100:00100000:0.0:1787826899.575107:0:15166:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e78b40 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669225592832:192.168.203.114@tcp:101:ll_cfg_requeue.0 00010000:00010000:0.0:1787826899.575137:0:15166:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.114@tcp lock: ffff985609eb1a00/0xb3f83dad1b7e09a3 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x5aa6d504d91a2a75 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826899.575150:0:15166:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.114@tcp lock: ffff985609eb1a00/0xb3f83dad1b7e09a3 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x5aa6d504d91a2a75 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826899.575226:0:15166:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e78b40 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669225592960:192.168.203.114@tcp:256:ll_cfg_requeue.0 00000100:00100000:0.0:1787826899.575303:0:15166:0:(client.c:2715:ptlrpc_set_wait()) set ffff985606a6eb40 going to sleep for 11 seconds 00000100:00100000:0.0:1787826899.577748:0:15166:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e78b40 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669225592960:192.168.203.114@tcp:256:ll_cfg_requeue.0 10000000:01000000:0.0:1787826899.577856:0:15166:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.114@tcp: configuration from log 'lustre-cliir' succeeded (0). 00010000:00010000:0.0:1787826899.577866:0:15166:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.114@tcp lock: ffff985609eb1a00/0xb3f83dad1b7e09a3 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x5aa6d504d91a2a75 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826899.577874:0:15166:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.114@tcp lock: ffff985609eb1a00/0xb3f83dad1b7e09a3 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x5aa6d504d91a2a75 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787826899.577885:0:15166:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 10000000:01000000:0.0:1787826899.577887:0:15166:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff985602bc9800 from 52 10000000:01000000:0.0:1787826899.577889:0:15166:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00010000:00010000:0.0:1787826899.577926:0:15166:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.114@tcp lock: ffff985609eb0400/0xb3f83dad1b7e09aa lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826899.577934:0:15166:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.114@tcp lock: ffff985609eb0400/0xb3f83dad1b7e09aa lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826899.577944:0:15166:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.114@tcp lock: ffff985609eb0400/0xb3f83dad1b7e09aa lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826899.577963:0:15166:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e78b40 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669225593088:192.168.203.114@tcp:101:ll_cfg_requeue.0 00000100:00100000:0.0:1787826899.578022:0:15166:0:(client.c:2715:ptlrpc_set_wait()) set ffff985606a6eb40 going to sleep for 11 seconds 00000100:00100000:0.0:1787826899.579373:0:15166:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e78b40 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669225593088:192.168.203.114@tcp:101:ll_cfg_requeue.0 00010000:00010000:0.0:1787826899.579393:0:15166:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.114@tcp lock: ffff985609eb0400/0xb3f83dad1b7e09aa lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x5aa6d504d91a2a7c expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826899.579404:0:15166:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.114@tcp lock: ffff985609eb0400/0xb3f83dad1b7e09aa lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x5aa6d504d91a2a7c expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826899.579461:0:15166:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e78b40 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669225593216:192.168.203.114@tcp:501:ll_cfg_requeue.0 00000100:00100000:0.0:1787826899.579512:0:15166:0:(client.c:2715:ptlrpc_set_wait()) set ffff985606a6eb40 going to sleep for 11 seconds 00000100:00100000:0.0:1787826899.581016:0:15166:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e78b40 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669225593216:192.168.203.114@tcp:501:ll_cfg_requeue.0 00000100:00100000:0.0:1787826899.581063:0:15166:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e78b40 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669225593344:192.168.203.114@tcp:503:ll_cfg_requeue.0 00000100:00100000:0.0:1787826899.581185:0:15166:0:(client.c:2715:ptlrpc_set_wait()) set ffff985606a6eb40 going to sleep for 11 seconds 00000100:00100000:0.0:1787826899.582534:0:15166:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e78b40 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669225593344:192.168.203.114@tcp:503:ll_cfg_requeue.0 00000040:00080000:3.0:1787826899.583367:0:82524:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x3:0x0] index 64768 count 51 00000020:01000000:0.0:1787826899.583474:0:15166:0:(obd_config.c:2145:class_config_parse_llog()) Processed log lustre-client gen 52-51 (rc=0) 10000000:01000000:0.0:1787826899.583492:0:15166:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.114@tcp: configuration from log 'lustre-client' succeeded (0). 00010000:00010000:0.0:1787826899.583503:0:15166:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.114@tcp lock: ffff985609eb0400/0xb3f83dad1b7e09aa lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x5aa6d504d91a2a7c expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826899.583514:0:15166:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.114@tcp lock: ffff985609eb0400/0xb3f83dad1b7e09aa lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x5aa6d504d91a2a7c expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787826899.583524:0:15166:0:(mgc_request.c:611:do_requeue()) updating log params 10000000:01000000:0.0:1787826899.583527:0:15166:0:(mgc_request.c:1802:mgc_process_log()) Process log params-ffff985602bc9800 from 7 10000000:01000000:0.0:1787826899.583530:0:15166:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for params (res 0x736d61726170) 00010000:00010000:0.0:1787826899.583558:0:15166:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.114@tcp lock: ffff985609eb0c00/0xb3f83dad1b7e09b1 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826899.583564:0:15166:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.114@tcp lock: ffff985609eb0c00/0xb3f83dad1b7e09b1 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826899.583571:0:15166:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.114@tcp lock: ffff985609eb0c00/0xb3f83dad1b7e09b1 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826899.583586:0:15166:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e78b40 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669225593472:192.168.203.114@tcp:101:ll_cfg_requeue.0 00000100:00100000:0.0:1787826899.583645:0:15166:0:(client.c:2715:ptlrpc_set_wait()) set ffff985606a6eb40 going to sleep for 11 seconds 00000100:00100000:0.0:1787826899.586976:0:15166:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e78b40 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669225593472:192.168.203.114@tcp:101:ll_cfg_requeue.0 00010000:00010000:0.0:1787826899.587007:0:15166:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.114@tcp lock: ffff985609eb0c00/0xb3f83dad1b7e09b1 lrc: 4/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x5aa6d504d91a2a83 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826899.587025:0:15166:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.114@tcp lock: ffff985609eb0c00/0xb3f83dad1b7e09b1 lrc: 4/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x5aa6d504d91a2a83 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826899.587110:0:15166:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e78b40 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669225593600:192.168.203.114@tcp:501:ll_cfg_requeue.0 00000100:00100000:0.0:1787826899.587186:0:15166:0:(client.c:2715:ptlrpc_set_wait()) set ffff985606a6eb40 going to sleep for 11 seconds 00000100:00100000:0.0:1787826899.588199:0:15166:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e78b40 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669225593600:192.168.203.114@tcp:501:ll_cfg_requeue.0 00000100:00100000:0.0:1787826899.588234:0:15166:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e78b40 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669225593728:192.168.203.114@tcp:503:ll_cfg_requeue.0 00000100:00100000:0.0:1787826899.588299:0:15166:0:(client.c:2715:ptlrpc_set_wait()) set ffff985606a6eb40 going to sleep for 11 seconds 00000100:00100000:0.0:1787826899.589351:0:15166:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e78b40 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669225593728:192.168.203.114@tcp:503:ll_cfg_requeue.0 00000040:00080000:0.0:1787826899.590229:0:82525:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x2:0x0] index 64768 count 7 00000020:01000000:1.0:1787826899.590421:0:15166:0:(obd_config.c:2145:class_config_parse_llog()) Processed log params gen 7-6 (rc=0) 10000000:01000000:1.0:1787826899.590439:0:15166:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.114@tcp: configuration from log 'params' succeeded (0). 00010000:00010000:1.0:1787826899.590583:0:15166:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.114@tcp lock: ffff985609eb0c00/0xb3f83dad1b7e09b1 lrc: 3/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x5aa6d504d91a2a83 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826899.590594:0:15166:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.114@tcp lock: ffff985609eb0c00/0xb3f83dad1b7e09b1 lrc: 2/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x5aa6d504d91a2a83 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787826899.590604:0:15166:0:(mgc_request.c:611:do_requeue()) updating log lustre-sptlrpc 10000000:01000000:1.0:1787826899.590606:0:15166:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-sptlrpc-ffff985606a30000 from 1 10000000:01000000:1.0:1787826899.590609:0:15166:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-sptlrpc (res 0x65727473756c) 00010000:00010000:1.0:1787826899.590641:0:15166:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.114@tcp lock: ffff985642051000/0xb3f83dad1b7e09b8 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826899.590649:0:15166:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.114@tcp lock: ffff985642051000/0xb3f83dad1b7e09b8 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826899.590659:0:15166:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.114@tcp lock: ffff985642051000/0xb3f83dad1b7e09b8 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826899.590681:0:15166:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610db5a40 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669225593856:192.168.203.114@tcp:101:ll_cfg_requeue.0 00000100:00100000:1.0:1787826899.590782:0:15166:0:(client.c:2715:ptlrpc_set_wait()) set ffff9856108f2240 going to sleep for 11 seconds 00000100:00100000:1.0:1787826899.593035:0:15166:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610db5a40 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669225593856:192.168.203.114@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787826899.593063:0:15166:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.114@tcp lock: ffff985642051000/0xb3f83dad1b7e09b8 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0x5aa6d504d91a2a8a expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826899.593076:0:15166:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.114@tcp lock: ffff985642051000/0xb3f83dad1b7e09b8 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1000000000000 nid: local remote: 0x5aa6d504d91a2a8a expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826899.593145:0:15166:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610db5a40 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669225593984:192.168.203.114@tcp:501:ll_cfg_requeue.0 00000100:00100000:1.0:1787826899.593224:0:15166:0:(client.c:2715:ptlrpc_set_wait()) set ffff9856108f2240 going to sleep for 11 seconds 00000100:00100000:1.0:1787826899.594620:0:15166:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610db5a40 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669225593984:192.168.203.114@tcp:501:ll_cfg_requeue.0 10000000:01000000:1.0:1787826899.594655:0:15166:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.114@tcp: configuration from log 'lustre-sptlrpc' failed (-2). 00010000:00010000:1.0:1787826899.594663:0:15166:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.114@tcp lock: ffff985642051000/0xb3f83dad1b7e09b8 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0x5aa6d504d91a2a8a expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826899.594676:0:15166:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.114@tcp lock: ffff985642051000/0xb3f83dad1b7e09b8 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0x5aa6d504d91a2a8a expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826899.764907:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200008:3.0:1787826900.289455:0:82552:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:3.0:1787826900.289469:0:82552:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:3.0:1787826900.289493:0:82552:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826900.289548:0:82552:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e09bf lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 82552 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826900.289555:0:82552:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e09bf lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 82552 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826900.289562:0:82552:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e09bf lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 82552 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826900.289580:0:82552:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e4c780 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:82552:1874669225594112:192.168.203.114@tcp:101:lfs.0 00000100:00100000:3.0:1787826900.289713:0:82552:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607ee2480 going to sleep for 11 seconds 00000100:00100000:0.0:1787826900.301533:0:82552:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e4c780 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:82552:1874669225594112:192.168.203.114@tcp:101:lfs.0 00010000:00010000:0.0:1787826900.301555:0:82552:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e09bf lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a2a98 expref: -99 pid: 82552 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826900.301564:0:82552:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e09bf lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a2a98 expref: -99 pid: 82552 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826900.301574:0:82552:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e09bf lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a2a98 expref: -99 pid: 82552 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826900.301584:0:82552:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e09bf lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a2a98 expref: -99 pid: 82552 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826900.301597:0:82552:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e09bf lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a2a98 expref: -99 pid: 82552 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826900.301606:0:82552:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e09bf lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a2a98 expref: -99 pid: 82552 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826900.301613:0:82552:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e09bf lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a2a98 expref: -99 pid: 82552 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787826900.301625:0:82552:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff985610e4c780 x1874669225594112/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826916 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787826900.301640:0:82552:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e09bf lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a2a98 expref: -99 pid: 82552 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826900.301659:0:82552:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:0.0:1787826900.301666:0:82552:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:0.0:1787826900.301726:0:82552:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:0.0:1787826900.301733:0:82552:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e09bf 00000080:00010000:0.0:1787826900.301740:0:82552:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:0.0:1787826900.301743:0:82552:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e09bf from it ffffab3ac2087cb8 00010000:00010000:0.0:1787826900.301749:0:82552:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e09bf lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a2a98 expref: -99 pid: 82552 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826900.301759:0:82552:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e09bf lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a2a98 expref: -99 pid: 82552 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:0.0:1787826900.302379:0:82552:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 0, flags 0, foreign 0 00000080:00200000:0.0:1787826900.302385:0:82552:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:0.0:1787826900.302402:0:82552:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826900.302412:0:82552:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e09bf lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a2a98 expref: -99 pid: 82552 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826900.302422:0:82552:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e09bf lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a2a98 expref: -99 pid: 82552 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826900.302429:0:82552:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826900.302433:0:82552:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:0.0:1787826900.302436:0:82552:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e09bf 00000080:00010000:0.0:1787826900.302440:0:82552:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e09bf from it ffffab3ac2087cb8 00010000:00010000:0.0:1787826900.302442:0:82552:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e09bf lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a2a98 expref: -99 pid: 82552 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826900.302449:0:82552:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e09bf lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a2a98 expref: -99 pid: 82552 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1787826900.302548:0:82552:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:0.0:1787826900.302556:0:82552:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826900.302564:0:82552:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e09bf lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a2a98 expref: -99 pid: 82552 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1787826900.302572:0:82552:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e09bf lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a2a98 expref: -99 pid: 82552 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1787826900.302579:0:82552:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826900.302583:0:82552:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:0.0:1787826900.302586:0:82552:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e09bf 00000080:00010000:0.0:1787826900.302589:0:82552:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e09bf from it ffffab3ac2087bb0 00010000:00010000:0.0:1787826900.302594:0:82552:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e09bf lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a2a98 expref: -99 pid: 82552 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1787826900.302600:0:82552:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e09bf lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a2a98 expref: -99 pid: 82552 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:0.0:1787826900.302609:0:82552:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 44 00000080:00200008:0.0:1787826900.302616:0:82552:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 100000 00000080:00200080:0.0:1787826900.302752:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00800000:00000080:0.0:1787826900.302766:0:82552:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00800000:00000004:0.0:1787826900.302773:0:82552:0:(obd_class.h:1091:obd_statfs()) lustre-MDT0000-mdc-ffff985602bc9800: age 3039, max_age 3050 00000100:00100000:0.0:1787826900.302807:0:82552:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e7b480 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:82552:1874669225594240:192.168.203.114@tcp:41:lfs.0 00000100:00100000:0.0:1787826900.302879:0:82552:0:(client.c:2715:ptlrpc_set_wait()) set ffff98561198a9c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826900.304532:0:82552:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e7b480 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:82552:1874669225594240:192.168.203.114@tcp:41:lfs.0 00800000:00000004:0.0:1787826900.304555:0:82552:0:(obd_class.h:1117:obd_statfs()) lustre-MDT0000-mdc-ffff985602bc9800: update ffff985606a337f0 cache blocks 317355/353529 objects 1023689/1024000 00000080:00200080:0.0:1787826900.304611:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00800000:00000080:0.0:1787826900.304625:0:82552:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00800000:00000004:0.0:1787826900.304631:0:82552:0:(obd_class.h:1091:obd_statfs()) lustre-MDT0001-mdc-ffff985602bc9800: age 3039, max_age 3050 00000100:00100000:0.0:1787826900.304657:0:82552:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e7b480 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:82552:1874669225594368:192.168.203.114@tcp:41:lfs.0 00000100:00100000:0.0:1787826900.304751:0:82552:0:(client.c:2715:ptlrpc_set_wait()) set ffff98561198a9c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826900.305767:0:82552:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e7b480 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:82552:1874669225594368:192.168.203.114@tcp:41:lfs.0 00800000:00000004:0.0:1787826900.305784:0:82552:0:(obd_class.h:1117:obd_statfs()) lustre-MDT0001-mdc-ffff985602bc9800: update ffff98560952b7f0 cache blocks 317401/353529 objects 1023737/1024000 00000080:00200080:0.0:1787826900.305823:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00800000:00000080:0.0:1787826900.305834:0:82552:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.305841:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00800000:00000080:0.0:1787826900.305847:0:82552:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.305852:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00800000:00000080:0.0:1787826900.305857:0:82552:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.305862:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00800000:00000080:0.0:1787826900.305866:0:82552:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.305870:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00800000:00000080:0.0:1787826900.305874:0:82552:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.305879:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00800000:00000080:0.0:1787826900.305884:0:82552:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.305888:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00800000:00000080:0.0:1787826900.305892:0:82552:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.305897:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00800000:00000080:0.0:1787826900.305901:0:82552:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.305906:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00800000:00000080:0.0:1787826900.305909:0:82552:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.305914:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00800000:00000080:0.0:1787826900.305918:0:82552:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.305929:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00800000:00000080:0.0:1787826900.305934:0:82552:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.305939:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00800000:00000080:0.0:1787826900.305944:0:82552:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.305948:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00800000:00000080:0.0:1787826900.305952:0:82552:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.305957:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00800000:00000080:0.0:1787826900.305961:0:82552:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.305967:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00800000:00000080:0.0:1787826900.305971:0:82552:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.305975:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00800000:00000080:0.0:1787826900.305979:0:82552:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.305983:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00800000:00000080:0.0:1787826900.305988:0:82552:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.305992:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00800000:00000080:0.0:1787826900.305996:0:82552:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.306000:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00800000:00000080:0.0:1787826900.306004:0:82552:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.306008:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00800000:00000080:0.0:1787826900.306013:0:82552:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.306017:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00800000:00000080:0.0:1787826900.306021:0:82552:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.306026:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00800000:00000080:0.0:1787826900.306029:0:82552:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.306034:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00800000:00000080:0.0:1787826900.306038:0:82552:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.306043:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00800000:00000080:0.0:1787826900.306047:0:82552:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.306051:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00800000:00000080:0.0:1787826900.306054:0:82552:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.306059:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00800000:00000080:0.0:1787826900.306063:0:82552:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.306067:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00800000:00000080:0.0:1787826900.306071:0:82552:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.306075:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00800000:00000080:0.0:1787826900.306079:0:82552:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.306083:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00800000:00000080:0.0:1787826900.306086:0:82552:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.306089:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00800000:00000080:0.0:1787826900.306093:0:82552:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.306097:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00800000:00000080:0.0:1787826900.306100:0:82552:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.306104:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00800000:00000080:0.0:1787826900.306107:0:82552:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.306111:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00800000:00000080:0.0:1787826900.306114:0:82552:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.306117:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00800000:00000080:0.0:1787826900.306120:0:82552:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.306124:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00800000:00000080:0.0:1787826900.306128:0:82552:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.306131:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00800000:00000080:0.0:1787826900.306135:0:82552:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.306139:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00800000:00000080:0.0:1787826900.306142:0:82552:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.306146:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00800000:00000080:0.0:1787826900.306149:0:82552:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.306153:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00800000:00000080:0.0:1787826900.306156:0:82552:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.306160:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00800000:00000080:0.0:1787826900.306163:0:82552:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.306167:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00800000:00000080:0.0:1787826900.306171:0:82552:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.306174:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00800000:00000080:0.0:1787826900.306178:0:82552:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.306182:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00800000:00000080:0.0:1787826900.306185:0:82552:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.306189:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00800000:00000080:0.0:1787826900.306192:0:82552:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.306196:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00800000:00000080:0.0:1787826900.306200:0:82552:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.306212:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00800000:00000080:0.0:1787826900.306217:0:82552:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.306221:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00800000:00000080:0.0:1787826900.306225:0:82552:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.306229:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00800000:00000080:0.0:1787826900.306232:0:82552:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.306236:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00800000:00000080:0.0:1787826900.306239:0:82552:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.306243:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00800000:00000080:0.0:1787826900.306246:0:82552:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.306250:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00800000:00000080:0.0:1787826900.306253:0:82552:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.306257:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00800000:00000080:0.0:1787826900.306260:0:82552:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.306264:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00800000:00000080:0.0:1787826900.306267:0:82552:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.306271:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00800000:00000080:0.0:1787826900.306275:0:82552:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.306279:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00800000:00000080:0.0:1787826900.306282:0:82552:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.306286:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00800000:00000080:0.0:1787826900.306289:0:82552:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.306293:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00800000:00000080:0.0:1787826900.306296:0:82552:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.306300:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00800000:00000080:0.0:1787826900.306303:0:82552:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.306307:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00800000:00000080:0.0:1787826900.306310:0:82552:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.306314:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00800000:00000080:0.0:1787826900.306318:0:82552:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.306321:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00800000:00000080:0.0:1787826900.306324:0:82552:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.306328:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00800000:00000080:0.0:1787826900.306331:0:82552:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.306335:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00800000:00000080:0.0:1787826900.306338:0:82552:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.306342:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00020000:00000080:0.0:1787826900.306362:0:82552:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00020000:00000004:0.0:1787826900.306371:0:82552:0:(obd_class.h:1091:obd_statfs()) lustre-OST0000-osc-ffff985602bc9800: age 3039, max_age 3050 00000100:00100000:0.0:1787826900.306396:0:82552:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e7b480 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:82552:1874669225594496:192.168.203.114@tcp:13:lfs.0 00000100:00100000:0.0:1787826900.306452:0:82552:0:(client.c:2715:ptlrpc_set_wait()) set ffff98561198a9c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826900.307934:0:82552:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e7b480 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:82552:1874669225594496:192.168.203.114@tcp:13:lfs.0 00020000:00000004:0.0:1787826900.307961:0:82552:0:(obd_class.h:1117:obd_statfs()) lustre-OST0000-osc-ffff985602bc9800: update ffff985609528450 cache blocks 901357/958279 objects 261728/262144 00000080:00200080:0.0:1787826900.308015:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00020000:00000080:0.0:1787826900.308038:0:82552:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00020000:00000004:0.0:1787826900.308045:0:82552:0:(obd_class.h:1091:obd_statfs()) lustre-OST0001-osc-ffff985602bc9800: age 3039, max_age 3050 00000100:00100000:0.0:1787826900.308068:0:82552:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e7b480 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:82552:1874669225594624:192.168.203.114@tcp:13:lfs.0 00000100:00100000:0.0:1787826900.308161:0:82552:0:(client.c:2715:ptlrpc_set_wait()) set ffff98561198a9c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826900.309386:0:82552:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e7b480 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:82552:1874669225594624:192.168.203.114@tcp:13:lfs.0 00020000:00000004:0.0:1787826900.309412:0:82552:0:(obd_class.h:1117:obd_statfs()) lustre-OST0001-osc-ffff985602bc9800: update ffff98560952d1c0 cache blocks 901363/958279 objects 261735/262144 00000080:00200080:0.0:1787826900.309448:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00020000:00000080:0.0:1787826900.309457:0:82552:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.309461:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00020000:00000080:0.0:1787826900.309464:0:82552:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.309467:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00020000:00000080:0.0:1787826900.309469:0:82552:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.309472:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00020000:00000080:0.0:1787826900.309474:0:82552:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.309477:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00020000:00000080:0.0:1787826900.309479:0:82552:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.309481:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00020000:00000080:0.0:1787826900.309483:0:82552:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.309487:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00020000:00000080:0.0:1787826900.309490:0:82552:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.309493:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00020000:00000080:0.0:1787826900.309495:0:82552:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.309498:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00020000:00000080:0.0:1787826900.309500:0:82552:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.309502:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00020000:00000080:0.0:1787826900.309504:0:82552:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.309506:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00020000:00000080:0.0:1787826900.309508:0:82552:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.309510:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00020000:00000080:0.0:1787826900.309512:0:82552:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.309514:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00020000:00000080:0.0:1787826900.309517:0:82552:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.309520:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00020000:00000080:0.0:1787826900.309522:0:82552:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.309525:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00020000:00000080:0.0:1787826900.309528:0:82552:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.309530:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00020000:00000080:0.0:1787826900.309532:0:82552:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.309534:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00020000:00000080:0.0:1787826900.309536:0:82552:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.309538:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00020000:00000080:0.0:1787826900.309541:0:82552:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.309543:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00020000:00000080:0.0:1787826900.309546:0:82552:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.309548:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00020000:00000080:0.0:1787826900.309551:0:82552:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.309553:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00020000:00000080:0.0:1787826900.309555:0:82552:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.309557:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00020000:00000080:0.0:1787826900.309559:0:82552:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.309561:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00020000:00000080:0.0:1787826900.309564:0:82552:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.309568:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00020000:00000080:0.0:1787826900.309577:0:82552:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.309583:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00020000:00000080:0.0:1787826900.309587:0:82552:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.309592:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00020000:00000080:0.0:1787826900.309595:0:82552:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.309599:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00020000:00000080:0.0:1787826900.309603:0:82552:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.309607:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00020000:00000080:0.0:1787826900.309610:0:82552:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.309614:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00020000:00000080:0.0:1787826900.309618:0:82552:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.309622:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00020000:00000080:0.0:1787826900.309625:0:82552:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.309628:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00020000:00000080:0.0:1787826900.309632:0:82552:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.309635:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00020000:00000080:0.0:1787826900.309638:0:82552:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.309642:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00020000:00000080:0.0:1787826900.309645:0:82552:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.309649:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00020000:00000080:0.0:1787826900.309652:0:82552:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.309655:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00020000:00000080:0.0:1787826900.309658:0:82552:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.309662:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00020000:00000080:0.0:1787826900.309665:0:82552:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.309668:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00020000:00000080:0.0:1787826900.309671:0:82552:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.309675:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00020000:00000080:0.0:1787826900.309716:0:82552:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.309723:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00020000:00000080:0.0:1787826900.309728:0:82552:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.309732:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00020000:00000080:0.0:1787826900.309736:0:82552:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.309741:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00020000:00000080:0.0:1787826900.309744:0:82552:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.309748:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00020000:00000080:0.0:1787826900.309751:0:82552:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.309756:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00020000:00000080:0.0:1787826900.309760:0:82552:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.309764:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00020000:00000080:0.0:1787826900.309769:0:82552:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.309773:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00020000:00000080:0.0:1787826900.309776:0:82552:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.309781:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00020000:00000080:0.0:1787826900.309785:0:82552:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.309790:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00020000:00000080:0.0:1787826900.309793:0:82552:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.309797:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00020000:00000080:0.0:1787826900.309801:0:82552:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.309805:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00020000:00000080:0.0:1787826900.309808:0:82552:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.309813:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00020000:00000080:0.0:1787826900.309817:0:82552:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.309822:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00020000:00000080:0.0:1787826900.309826:0:82552:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.309830:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00020000:00000080:0.0:1787826900.309833:0:82552:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.309836:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00020000:00000080:0.0:1787826900.309840:0:82552:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.309844:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00020000:00000080:0.0:1787826900.309848:0:82552:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.309852:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00020000:00000080:0.0:1787826900.309856:0:82552:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.309868:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00020000:00000080:0.0:1787826900.309872:0:82552:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.309877:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00020000:00000080:0.0:1787826900.309880:0:82552:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.309884:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00020000:00000080:0.0:1787826900.309887:0:82552:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.309891:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00020000:00000080:0.0:1787826900.309894:0:82552:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.309898:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00020000:00000080:0.0:1787826900.309901:0:82552:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.309904:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00020000:00000080:0.0:1787826900.309907:0:82552:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.309911:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00020000:00000080:0.0:1787826900.309914:0:82552:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200080:0.0:1787826900.309918:0:82552:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2ed7eec0 00020000:00000080:0.0:1787826900.309921:0:82552:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606681000 uarg=0000000000000000 00000080:00200008:0.0:1787826900.309934:0:82552:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 100000 00010000:00010000:1.0:1787826900.788868:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200008:0.0:1787826901.590766:0:82604:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:0.0:1787826901.590783:0:82604:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:0.0:1787826901.590809:0:82604:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826901.590832:0:82604:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e09bf lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a2a98 expref: -99 pid: 82552 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787826901.590849:0:82604:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e09bf lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a2a98 expref: -99 pid: 82552 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:0.0:1787826901.590857:0:82604:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826901.590865:0:82604:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:0.0:1787826901.590870:0:82604:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e09bf 00000080:00010000:0.0:1787826901.590877:0:82604:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e09bf from it ffffab3ac20dfcb8 00010000:00010000:0.0:1787826901.590881:0:82604:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e09bf lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a2a98 expref: -99 pid: 82552 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787826901.590889:0:82604:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e09bf lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a2a98 expref: -99 pid: 82552 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00000080:00200008:0.0:1787826901.591426:0:82604:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 0, flags 0, foreign 0 00000080:00200000:0.0:1787826901.591431:0:82604:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:0.0:1787826901.591439:0:82604:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826901.591447:0:82604:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e09bf lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a2a98 expref: -99 pid: 82552 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00010000:00010000:0.0:1787826901.591456:0:82604:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e09bf lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a2a98 expref: -99 pid: 82552 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00000080:00200000:0.0:1787826901.591463:0:82604:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826901.591468:0:82604:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:0.0:1787826901.591470:0:82604:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e09bf 00000080:00010000:0.0:1787826901.591475:0:82604:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e09bf from it ffffab3ac20dfcb8 00010000:00010000:0.0:1787826901.591478:0:82604:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e09bf lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a2a98 expref: -99 pid: 82552 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00010000:00010000:0.0:1787826901.591485:0:82604:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e09bf lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a2a98 expref: -99 pid: 82552 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00000080:00200000:0.0:1787826901.591571:0:82604:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:0.0:1787826901.591581:0:82604:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826901.591588:0:82604:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e09bf lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a2a98 expref: -99 pid: 82552 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00010000:00010000:0.0:1787826901.591597:0:82604:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e09bf lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a2a98 expref: -99 pid: 82552 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00000080:00200000:0.0:1787826901.591604:0:82604:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826901.591608:0:82604:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:0.0:1787826901.591611:0:82604:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e09bf 00000080:00010000:0.0:1787826901.591614:0:82604:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e09bf from it ffffab3ac20dfbb0 00010000:00010000:0.0:1787826901.591618:0:82604:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e09bf lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91a2a98 expref: -99 pid: 82552 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00010000:00010000:0.0:1787826901.591625:0:82604:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06e00/0xb3f83dad1b7e09bf lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91a2a98 expref: -99 pid: 82552 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 0 00000080:00200000:0.0:1787826901.591635:0:82604:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 44 00000080:00200008:0.0:1787826901.591642:0:82604:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 100000 00000080:00200080:0.0:1787826901.591787:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00800000:00000080:0.0:1787826901.591804:0:82604:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00800000:00000004:0.0:1787826901.591811:0:82604:0:(obd_class.h:1091:obd_statfs()) lustre-MDT0000-mdc-ffff985602bc9800: age 3051, max_age 3051 00800000:00000004:0.0:1787826901.591816:0:82604:0:(obd_class.h:1131:obd_statfs()) lustre-MDT0000-mdc-ffff985602bc9800: use ffff985606a337f0 cache blocks 317355/353529 objects 1023689/1024000 00000080:00200080:0.0:1787826901.591840:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00800000:00000080:0.0:1787826901.591846:0:82604:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00800000:00000004:0.0:1787826901.591851:0:82604:0:(obd_class.h:1091:obd_statfs()) lustre-MDT0001-mdc-ffff985602bc9800: age 3051, max_age 3051 00800000:00000004:0.0:1787826901.591853:0:82604:0:(obd_class.h:1131:obd_statfs()) lustre-MDT0001-mdc-ffff985602bc9800: use ffff98560952b7f0 cache blocks 317401/353529 objects 1023737/1024000 00000080:00200080:0.0:1787826901.591866:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00800000:00000080:0.0:1787826901.591872:0:82604:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.591877:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00800000:00000080:0.0:1787826901.591882:0:82604:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.591887:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00800000:00000080:0.0:1787826901.591891:0:82604:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.591896:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00800000:00000080:0.0:1787826901.591900:0:82604:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.591907:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00800000:00000080:0.0:1787826901.591911:0:82604:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.591915:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00800000:00000080:0.0:1787826901.591919:0:82604:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.591924:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00800000:00000080:0.0:1787826901.591928:0:82604:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.591933:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00800000:00000080:0.0:1787826901.591938:0:82604:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.591943:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00800000:00000080:0.0:1787826901.591948:0:82604:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.591954:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00800000:00000080:0.0:1787826901.591959:0:82604:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.591964:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00800000:00000080:0.0:1787826901.591969:0:82604:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.591975:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00800000:00000080:0.0:1787826901.591980:0:82604:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.591986:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00800000:00000080:0.0:1787826901.591992:0:82604:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.591999:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00800000:00000080:0.0:1787826901.592004:0:82604:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592011:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00800000:00000080:0.0:1787826901.592016:0:82604:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592022:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00800000:00000080:0.0:1787826901.592027:0:82604:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592034:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00800000:00000080:0.0:1787826901.592037:0:82604:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592042:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00800000:00000080:0.0:1787826901.592046:0:82604:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592050:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00800000:00000080:0.0:1787826901.592056:0:82604:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592063:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00800000:00000080:0.0:1787826901.592068:0:82604:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592072:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00800000:00000080:0.0:1787826901.592076:0:82604:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592080:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00800000:00000080:0.0:1787826901.592085:0:82604:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592092:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00800000:00000080:0.0:1787826901.592096:0:82604:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592101:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00800000:00000080:0.0:1787826901.592107:0:82604:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592113:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00800000:00000080:0.0:1787826901.592117:0:82604:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592122:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00800000:00000080:0.0:1787826901.592127:0:82604:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592132:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00800000:00000080:0.0:1787826901.592137:0:82604:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592142:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00800000:00000080:0.0:1787826901.592146:0:82604:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592151:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00800000:00000080:0.0:1787826901.592155:0:82604:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592160:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00800000:00000080:0.0:1787826901.592164:0:82604:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592168:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00800000:00000080:0.0:1787826901.592172:0:82604:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592177:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00800000:00000080:0.0:1787826901.592181:0:82604:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592186:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00800000:00000080:0.0:1787826901.592190:0:82604:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592194:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00800000:00000080:0.0:1787826901.592198:0:82604:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592203:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00800000:00000080:0.0:1787826901.592208:0:82604:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592212:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00800000:00000080:0.0:1787826901.592217:0:82604:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592221:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00800000:00000080:0.0:1787826901.592225:0:82604:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592229:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00800000:00000080:0.0:1787826901.592233:0:82604:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592237:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00800000:00000080:0.0:1787826901.592240:0:82604:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592245:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00800000:00000080:0.0:1787826901.592249:0:82604:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592254:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00800000:00000080:0.0:1787826901.592258:0:82604:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592262:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00800000:00000080:0.0:1787826901.592266:0:82604:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592271:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00800000:00000080:0.0:1787826901.592275:0:82604:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592280:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00800000:00000080:0.0:1787826901.592284:0:82604:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592290:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00800000:00000080:0.0:1787826901.592294:0:82604:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592299:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00800000:00000080:0.0:1787826901.592303:0:82604:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592308:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00800000:00000080:0.0:1787826901.592312:0:82604:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592317:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00800000:00000080:0.0:1787826901.592321:0:82604:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592326:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00800000:00000080:0.0:1787826901.592330:0:82604:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592335:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00800000:00000080:0.0:1787826901.592339:0:82604:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592343:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00800000:00000080:0.0:1787826901.592347:0:82604:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592351:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00800000:00000080:0.0:1787826901.592354:0:82604:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592358:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00800000:00000080:0.0:1787826901.592362:0:82604:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592366:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00800000:00000080:0.0:1787826901.592369:0:82604:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592373:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00800000:00000080:0.0:1787826901.592377:0:82604:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592381:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00800000:00000080:0.0:1787826901.592385:0:82604:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592389:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00800000:00000080:0.0:1787826901.592393:0:82604:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592397:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00800000:00000080:0.0:1787826901.592401:0:82604:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592405:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00800000:00000080:0.0:1787826901.592409:0:82604:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592413:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00800000:00000080:0.0:1787826901.592416:0:82604:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592420:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00800000:00000080:0.0:1787826901.592424:0:82604:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592428:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00800000:00000080:0.0:1787826901.592432:0:82604:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592436:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00800000:00000080:0.0:1787826901.592439:0:82604:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592444:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00020000:00000080:0.0:1787826901.592451:0:82604:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00020000:00000004:0.0:1787826901.592458:0:82604:0:(obd_class.h:1091:obd_statfs()) lustre-OST0000-osc-ffff985602bc9800: age 3051, max_age 3051 00020000:00000004:0.0:1787826901.592462:0:82604:0:(obd_class.h:1131:obd_statfs()) lustre-OST0000-osc-ffff985602bc9800: use ffff985609528450 cache blocks 901357/958279 objects 261728/262144 00000080:00200080:0.0:1787826901.592485:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00020000:00000080:0.0:1787826901.592491:0:82604:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00020000:00000004:0.0:1787826901.592495:0:82604:0:(obd_class.h:1091:obd_statfs()) lustre-OST0001-osc-ffff985602bc9800: age 3051, max_age 3051 00020000:00000004:0.0:1787826901.592498:0:82604:0:(obd_class.h:1131:obd_statfs()) lustre-OST0001-osc-ffff985602bc9800: use ffff98560952d1c0 cache blocks 901363/958279 objects 261735/262144 00000080:00200080:0.0:1787826901.592511:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00020000:00000080:0.0:1787826901.592516:0:82604:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592522:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00020000:00000080:0.0:1787826901.592526:0:82604:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592531:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00020000:00000080:0.0:1787826901.592536:0:82604:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592542:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00020000:00000080:0.0:1787826901.592546:0:82604:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592551:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00020000:00000080:0.0:1787826901.592554:0:82604:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592559:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00020000:00000080:0.0:1787826901.592564:0:82604:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592569:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00020000:00000080:0.0:1787826901.592574:0:82604:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592579:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00020000:00000080:0.0:1787826901.592583:0:82604:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592588:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00020000:00000080:0.0:1787826901.592593:0:82604:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592598:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00020000:00000080:0.0:1787826901.592603:0:82604:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592608:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00020000:00000080:0.0:1787826901.592613:0:82604:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592618:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00020000:00000080:0.0:1787826901.592624:0:82604:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592630:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00020000:00000080:0.0:1787826901.592635:0:82604:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592639:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00020000:00000080:0.0:1787826901.592645:0:82604:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592650:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00020000:00000080:0.0:1787826901.592655:0:82604:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592660:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00020000:00000080:0.0:1787826901.592665:0:82604:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592683:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00020000:00000080:0.0:1787826901.592689:0:82604:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592766:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00020000:00000080:0.0:1787826901.592771:0:82604:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592777:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00020000:00000080:0.0:1787826901.592782:0:82604:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592787:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00020000:00000080:0.0:1787826901.592791:0:82604:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592797:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00020000:00000080:0.0:1787826901.592801:0:82604:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592806:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00020000:00000080:0.0:1787826901.592810:0:82604:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592815:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00020000:00000080:0.0:1787826901.592819:0:82604:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592825:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00020000:00000080:0.0:1787826901.592829:0:82604:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592834:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00020000:00000080:0.0:1787826901.592838:0:82604:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592843:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00020000:00000080:0.0:1787826901.592847:0:82604:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592852:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00020000:00000080:0.0:1787826901.592855:0:82604:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592860:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00020000:00000080:0.0:1787826901.592864:0:82604:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592869:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00020000:00000080:0.0:1787826901.592873:0:82604:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592878:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00020000:00000080:0.0:1787826901.592882:0:82604:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592886:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00020000:00000080:0.0:1787826901.592890:0:82604:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592896:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00020000:00000080:0.0:1787826901.592900:0:82604:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592905:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00020000:00000080:0.0:1787826901.592909:0:82604:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592913:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00020000:00000080:0.0:1787826901.592917:0:82604:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592922:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00020000:00000080:0.0:1787826901.592926:0:82604:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592932:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00020000:00000080:0.0:1787826901.592936:0:82604:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592941:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00020000:00000080:0.0:1787826901.592945:0:82604:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592949:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00020000:00000080:0.0:1787826901.592954:0:82604:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592959:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00020000:00000080:0.0:1787826901.592963:0:82604:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592967:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00020000:00000080:0.0:1787826901.592970:0:82604:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592975:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00020000:00000080:0.0:1787826901.592979:0:82604:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592984:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00020000:00000080:0.0:1787826901.592988:0:82604:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.592993:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00020000:00000080:0.0:1787826901.592996:0:82604:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.593000:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00020000:00000080:0.0:1787826901.593010:0:82604:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.593016:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00020000:00000080:0.0:1787826901.593020:0:82604:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.593025:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00020000:00000080:0.0:1787826901.593029:0:82604:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.593035:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00020000:00000080:0.0:1787826901.593039:0:82604:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.593044:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00020000:00000080:0.0:1787826901.593047:0:82604:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.593053:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00020000:00000080:0.0:1787826901.593057:0:82604:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.593062:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00020000:00000080:0.0:1787826901.593066:0:82604:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.593071:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00020000:00000080:0.0:1787826901.593074:0:82604:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.593078:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00020000:00000080:0.0:1787826901.593081:0:82604:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.593086:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00020000:00000080:0.0:1787826901.593089:0:82604:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.593093:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00020000:00000080:0.0:1787826901.593097:0:82604:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.593101:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00020000:00000080:0.0:1787826901.593105:0:82604:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.593109:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00020000:00000080:0.0:1787826901.593112:0:82604:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.593117:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00020000:00000080:0.0:1787826901.593120:0:82604:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.593125:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00020000:00000080:0.0:1787826901.593128:0:82604:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.593132:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00020000:00000080:0.0:1787826901.593136:0:82604:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.593140:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00020000:00000080:0.0:1787826901.593144:0:82604:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.593148:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00020000:00000080:0.0:1787826901.593151:0:82604:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.593155:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00020000:00000080:0.0:1787826901.593159:0:82604:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200080:0.0:1787826901.593162:0:82604:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffee30bede0 00020000:00000080:0.0:1787826901.593166:0:82604:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985606680c00 uarg=0000000000000000 00000080:00200008:0.0:1787826901.593175:0:82604:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 100000 00010000:00010000:0.0:1787826901.812877:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826902.836862:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826903.284963:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e7a1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:3c946890-15ca-497a-b838-fe1cba61f4dd:2384:1874669225594752:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787826903.284968:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e783c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669225594880:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787826903.285063:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e79680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669225595008:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787826903.285065:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e7ad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669225595136:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787826903.285108:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e7b840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669225595264:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787826903.286518:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e783c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669225594880:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787826903.286564:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e7a1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:3c946890-15ca-497a-b838-fe1cba61f4dd:2384:1874669225594752:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787826903.286584:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e79680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669225595008:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787826903.286600:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e7b840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669225595264:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787826903.286610:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e7ad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669225595136:192.168.203.114@tcp:400:kworker.0 00010000:00010000:1.0:1787826903.860917:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826904.884984:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826905.908950:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826906.932815:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200008:0.0:1787826907.704862:0:82141:0:(file.c:479:ll_file_release()) START file f37.replay-single:[0x200017b01:0x2:0x0](ffff98561166cb08), flags 300000 00000002:00100000:0.0:1787826907.704956:0:82141:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff985610e7a580 x1874669225589504/t206158430211(206158430211) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/608 e 0 to 0 dl 1787826904 ref 1 fl Complete:RMQU/604/ffffffff rc 301/-1 job:'multiop.0' uid:0 gid:0 projid:0 00000020:00100000:0.0:1787826907.704970:0:82141:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:0.0:1787826907.704977:0:82141:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610e7ad00 x1874669225595520/t0(0) o35->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:0.0:1787826907.705009:0:82141:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e7ad00 pname:cluuid:pid:xid:nid:opc:job multiop:7b4e8535-9db7-485d-abf6-bda966b63320:82141:1874669225595520:192.168.203.114@tcp:35:multiop.0 00000100:00100000:0.0:1787826907.705121:0:82141:0:(client.c:2715:ptlrpc_set_wait()) set ffff98561198af00 going to sleep for 11 seconds 00000100:00100000:1.0:1787826907.706940:0:82141:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e7ad00 pname:cluuid:pid:xid:nid:opc:job multiop:7b4e8535-9db7-485d-abf6-bda966b63320:82141:1874669225595520:192.168.203.114@tcp:35:multiop.0 00000002:00080000:1.0:1787826907.706979:0:82141:0:(mdc_request.c:1003:mdc_close()) @@@ Reset ESTALE = -116 req@ffff985610e7ad00 x1874669225595520/t0(0) o35->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:23/10 lens 392/456 e 0 to 0 dl 1787826923 ref 1 fl Complete:RQU/600/0 rc -116/-116 job:'multiop.0' uid:0 gid:0 projid:0 00000002:00100000:1.0:1787826907.707014:0:82141:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff985610e7a580 x1874669225589504/t206158430211(206158430211) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/608 e 0 to 0 dl 1787826904 ref 1 fl Complete:RMQU/604/ffffffff rc 301/-1 job:'multiop.0' uid:0 gid:0 projid:0 00000080:00200000:1.0:1787826907.707069:0:82141:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200017b01:0x2:0x0](ffff98561166cb08) 00010000:00010000:2.0:1787826907.957529:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826908.404838:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610db4f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2383:1874669225595648:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787826908.404877:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610db5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669225595776:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787826908.404928:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610db7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669225596032:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787826908.404929:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610db6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669225595904:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787826908.404967:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610db43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669225596160:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787826908.406800:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610db4f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2383:1874669225595648:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787826908.406835:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610db6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669225595904:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787826908.406845:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610db43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669225596160:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787826908.406916:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610db5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669225595776:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787826908.406930:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610db7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669225596032:192.168.203.114@tcp:400:kworker.0 00010000:00010000:1.0:1787826908.980843:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826910.004820:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:3.0:1787826910.234509:0:83314:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == replay-single test 38: test recovery from unlink llog (test llog_gen_rec) ========================================================== 06:35:09 (1787826909) 00010000:00010000:2.0:1787826914.100845:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826915.124822:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826916.148823:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200008:1.0:1787826916.811539:0:83714:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:1.0:1787826916.811554:0:83714:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826916.811588:0:83714:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826916.811651:0:83714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610f40200/0xb3f83dad1b7e406f lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 83714 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826916.811659:0:83714:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610f40200/0xb3f83dad1b7e406f lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 83714 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826916.811666:0:83714:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610f40200/0xb3f83dad1b7e406f lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 83714 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826916.811714:0:83714:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608660f00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:83714:1874669226262400:192.168.203.114@tcp:101:lfs.0 00000100:00100000:1.0:1787826916.811799:0:83714:0:(client.c:2715:ptlrpc_set_wait()) set ffff9856084cf780 going to sleep for 11 seconds 00000100:00100000:1.0:1787826916.813222:0:83714:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 210453399504 gen 9 00000100:00100000:1.0:1787826916.813268:0:83714:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608660f00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:83714:1874669226262400:192.168.203.114@tcp:101:lfs.0 00010000:00010000:1.0:1787826916.813283:0:83714:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610f40200/0xb3f83dad1b7e406f lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91b0589 expref: -99 pid: 83714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826916.813291:0:83714:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610f40200/0xb3f83dad1b7e406f lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91b0589 expref: -99 pid: 83714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826916.813301:0:83714:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610f40200/0xb3f83dad1b7e406f lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91b0589 expref: -99 pid: 83714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826916.813308:0:83714:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610f40200/0xb3f83dad1b7e406f lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91b0589 expref: -99 pid: 83714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826916.813315:0:83714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610f40200/0xb3f83dad1b7e406f lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91b0589 expref: -99 pid: 83714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826916.813321:0:83714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610f40200/0xb3f83dad1b7e406f lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91b0589 expref: -99 pid: 83714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826916.813325:0:83714:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610f40200/0xb3f83dad1b7e406f lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91b0589 expref: -99 pid: 83714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826916.813334:0:83714:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff985608660f00 x1874669226262400/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826932 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826916.813343:0:83714:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610f40200/0xb3f83dad1b7e406f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91b0589 expref: -99 pid: 83714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826916.813352:0:83714:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:1.0:1787826916.813356:0:83714:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:1.0:1787826916.813379:0:83714:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 69632 00000080:00010000:1.0:1787826916.813383:0:83714:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e406f 00000080:00010000:1.0:1787826916.813387:0:83714:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826916.813390:0:83714:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e406f from it ffffab3ac1ca7cb8 00010000:00010000:1.0:1787826916.813392:0:83714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610f40200/0xb3f83dad1b7e406f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91b0589 expref: -99 pid: 83714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826916.813398:0:83714:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610f40200/0xb3f83dad1b7e406f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91b0589 expref: -99 pid: 83714 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:1.0:1787826916.814107:0:83714:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 0, flags 0, foreign 0 00000080:00200000:1.0:1787826916.814111:0:83714:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826916.814121:0:83714:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826916.814129:0:83714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610f40200/0xb3f83dad1b7e406f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91b0589 expref: -99 pid: 83714 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826916.814139:0:83714:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610f40200/0xb3f83dad1b7e406f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91b0589 expref: -99 pid: 83714 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826916.814146:0:83714:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826916.814150:0:83714:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826916.814153:0:83714:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e406f 00000080:00010000:1.0:1787826916.814157:0:83714:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e406f from it ffffab3ac1ca7cb8 00010000:00010000:1.0:1787826916.814160:0:83714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610f40200/0xb3f83dad1b7e406f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91b0589 expref: -99 pid: 83714 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826916.814167:0:83714:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610f40200/0xb3f83dad1b7e406f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91b0589 expref: -99 pid: 83714 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826916.814262:0:83714:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826916.814268:0:83714:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826916.814273:0:83714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610f40200/0xb3f83dad1b7e406f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91b0589 expref: -99 pid: 83714 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826916.814279:0:83714:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610f40200/0xb3f83dad1b7e406f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91b0589 expref: -99 pid: 83714 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826916.814284:0:83714:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826916.814286:0:83714:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826916.814288:0:83714:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e406f 00000080:00010000:1.0:1787826916.814291:0:83714:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e406f from it ffffab3ac1ca7bb0 00010000:00010000:1.0:1787826916.814292:0:83714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610f40200/0xb3f83dad1b7e406f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91b0589 expref: -99 pid: 83714 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826916.814297:0:83714:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610f40200/0xb3f83dad1b7e406f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91b0589 expref: -99 pid: 83714 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 1 00000080:00200000:1.0:1787826916.814303:0:83714:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 44 00000080:00200008:1.0:1787826916.814309:0:83714:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 100000 00000080:00200080:1.0:1787826916.814393:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00800000:00000080:1.0:1787826916.814411:0:83714:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00800000:00000004:1.0:1787826916.814418:0:83714:0:(obd_class.h:1091:obd_statfs()) lustre-MDT0000-mdc-ffff985602bc9800: age 3051, max_age 3066 00000100:00100000:1.0:1787826916.814445:0:83714:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608660f00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:83714:1874669226262528:192.168.203.114@tcp:41:lfs.0 00000100:00100000:1.0:1787826916.814494:0:83714:0:(client.c:2715:ptlrpc_set_wait()) set ffff9856084cf780 going to sleep for 11 seconds 00000100:00100000:1.0:1787826916.815377:0:83714:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608660f00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:83714:1874669226262528:192.168.203.114@tcp:41:lfs.0 00800000:00000004:1.0:1787826916.815394:0:83714:0:(obd_class.h:1117:obd_statfs()) lustre-MDT0000-mdc-ffff985602bc9800: update ffff985606a337f0 cache blocks 317325/353529 objects 1023289/1024000 00000080:00200080:1.0:1787826916.815431:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00800000:00000080:1.0:1787826916.815438:0:83714:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00800000:00000004:1.0:1787826916.815442:0:83714:0:(obd_class.h:1091:obd_statfs()) lustre-MDT0001-mdc-ffff985602bc9800: age 3051, max_age 3066 00000100:00100000:1.0:1787826916.815456:0:83714:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608660f00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:83714:1874669226262656:192.168.203.114@tcp:41:lfs.0 00000100:00100000:1.0:1787826916.815484:0:83714:0:(client.c:2715:ptlrpc_set_wait()) set ffff9856084cf780 going to sleep for 11 seconds 00000100:00100000:1.0:1787826916.816095:0:83714:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608660f00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:83714:1874669226262656:192.168.203.114@tcp:41:lfs.0 00800000:00000004:1.0:1787826916.816103:0:83714:0:(obd_class.h:1117:obd_statfs()) lustre-MDT0001-mdc-ffff985602bc9800: update ffff98560952b7f0 cache blocks 317401/353529 objects 1023737/1024000 00000080:00200080:1.0:1787826916.816126:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00800000:00000080:1.0:1787826916.816133:0:83714:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.816137:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00800000:00000080:1.0:1787826916.816141:0:83714:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.816147:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00800000:00000080:1.0:1787826916.816151:0:83714:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.816156:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00800000:00000080:1.0:1787826916.816160:0:83714:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.816165:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00800000:00000080:1.0:1787826916.816169:0:83714:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.816173:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00800000:00000080:1.0:1787826916.816176:0:83714:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.816181:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00800000:00000080:1.0:1787826916.816185:0:83714:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.816189:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00800000:00000080:1.0:1787826916.816192:0:83714:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.816195:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00800000:00000080:1.0:1787826916.816197:0:83714:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.816200:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00800000:00000080:1.0:1787826916.816201:0:83714:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.816204:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00800000:00000080:1.0:1787826916.816205:0:83714:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.816207:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00800000:00000080:1.0:1787826916.816209:0:83714:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.816211:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00800000:00000080:1.0:1787826916.816213:0:83714:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.816214:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00800000:00000080:1.0:1787826916.816216:0:83714:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.816218:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00800000:00000080:1.0:1787826916.816220:0:83714:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.816222:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00800000:00000080:1.0:1787826916.816223:0:83714:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.816225:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00800000:00000080:1.0:1787826916.816227:0:83714:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.816229:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00800000:00000080:1.0:1787826916.816231:0:83714:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.816232:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00800000:00000080:1.0:1787826916.816234:0:83714:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.816236:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00800000:00000080:1.0:1787826916.816238:0:83714:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.816240:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00800000:00000080:1.0:1787826916.816242:0:83714:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.816244:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00800000:00000080:1.0:1787826916.816246:0:83714:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.816248:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00800000:00000080:1.0:1787826916.816250:0:83714:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.816252:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00800000:00000080:1.0:1787826916.816254:0:83714:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.816256:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00800000:00000080:1.0:1787826916.816258:0:83714:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.816259:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00800000:00000080:1.0:1787826916.816261:0:83714:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.816263:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00800000:00000080:1.0:1787826916.816265:0:83714:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.816267:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00800000:00000080:1.0:1787826916.816268:0:83714:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.816270:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00800000:00000080:1.0:1787826916.816272:0:83714:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.816274:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00800000:00000080:1.0:1787826916.816275:0:83714:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.816277:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00800000:00000080:1.0:1787826916.816279:0:83714:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.816281:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00800000:00000080:1.0:1787826916.816283:0:83714:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.816285:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00800000:00000080:1.0:1787826916.816286:0:83714:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.816289:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00800000:00000080:1.0:1787826916.816291:0:83714:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.816293:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00800000:00000080:1.0:1787826916.816295:0:83714:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.816297:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00800000:00000080:1.0:1787826916.816299:0:83714:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.816301:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00800000:00000080:1.0:1787826916.816302:0:83714:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.816304:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00800000:00000080:1.0:1787826916.816306:0:83714:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.816308:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00800000:00000080:1.0:1787826916.816309:0:83714:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.816311:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00800000:00000080:1.0:1787826916.816313:0:83714:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.816315:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00800000:00000080:1.0:1787826916.816317:0:83714:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.816319:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00800000:00000080:1.0:1787826916.816320:0:83714:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.816322:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00800000:00000080:1.0:1787826916.816324:0:83714:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.816326:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00800000:00000080:1.0:1787826916.816327:0:83714:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.816329:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00800000:00000080:1.0:1787826916.816331:0:83714:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.816333:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00800000:00000080:1.0:1787826916.816335:0:83714:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.816337:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00800000:00000080:1.0:1787826916.816339:0:83714:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.816341:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00800000:00000080:1.0:1787826916.816342:0:83714:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.816344:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00800000:00000080:1.0:1787826916.816346:0:83714:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.816348:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00800000:00000080:1.0:1787826916.816350:0:83714:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.816352:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00800000:00000080:1.0:1787826916.816353:0:83714:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.816355:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00800000:00000080:1.0:1787826916.816357:0:83714:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.816359:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00800000:00000080:1.0:1787826916.816360:0:83714:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.816362:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00800000:00000080:1.0:1787826916.816364:0:83714:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.816366:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00800000:00000080:1.0:1787826916.816369:0:83714:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.816373:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00800000:00000080:1.0:1787826916.816376:0:83714:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.816380:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00800000:00000080:1.0:1787826916.816383:0:83714:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.816386:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00800000:00000080:1.0:1787826916.816389:0:83714:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.816393:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00800000:00000080:1.0:1787826916.816397:0:83714:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.816401:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00800000:00000080:1.0:1787826916.816405:0:83714:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.816409:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00800000:00000080:1.0:1787826916.816413:0:83714:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.816416:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00800000:00000080:1.0:1787826916.816418:0:83714:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.816421:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00800000:00000080:1.0:1787826916.816423:0:83714:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.816425:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00020000:00000080:1.0:1787826916.816429:0:83714:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00020000:00000004:1.0:1787826916.816434:0:83714:0:(obd_class.h:1091:obd_statfs()) lustre-OST0000-osc-ffff985602bc9800: age 3051, max_age 3066 00000100:00100000:1.0:1787826916.816448:0:83714:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608660f00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:83714:1874669226262784:192.168.203.114@tcp:13:lfs.0 00000100:00100000:1.0:1787826916.816475:0:83714:0:(client.c:2715:ptlrpc_set_wait()) set ffff9856084cf780 going to sleep for 11 seconds 00000100:00100000:1.0:1787826916.817136:0:83714:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608660f00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:83714:1874669226262784:192.168.203.114@tcp:13:lfs.0 00020000:00000004:1.0:1787826916.817148:0:83714:0:(obd_class.h:1117:obd_statfs()) lustre-OST0000-osc-ffff985602bc9800: update ffff985609528450 cache blocks 901357/958279 objects 261345/262144 00000080:00200080:1.0:1787826916.817174:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00020000:00000080:1.0:1787826916.817182:0:83714:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00020000:00000004:1.0:1787826916.817187:0:83714:0:(obd_class.h:1091:obd_statfs()) lustre-OST0001-osc-ffff985602bc9800: age 3051, max_age 3066 00000100:00100000:1.0:1787826916.817202:0:83714:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608660f00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:83714:1874669226262912:192.168.203.114@tcp:13:lfs.0 00000100:00100000:1.0:1787826916.817226:0:83714:0:(client.c:2715:ptlrpc_set_wait()) set ffff9856084cf780 going to sleep for 11 seconds 00000100:00100000:1.0:1787826916.817896:0:83714:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608660f00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:83714:1874669226262912:192.168.203.114@tcp:13:lfs.0 00020000:00000004:1.0:1787826916.817909:0:83714:0:(obd_class.h:1117:obd_statfs()) lustre-OST0001-osc-ffff985602bc9800: update ffff98560952d1c0 cache blocks 901363/958279 objects 261352/262144 00000080:00200080:1.0:1787826916.817934:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00020000:00000080:1.0:1787826916.817940:0:83714:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.817944:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00020000:00000080:1.0:1787826916.817947:0:83714:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.817950:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00020000:00000080:1.0:1787826916.817953:0:83714:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.817955:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00020000:00000080:1.0:1787826916.817958:0:83714:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.817961:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00020000:00000080:1.0:1787826916.817965:0:83714:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.817968:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00020000:00000080:1.0:1787826916.817972:0:83714:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.817975:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00020000:00000080:1.0:1787826916.817979:0:83714:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.817983:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00020000:00000080:1.0:1787826916.817987:0:83714:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.817991:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00020000:00000080:1.0:1787826916.817995:0:83714:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.817999:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00020000:00000080:1.0:1787826916.818002:0:83714:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.818007:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00020000:00000080:1.0:1787826916.818009:0:83714:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.818012:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00020000:00000080:1.0:1787826916.818014:0:83714:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.818016:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00020000:00000080:1.0:1787826916.818019:0:83714:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.818021:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00020000:00000080:1.0:1787826916.818023:0:83714:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.818026:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00020000:00000080:1.0:1787826916.818028:0:83714:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.818030:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00020000:00000080:1.0:1787826916.818031:0:83714:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.818033:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00020000:00000080:1.0:1787826916.818035:0:83714:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.818037:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00020000:00000080:1.0:1787826916.818038:0:83714:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.818040:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00020000:00000080:1.0:1787826916.818042:0:83714:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.818044:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00020000:00000080:1.0:1787826916.818046:0:83714:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.818047:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00020000:00000080:1.0:1787826916.818049:0:83714:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.818051:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00020000:00000080:1.0:1787826916.818053:0:83714:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.818055:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00020000:00000080:1.0:1787826916.818056:0:83714:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.818058:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00020000:00000080:1.0:1787826916.818060:0:83714:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.818062:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00020000:00000080:1.0:1787826916.818063:0:83714:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.818066:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00020000:00000080:1.0:1787826916.818068:0:83714:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.818070:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00020000:00000080:1.0:1787826916.818072:0:83714:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.818074:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00020000:00000080:1.0:1787826916.818076:0:83714:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.818078:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00020000:00000080:1.0:1787826916.818079:0:83714:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.818081:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00020000:00000080:1.0:1787826916.818083:0:83714:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.818085:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00020000:00000080:1.0:1787826916.818086:0:83714:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.818088:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00020000:00000080:1.0:1787826916.818090:0:83714:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.818092:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00020000:00000080:1.0:1787826916.818094:0:83714:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.818095:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00020000:00000080:1.0:1787826916.818097:0:83714:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.818099:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00020000:00000080:1.0:1787826916.818101:0:83714:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.818103:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00020000:00000080:1.0:1787826916.818104:0:83714:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.818106:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00020000:00000080:1.0:1787826916.818108:0:83714:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.818110:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00020000:00000080:1.0:1787826916.818112:0:83714:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.818114:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00020000:00000080:1.0:1787826916.818116:0:83714:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.818118:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00020000:00000080:1.0:1787826916.818120:0:83714:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.818122:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00020000:00000080:1.0:1787826916.818124:0:83714:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.818126:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00020000:00000080:1.0:1787826916.818129:0:83714:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.818133:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00020000:00000080:1.0:1787826916.818136:0:83714:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.818140:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00020000:00000080:1.0:1787826916.818143:0:83714:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.818146:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00020000:00000080:1.0:1787826916.818149:0:83714:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.818152:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00020000:00000080:1.0:1787826916.818155:0:83714:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.818159:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00020000:00000080:1.0:1787826916.818162:0:83714:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.818166:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00020000:00000080:1.0:1787826916.818169:0:83714:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.818173:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00020000:00000080:1.0:1787826916.818176:0:83714:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.818179:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00020000:00000080:1.0:1787826916.818183:0:83714:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.818187:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00020000:00000080:1.0:1787826916.818190:0:83714:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.818194:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00020000:00000080:1.0:1787826916.818197:0:83714:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.818201:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00020000:00000080:1.0:1787826916.818204:0:83714:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.818208:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00020000:00000080:1.0:1787826916.818211:0:83714:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.818215:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00020000:00000080:1.0:1787826916.818219:0:83714:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.818223:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00020000:00000080:1.0:1787826916.818226:0:83714:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.818231:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00020000:00000080:1.0:1787826916.818233:0:83714:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.818236:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00020000:00000080:1.0:1787826916.818238:0:83714:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.818240:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00020000:00000080:1.0:1787826916.818242:0:83714:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.818244:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00020000:00000080:1.0:1787826916.818245:0:83714:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.818247:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00020000:00000080:1.0:1787826916.818249:0:83714:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.818251:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00020000:00000080:1.0:1787826916.818252:0:83714:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200080:1.0:1787826916.818254:0:83714:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffef76d630 00020000:00000080:1.0:1787826916.818257:0:83714:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863e800 uarg=0000000000000000 00000080:00200008:1.0:1787826916.818269:0:83714:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 100000 00000080:00200000:2.0:1787826916.835832:0:83721:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:2.0:1787826916.835864:0:83721:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826916.835885:0:83721:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610f40200/0xb3f83dad1b7e406f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91b0589 expref: -99 pid: 83714 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 1 00010000:00010000:2.0:1787826916.835902:0:83721:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610f40200/0xb3f83dad1b7e406f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91b0589 expref: -99 pid: 83714 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 1 00000080:00200000:2.0:1787826916.835911:0:83721:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787826916.835919:0:83721:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:2.0:1787826916.835922:0:83721:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e406f 00000080:00010000:2.0:1787826916.835930:0:83721:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e406f from it ffffab3ac1ca7bd0 00010000:00010000:2.0:1787826916.835934:0:83721:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610f40200/0xb3f83dad1b7e406f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91b0589 expref: -99 pid: 83714 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 1 00010000:00010000:2.0:1787826916.835944:0:83721:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610f40200/0xb3f83dad1b7e406f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91b0589 expref: -99 pid: 83714 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00000080:00200000:2.0:1787826916.835968:0:83721:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:2.0:1787826916.835982:0:83721:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=fsa-oleg314-client.virtnet, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), flags=1536 00000080:00200000:2.0:1787826916.835986:0:83721:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:2.0:1787826916.835992:0:83721:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826916.835999:0:83721:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610f40200/0xb3f83dad1b7e406f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91b0589 expref: -99 pid: 83714 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00010000:00010000:2.0:1787826916.836007:0:83721:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610f40200/0xb3f83dad1b7e406f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91b0589 expref: -99 pid: 83714 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00000080:00200000:2.0:1787826916.836013:0:83721:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787826916.836017:0:83721:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:2.0:1787826916.836020:0:83721:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e406f 00000080:00010000:2.0:1787826916.836023:0:83721:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e406f from it ffffab3ac1ca7cd0 00010000:00010000:2.0:1787826916.836026:0:83721:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610f40200/0xb3f83dad1b7e406f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91b0589 expref: -99 pid: 83714 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00010000:00010000:2.0:1787826916.836034:0:83721:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610f40200/0xb3f83dad1b7e406f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91b0589 expref: -99 pid: 83714 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00000080:00200000:2.0:1787826916.836041:0:83721:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 3 00000080:00200000:2.0:1787826916.836047:0:83721:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:2.0:1787826916.836052:0:83721:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826916.836057:0:83721:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610f40200/0xb3f83dad1b7e406f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91b0589 expref: -99 pid: 83714 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00010000:00010000:2.0:1787826916.836070:0:83721:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610f40200/0xb3f83dad1b7e406f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91b0589 expref: -99 pid: 83714 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00000080:00200000:2.0:1787826916.836077:0:83721:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787826916.836080:0:83721:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:2.0:1787826916.836083:0:83721:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e406f 00000080:00010000:2.0:1787826916.836086:0:83721:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e406f from it ffffab3ac1ca7db0 00010000:00010000:2.0:1787826916.836089:0:83721:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610f40200/0xb3f83dad1b7e406f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91b0589 expref: -99 pid: 83714 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00010000:00010000:2.0:1787826916.836096:0:83721:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610f40200/0xb3f83dad1b7e406f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91b0589 expref: -99 pid: 83714 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00000080:00200000:2.0:1787826916.836103:0:83721:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 3 00000080:00200000:2.0:1787826916.836108:0:83721:0:(namei.c:2129:ll_create_nd()) VFS Op:name=fsa-oleg314-client.virtnet, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), flags=33188, excl=1 00000080:00200000:2.0:1787826916.836112:0:83721:0:(namei.c:2084:ll_mknod()) VFS Op:name=fsa-oleg314-client.virtnet, dir=[0x200000007:0x1:0x0](ffff98561158e3c8) mode 100644 dev 0 00010000:00010000:2.0:1787826916.836139:0:83721:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610f40200/0xb3f83dad1b7e406f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x5aa6d504d91b0589 expref: -99 pid: 83714 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00010000:00010000:2.0:1787826916.836255:0:83721:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610f40200/0xb3f83dad1b7e406f lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91b0589 expref: -99 pid: 83714 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00010000:00010000:2.0:1787826916.836263:0:83721:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826916.836267:0:83721:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610f40200/0xb3f83dad1b7e406f lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91b0589 expref: -99 pid: 83714 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00000020:00100000:2.0:1787826916.836290:0:83721:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787826916.836306:0:83721:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985602ee3c00 x1874669226263168/t0(0) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 528/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826916.836334:0:83721:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee3c00 pname:cluuid:pid:xid:nid:opc:job mcreate:7b4e8535-9db7-485d-abf6-bda966b63320:83721:1874669226263168:192.168.203.114@tcp:36:mcreate.0 00000100:00100000:2.0:1787826916.836422:0:83721:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607ee2480 going to sleep for 11 seconds 00000100:00100000:1.0:1787826916.838127:0:83721:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee3c00 pname:cluuid:pid:xid:nid:opc:job mcreate:7b4e8535-9db7-485d-abf6-bda966b63320:83721:1874669226263168:192.168.203.114@tcp:36:mcreate.0 00000080:00200000:1.0:1787826916.838173:0:83721:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x2000182d1:0x321:0x0](ffff9856102c2a08) 00000080:00200000:1.0:1787826916.838223:0:83721:0:(namei.c:2137:ll_create_nd()) VFS Op:name=fsa-oleg314-client.virtnet, unhashed 1 00000080:00200000:2.0:1787826916.844911:0:83724:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:2.0:1787826916.844940:0:83724:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826916.845013:0:83724:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5c00/0xb3f83dad1b7e4076 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 83724 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826916.845024:0:83724:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5c00/0xb3f83dad1b7e4076 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 83724 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826916.845036:0:83724:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5c00/0xb3f83dad1b7e4076 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 83724 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826916.845064:0:83724:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee2940 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:83724:1874669226263296:192.168.203.114@tcp:101:rm.0 00000100:00100000:2.0:1787826916.845147:0:83724:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607ee26c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826916.846457:0:83724:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee2940 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:83724:1874669226263296:192.168.203.114@tcp:101:rm.0 00010000:00010000:1.0:1787826916.846479:0:83724:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5c00/0xb3f83dad1b7e4076 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91b05a5 expref: -99 pid: 83724 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826916.846491:0:83724:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5c00/0xb3f83dad1b7e4076 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91b05a5 expref: -99 pid: 83724 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826916.846506:0:83724:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5c00/0xb3f83dad1b7e4076 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91b05a5 expref: -99 pid: 83724 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826916.846518:0:83724:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5c00/0xb3f83dad1b7e4076 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91b05a5 expref: -99 pid: 83724 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826916.846539:0:83724:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5c00/0xb3f83dad1b7e4076 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91b05a5 expref: -99 pid: 83724 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826916.846548:0:83724:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5c00/0xb3f83dad1b7e4076 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91b05a5 expref: -99 pid: 83724 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826916.846557:0:83724:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5c00/0xb3f83dad1b7e4076 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91b05a5 expref: -99 pid: 83724 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826916.846587:0:83724:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff985602ee2940 x1874669226263296/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826932 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826916.846602:0:83724:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5c00/0xb3f83dad1b7e4076 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91b05a5 expref: -99 pid: 83724 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826916.846613:0:83724:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:1.0:1787826916.846620:0:83724:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:1.0:1787826916.846647:0:83724:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 69632 00000080:00010000:1.0:1787826916.846654:0:83724:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e4076 00000080:00010000:1.0:1787826916.846661:0:83724:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826916.846665:0:83724:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e4076 from it ffffab3ac1ca7b58 00010000:00010000:1.0:1787826916.846668:0:83724:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5c00/0xb3f83dad1b7e4076 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91b05a5 expref: -99 pid: 83724 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826916.846677:0:83724:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5c00/0xb3f83dad1b7e4076 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91b05a5 expref: -99 pid: 83724 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826916.846726:0:83724:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787826916.846744:0:83724:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=fsa-oleg314-client.virtnet, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), flags=8 00000080:00200000:1.0:1787826916.846749:0:83724:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=fsa-oleg314-client.virtnet, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=getattr 00000002:00010000:1.0:1787826916.846768:0:83724:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: fsa-oleg314-client.virtnet,[0x0:0x0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 01 00010000:00010000:1.0:1787826916.846802:0:83724:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610f41600/0xb3f83dad1b7e407d lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 83724 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826916.846810:0:83724:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610f41600/0xb3f83dad1b7e407d lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 83724 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826916.846823:0:83724:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610f41600/0xb3f83dad1b7e407d lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 83724 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826916.846839:0:83724:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608660f00 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:83724:1874669226263424:192.168.203.114@tcp:101:rm.0 00000100:00100000:1.0:1787826916.846892:0:83724:0:(client.c:2715:ptlrpc_set_wait()) set ffff9856084cf780 going to sleep for 11 seconds 00000100:00100000:1.0:1787826916.847928:0:83724:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608660f00 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:83724:1874669226263424:192.168.203.114@tcp:101:rm.0 00010000:00010000:1.0:1787826916.847938:0:83724:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610f41600/0xb3f83dad1b7e407d lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91b05c1 expref: -99 pid: 83724 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826916.847958:0:83724:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610f41600/0xb3f83dad1b7e407d lrc: 4/1,0 mode: --/PR res: [0x2000182d1:0x321:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91b05c1 expref: -99 pid: 83724 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826916.847963:0:83724:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610f41600/0xb3f83dad1b7e407d lrc: 4/1,0 mode: --/PR res: [0x2000182d1:0x321:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91b05c1 expref: -99 pid: 83724 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826916.847968:0:83724:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610f41600/0xb3f83dad1b7e407d lrc: 4/1,0 mode: PR/PR res: [0x2000182d1:0x321:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91b05c1 expref: -99 pid: 83724 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826916.847973:0:83724:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610f41600/0xb3f83dad1b7e407d lrc: 4/1,0 mode: PR/PR res: [0x2000182d1:0x321:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91b05c1 expref: -99 pid: 83724 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826916.847980:0:83724:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610f41600/0xb3f83dad1b7e407d lrc: 5/2,0 mode: PR/PR res: [0x2000182d1:0x321:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91b05c1 expref: -99 pid: 83724 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826916.847985:0:83724:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610f41600/0xb3f83dad1b7e407d lrc: 5/2,0 mode: PR/PR res: [0x2000182d1:0x321:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91b05c1 expref: -99 pid: 83724 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826916.847994:0:83724:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610f41600/0xb3f83dad1b7e407d lrc: 4/1,0 mode: PR/PR res: [0x2000182d1:0x321:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91b05c1 expref: -99 pid: 83724 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826916.848000:0:83724:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff985608660f00 x1874669226263424/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 600/584 e 0 to 0 dl 1787826932 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826916.848006:0:83724:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610f41600/0xb3f83dad1b7e407d lrc: 3/1,0 mode: PR/PR res: [0x2000182d1:0x321:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91b05c1 expref: -99 pid: 83724 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826916.848012:0:83724:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8590033617/801 (0 0) 00000080:00200000:1.0:1787826916.848015:0:83724:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:1.0:1787826916.848026:0:83724:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x2000182d1:0x321:0x0], updating i_size 0 00000080:00200000:1.0:1787826916.848029:0:83724:0:(namei.c:147:ll_iget()) got inode: [0x2000182d1:0x321:0x0](ffff9856102c2a08): rc = 0 00000080:00200000:1.0:1787826916.848042:0:83724:0:(file.c:6854:ll_layout_conf()) [0x2000182d1:0x321:0x0]: layout version change: 4294967294 -> 4294967295 00000080:00010000:1.0:1787826916.848046:0:83724:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x2000182d1:0x321:0x0](ffff9856102c2a08) for lock 0xb3f83dad1b7e407d 00000080:00010000:1.0:1787826916.848053:0:83724:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x2000182d1:0x321:0x0](ffff9856102c2a08) 00000080:00010000:1.0:1787826916.848055:0:83724:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e407d from it ffffab3ac1ca7b20 00010000:00010000:1.0:1787826916.848057:0:83724:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610f41600/0xb3f83dad1b7e407d lrc: 3/1,0 mode: PR/PR res: [0x2000182d1:0x321:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91b05c1 expref: -99 pid: 83724 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826916.848063:0:83724:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610f41600/0xb3f83dad1b7e407d lrc: 2/0,0 mode: PR/PR res: [0x2000182d1:0x321:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91b05c1 expref: -99 pid: 83724 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:1.0:1787826916.848083:0:83724:0:(file.c:6313:ll_getattr_dentry()) START file fsa-oleg314-client.virtnet:[0x2000182d1:0x321:0x0](ffff9856102c2a08), request_mask 2047, flags 0, foreign 0 00000080:00200000:1.0:1787826916.848088:0:83724:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x2000182d1:0x321:0x0](ffff9856102c2a08),name=fsa-oleg314-client.virtnet 00000002:00010000:1.0:1787826916.848097:0:83724:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x2000182d1:0x321:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826916.848103:0:83724:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610f41600/0xb3f83dad1b7e407d lrc: 2/1,0 mode: PR/PR res: [0x2000182d1:0x321:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91b05c1 expref: -99 pid: 83724 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826916.848109:0:83724:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610f41600/0xb3f83dad1b7e407d lrc: 2/1,0 mode: PR/PR res: [0x2000182d1:0x321:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91b05c1 expref: -99 pid: 83724 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826916.848113:0:83724:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826916.848116:0:83724:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x2000182d1:0x321:0x0](ffff9856102c2a08) 00000080:00010000:1.0:1787826916.848117:0:83724:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x2000182d1:0x321:0x0](ffff9856102c2a08) for lock 0xb3f83dad1b7e407d 00000080:00010000:1.0:1787826916.848120:0:83724:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e407d from it ffffab3ac1ca7cb8 00010000:00010000:1.0:1787826916.848121:0:83724:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610f41600/0xb3f83dad1b7e407d lrc: 3/1,0 mode: PR/PR res: [0x2000182d1:0x321:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91b05c1 expref: -99 pid: 83724 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826916.848126:0:83724:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610f41600/0xb3f83dad1b7e407d lrc: 2/0,0 mode: PR/PR res: [0x2000182d1:0x321:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91b05c1 expref: -99 pid: 83724 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826916.848170:0:83724:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826916.848177:0:83724:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826916.848183:0:83724:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5c00/0xb3f83dad1b7e4076 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91b05a5 expref: -99 pid: 83724 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826916.848190:0:83724:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5c00/0xb3f83dad1b7e4076 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91b05a5 expref: -99 pid: 83724 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826916.848196:0:83724:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826916.848200:0:83724:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826916.848203:0:83724:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e4076 00000080:00010000:1.0:1787826916.848206:0:83724:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e4076 from it ffffab3ac1ca7c10 00010000:00010000:1.0:1787826916.848210:0:83724:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5c00/0xb3f83dad1b7e4076 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91b05a5 expref: -99 pid: 83724 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826916.848217:0:83724:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5c00/0xb3f83dad1b7e4076 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91b05a5 expref: -99 pid: 83724 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826916.848225:0:83724:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787826916.848233:0:83724:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=fsa-oleg314-client.virtnet, flags=0 00000080:00200000:1.0:1787826916.848238:0:83724:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826916.848244:0:83724:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826916.848250:0:83724:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5c00/0xb3f83dad1b7e4076 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91b05a5 expref: -99 pid: 83724 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826916.848256:0:83724:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5c00/0xb3f83dad1b7e4076 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91b05a5 expref: -99 pid: 83724 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826916.848260:0:83724:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826916.848262:0:83724:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826916.848264:0:83724:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e4076 00000080:00010000:1.0:1787826916.848266:0:83724:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e4076 from it ffffab3ac1ca7d68 00010000:00010000:1.0:1787826916.848269:0:83724:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5c00/0xb3f83dad1b7e4076 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91b05a5 expref: -99 pid: 83724 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826916.848273:0:83724:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5c00/0xb3f83dad1b7e4076 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91b05a5 expref: -99 pid: 83724 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787826916.848277:0:83724:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787826916.848282:0:83724:0:(namei.c:2418:ll_unlink()) VFS Op:name=fsa-oleg314-client.virtnet, dir=[0x200000007:0x1:0x0](ffff98561158e3c8) 00010000:00010000:1.0:1787826916.848297:0:83724:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5c00/0xb3f83dad1b7e4076 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x5aa6d504d91b05a5 expref: -99 pid: 83724 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787826916.848376:0:83724:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610f41600/0xb3f83dad1b7e407d lrc: 2/0,0 mode: PR/PR res: [0x2000182d1:0x321:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x5aa6d504d91b05c1 expref: -99 pid: 83724 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826916.848387:0:83724:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x321:0x0]: losing layout lock 00010000:00010000:1.0:1787826916.848411:0:83724:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610f41600/0xb3f83dad1b7e407d lrc: 1/0,0 mode: --/PR res: [0x2000182d1:0x321:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x5aa6d504d91b05c1 expref: -99 pid: 83724 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826916.848419:0:83724:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5c00/0xb3f83dad1b7e4076 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91b05a5 expref: -99 pid: 83724 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787826916.848424:0:83724:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 2 locks packed 00010000:00010000:1.0:1787826916.848427:0:83724:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610f41600/0xb3f83dad1b7e407d lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x321:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x5aa6d504d91b05c1 expref: -99 pid: 83724 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826916.848441:0:83724:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5c00/0xb3f83dad1b7e4076 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91b05a5 expref: -99 pid: 83724 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000020:00100000:1.0:1787826916.848458:0:83724:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787826916.848466:0:83724:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985608660f00 x1874669226263680/t0(0) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826916.848480:0:83724:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608660f00 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:83724:1874669226263680:192.168.203.114@tcp:36:rm.0 00000100:00100000:1.0:1787826916.848527:0:83724:0:(client.c:2715:ptlrpc_set_wait()) set ffff9856084cf780 going to sleep for 11 seconds 00000100:00100000:1.0:1787826916.850029:0:83724:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608660f00 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:83724:1874669226263680:192.168.203.114@tcp:36:rm.0 00000080:00200000:1.0:1787826916.850080:0:83724:0:(vvp_io.c:1885:vvp_io_init()) [0x2000182d1:0x321:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826916.850090:0:83724:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x321:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826916.850100:0:83724:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x321:0x0](ffff9856102c2a08) 00010000:00010000:2.0:1787826917.172808:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826918.196856:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826918.644951:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856087712c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669226263936:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787826918.644956:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608771e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:3c946890-15ca-497a-b838-fe1cba61f4dd:2384:1874669226263808:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787826918.645190:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608771680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669226264192:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787826918.645203:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608772d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669226264064:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787826918.645269:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856087703c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669226264320:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787826918.647779:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608771e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:3c946890-15ca-497a-b838-fe1cba61f4dd:2384:1874669226263808:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787826918.647799:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608772d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669226264064:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787826918.647809:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856087703c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669226264320:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787826918.649477:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856087712c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669226263936:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787826918.649498:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608771680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669226264192:192.168.203.114@tcp:400:kworker.0 00000001:02000400:3.0:1787826919.070142:0:83789:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: local REPLAY BARRIER on lustre-MDT0000 00010000:00010000:2.0:1787826919.221046:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826920.244913:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826921.268832:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826922.292872:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826923.316853:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826923.765019:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2385:1874669226264448:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787826923.765035:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669226264576:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787826923.765174:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669226264704:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787826923.765179:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669226264832:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787826923.765263:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669226264960:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787826923.766955:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669226264704:192.168.203.114@tcp:400:kworker.0 00000100:00080000:3.0:1787826923.766955:0:2386:0:(recover.c:205:ptlrpc_request_handle_notconn()) import lustre-MDT0000-mdc-ffff985602bc9800 of lustre-MDT0000_UUID@192.168.203.114@tcp abruptly disconnected: reconnecting 00000100:00080000:3.0:1787826923.766966:0:2386:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from FULL to DISCONN 00000100:02000400:3.0:1787826923.766974:0:2386:0:(import.c:182:ptlrpc_set_import_discon()) lustre-MDT0000-mdc-ffff985602bc9800: Connection to lustre-MDT0000 (at 192.168.203.114@tcp) was lost; in progress operations using this service will wait for recovery to complete 00000100:00100000:2.0:1787826923.766981:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669226264960:192.168.203.114@tcp:400:kworker.0 00000100:00080000:3.0:1787826923.766983:0:2386:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826923.766995:0:2386:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3048 00000100:00080000:3.0:1787826923.767010:0:2386:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787826923.767024:0:2386:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787826923.767053:0:2386:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610e83840 x1874669226265088/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826923.767077:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669226264576:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787826923.767090:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669226264832:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787826923.767132:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669226265088:192.168.203.114@tcp:38:ptlrpcd_01_01.0 00000100:00080000:1.0:1787826923.767904:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787826923.767912:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-19) 00000100:00100000:1.0:1787826923.767916:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669226265088:192.168.203.114@tcp:38:ptlrpcd_01_01.0 00010000:00010000:2.0:1787826924.340841:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826925.364844:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826926.389202:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826927.412863:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826928.436884:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787826928.884914:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826928.884927:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00100000:2.0:1787826928.884930:0:2385:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00080000:3.0:1787826928.884932:0:38:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826928.884946:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3074 00000100:00100000:2.0:1787826928.884953:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2385:1874669226265216:192.168.203.114@tcp:400:kworker.0 00000100:00080000:3.0:1787826928.884960:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 5s 00000100:00080000:3.0:1787826928.884976:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787826928.884997:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787826928.885021:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610e82d00 x1874669226265344/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826928.885079:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e83c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669226265472:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787826928.885121:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669226265728:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787826928.885142:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669226265344:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787826928.885153:0:2385:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1787826928.885165:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669226265600:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787826928.886776:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669226265728:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787826928.886793:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e83c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669226265472:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787826928.886833:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669226265600:192.168.203.114@tcp:400:kworker.0 00000100:00080000:1.0:1787826928.886894:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787826928.886903:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-19) 00000100:00100000:1.0:1787826928.886908:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669226265344:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787826929.460863:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826930.484856:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826931.508840:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826932.532980:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826933.556904:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787826934.004890:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826934.004899:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787826934.004903:0:38:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826934.004921:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3079 00000100:00080000:3.0:1787826934.004935:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 10s 00000100:00080000:3.0:1787826934.004943:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787826934.004954:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787826934.004987:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610e821c0 x1874669226265984/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826934.005027:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:3c946890-15ca-497a-b838-fe1cba61f4dd:2386:1874669226265856:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787826934.005074:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e821c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669226265984:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787826934.005100:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669226266112:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787826934.005166:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669226266240:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787826934.005187:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e83480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669226266368:192.168.203.114@tcp:400:kworker.0 00000100:00080000:1.0:1787826934.006529:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787826934.006539:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-19) 00000100:00100000:1.0:1787826934.006547:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e821c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669226265984:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787826934.006547:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669226266240:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787826934.006591:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669226266112:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787826934.006604:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e83480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669226266368:192.168.203.114@tcp:400:kworker.0 00010000:00010000:1.0:1787826934.581658:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826935.605241:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826936.628986:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826937.652865:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200008:2.0:1787826937.920980:0:84106:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:2.0:1787826937.920990:0:84106:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:2.0:1787826937.921010:0:84106:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826937.921067:0:84106:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5400/0xb3f83dad1b7e4084 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 84106 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826937.921078:0:84106:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5400/0xb3f83dad1b7e4084 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 84106 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826937.921088:0:84106:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5400/0xb3f83dad1b7e4084 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 84106 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787826937.921110:0:84106:0:(client.c:1845:ptlrpc_send_new_req()) @@@ req waiting for recovery: (FULL != DISCONN) req@ffff985602ee0780 x1874669226266496/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 576/1152 e 0 to 0 dl 0 ref 2 fl Rpc:WQU/600/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:0 00000100:00100000:2.0:1787826937.921118:0:84106:0:(client.c:2715:ptlrpc_set_wait()) set ffff985608840900 going to sleep for 0 seconds 00010000:00010000:2.0:1787826938.676875:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826938.932828:0:84106:0:(client.c:2715:ptlrpc_set_wait()) set ffff985608840900 going to sleep for 0 seconds 00000100:00000400:2.0:1787826939.124792:0:2385:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787826923/real 1787826923] req@ffff985610e803c0 x1874669226264448/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 1 dl 1787826939 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826939.124824:0:2385:0:(client.c:2547:ptlrpc_expire_one_request()) @@@ TIMEOUT-NORESEND: req@ffff985610e803c0 x1874669226264448/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 1 dl 1787826939 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826939.124834:0:2385:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from FULL to DISCONN 00000100:02020000:2.0:1787826939.124839:0:2385:0:(import.c:186:ptlrpc_set_import_discon()) MGC192.168.203.114@tcp: Connection to MGS (at 192.168.203.114@tcp) was lost; in progress operations using this service will fail 00000100:00080000:2.0:1787826939.124842:0:2385:0:(import.c:192:ptlrpc_set_import_discon()) import MGS@192.168.203.114@tcp for MGC192.168.203.114@tcp not replayable, auto-deactivating 00000100:00080000:2.0:1787826939.124843:0:2385:0:(import.c:138:ptlrpc_deactivate_import_nolock()) setting import MGS INVALID 00000100:00100000:2.0:1787826939.124846:0:2385:0:(client.c:3589:ptlrpc_abort_inflight()) @@@ inflight req@ffff985610e803c0 x1874669226264448/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 1 dl 1787826939 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826939.124852:0:2385:0:(client.c:3589:ptlrpc_abort_inflight()) @@@ inflight req@ffff985610e81e00 x1874669226265216/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787826944 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787826939.124859:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1787826939.124860:0:2385:0:(client.c:3589:ptlrpc_abort_inflight()) @@@ inflight req@ffff985610e82580 x1874669226265856/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787826950 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787826939.124866:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787826939.124869:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787826939.124879:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3084 00000100:00080000:1.0:1787826939.124891:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 15s 10000000:01000000:2.0:1787826939.124892:0:2385:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808001 00000100:00080000:2.0:1787826939.124896:0:2385:0:(pinger.c:193:ptlrpc_pinger_ir_down()) IR down 10000000:01000000:2.0:1787826939.124898:0:2385:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808002 00000100:00100000:0.0:1787826939.124899:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608663840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:3c946890-15ca-497a-b838-fe1cba61f4dd:0:1874669226266624:192.168.203.114@tcp:400:kworker.0 00000100:00080000:1.0:1787826939.124900:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00080000:2.0:1787826939.124902:0:2385:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000008:00100000:1.0:1787826939.124917:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787826939.124935:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985608660000 x1874669226266752/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826939.124937:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2385:1874669226265216:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787826939.124944:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:3c946890-15ca-497a-b838-fe1cba61f4dd:2386:1874669226265856:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787826939.124952:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2385:1874669226264448:192.168.203.114@tcp:400:kworker.0 00000100:00080000:1.0:1787826939.125012:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1787826939.125017:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608660780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669226266880:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787826939.125022:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608660000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669226266752:192.168.203.114@tcp:38:kworker.0 00000100:00080000:1.0:1787826939.125025:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787826939.125032:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787826939.125038:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3048 00000100:00080000:1.0:1787826939.125044:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787826939.125057:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985608663480 x1874669226267264/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826939.125081:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856086630c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669226267008:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787826939.125170:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608661a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669226267136:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787826939.125175:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608663480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669226267264:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787826939.126673:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608660780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669226266880:192.168.203.114@tcp:400:kworker.0 00010000:00010000:1.0:1787826939.126721:0:2383:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 0/0 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1787826939.126730:0:2384:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 0/0 locks to regular blp list, 2 blwis in pool 00000100:00100000:1.0:1787826939.126732:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856086630c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669226267008:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787826939.126740:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608661a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669226267136:192.168.203.114@tcp:400:kworker.0 00010000:00010000:3.0:1787826939.126825:0:15163:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 0 locks of 0 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1787826939.126827:0:33341:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 0 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787826939.126842:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826939.126844:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826939.126846:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826939.126847:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826939.126850:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826939.126854:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787826939.127314:0:2382:0:(import.c:1132:ptlrpc_connect_interpret()) MGC192.168.203.114@tcp: connect to target with instance 0 10000000:01000000:2.0:1787826939.127322:0:2382:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808005 00000100:02000400:2.0:1787826939.127328:0:2382:0:(import.c:1284:ptlrpc_connect_interpret()) Evicted from MGS (at 192.168.203.114@tcp) after server handle changed from 0x5aa6d504d91a29aa to 0x5aa6d504d91b12c5 00000100:00080000:2.0:1787826939.127331:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to EVICTED 00000100:00080000:2.0:1787826939.127336:0:2382:0:(import.c:1674:ptlrpc_import_recovery_state_machine()) evicted from MGS@192.168.203.114@tcp; invalidating 00000100:00100000:2.0:1787826939.127554:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608663480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669226267264:192.168.203.114@tcp:250:kworker.0 00000100:00080000:0.0:1787826939.127592:0:84121:0:(import.c:1598:ptlrpc_invalidate_import_thread()) thread invalidate import MGC192.168.203.114@tcp to MGS@192.168.203.114@tcp 00000100:00100000:0.0:1787826939.127597:0:84121:0:(import.c:323:ptlrpc_invalidate_import()) Sleeping 20s for inflight to error out 10000000:01000000:0.0:1787826939.127600:0:84121:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808003 00010000:00010000:0.0:1787826939.127614:0:84121:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.114@tcp lock: ffff985609eb0400/0xb3f83dad1b7e09aa lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x110e400000000 nid: local remote: 0x5aa6d504d91a2a7c expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:0.0:1787826939.127625:0:84121:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.114@tcp lock: ffff985609eb0400/0xb3f83dad1b7e09aa lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x110f400000000 nid: local remote: 0x5aa6d504d91a2a7c expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787826939.127631:0:84121:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:0.0:1787826939.127646:0:84121:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787826939.127654:0:84121:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.114@tcp lock: ffff985609eb0400/0xb3f83dad1b7e09aa lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x590f400000000 nid: local remote: 0x5aa6d504d91a2a7c expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826939.127668:0:84121:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.114@tcp lock: ffff985642051000/0xb3f83dad1b7e09b8 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x110e400000000 nid: local remote: 0x5aa6d504d91a2a8a expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:0.0:1787826939.127675:0:84121:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.114@tcp lock: ffff985642051000/0xb3f83dad1b7e09b8 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x110f400000000 nid: local remote: 0x5aa6d504d91a2a8a expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787826939.127680:0:84121:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:0.0:1787826939.127687:0:84121:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787826939.127720:0:84121:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.114@tcp lock: ffff985642051000/0xb3f83dad1b7e09b8 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x590f400000000 nid: local remote: 0x5aa6d504d91a2a8a expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826939.127738:0:84121:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.114@tcp lock: ffff985609eb1a00/0xb3f83dad1b7e09a3 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x110e400000000 nid: local remote: 0x5aa6d504d91a2a75 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:0.0:1787826939.127745:0:84121:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.114@tcp lock: ffff985609eb1a00/0xb3f83dad1b7e09a3 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x110f400000000 nid: local remote: 0x5aa6d504d91a2a75 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787826939.127750:0:84121:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x2:0x0].0x0 (lustre) 00010000:00010000:0.0:1787826939.127774:0:84121:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787826939.127780:0:84121:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.114@tcp lock: ffff985609eb1a00/0xb3f83dad1b7e09a3 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x590f400000000 nid: local remote: 0x5aa6d504d91a2a75 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826939.127794:0:84121:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.114@tcp lock: ffff985609eb0c00/0xb3f83dad1b7e09b1 lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x110e400000000 nid: local remote: 0x5aa6d504d91a2a83 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:0.0:1787826939.127801:0:84121:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.114@tcp lock: ffff985609eb0c00/0xb3f83dad1b7e09b1 lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x110f400000000 nid: local remote: 0x5aa6d504d91a2a83 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787826939.127807:0:84121:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x736d61726170:0x3:0x0].0x0 (params) 00010000:00010000:0.0:1787826939.127815:0:84121:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787826939.127820:0:84121:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.114@tcp lock: ffff985609eb0c00/0xb3f83dad1b7e09b1 lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x590f400000000 nid: local remote: 0x5aa6d504d91a2a83 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1787826939.127837:0:84121:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from EVICTED to RECOVER 00000100:00080000:0.0:1787826939.127841:0:84121:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from RECOVER to FULL 10000000:01000000:0.0:1787826939.127844:0:84121:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808004 00000100:00080000:0.0:1787826939.127854:0:84121:0:(pinger.c:186:ptlrpc_pinger_ir_up()) IR up 00000100:02000000:0.0:1787826939.127859:0:84121:0:(import.c:1743:ptlrpc_import_recovery_state_machine()) MGC192.168.203.114@tcp: Connection restored to 192.168.203.114@tcp (at 192.168.203.114@tcp) 00010000:00080000:3.0:1787826939.127999:0:2382:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=46162) req@ffff985608660000 x1874669226266752/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/416 e 0 to 0 dl 1787826964 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826939.128009:0:2382:0:(import.c:1132:ptlrpc_connect_interpret()) lustre-MDT0000-mdc-ffff985602bc9800: connect to target with instance 50 00000100:00080000:3.0:1787826939.128021:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to REPLAY 00000100:00080000:3.0:1787826939.128024:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:3.0:1787826939.128026:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 210453399504 gen 9 00000100:00100000:3.0:1787826939.128033:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985602ee3c00 x1874669226263168/t210453399505(210453399505) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 528/448 e 0 to 0 dl 1787826932 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826939.128041:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 210453399504 last 0 00000100:00080000:3.0:1787826939.128048:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985602ee3c00 x1874669226263168/t210453399505(210453399505) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 528/448 e 0 to 0 dl 1787826932 ref 1 fl New:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826939.128059:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608660000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669226266752:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787826939.128075:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669226263168:192.168.203.114@tcp:36:mcreate.0 00000100:00080000:3.0:1787826939.528537:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985602ee3c00 x1874669226263168/t210453399505(210453399505) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 528/448 e 0 to 0 dl 1787826955 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826939.528554:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:3.0:1787826939.528557:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 210453399504 gen 9 00000100:00100000:3.0:1787826939.528563:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985602ee3c00 x1874669226263168/t210453399505(210453399505) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 528/448 e 0 to 0 dl 1787826955 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826939.528571:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 210453399504 last 210453399505 00000100:00080000:3.0:1787826939.528581:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985608660f00 x1874669226263680/t210453399506(210453399506) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787826932 ref 1 fl New:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826939.528597:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669226263168:192.168.203.114@tcp:36:mcreate.0 00000100:00100000:3.0:1787826939.528625:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608660f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669226263680:192.168.203.114@tcp:36:rm.0 00000100:00080000:3.0:1787826939.530173:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985608660f00 x1874669226263680/t210453399506(210453399506) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787826955 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826939.530183:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:3.0:1787826939.530185:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 210453399504 gen 9 00000100:00100000:3.0:1787826939.530189:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985602ee3c00 x1874669226263168/t210453399505(210453399505) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 528/448 e 0 to 0 dl 1787826955 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826939.530194:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 210453399504 last 210453399506 00000100:00080000:3.0:1787826939.530198:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from REPLAY to REPLAY_LOCKS 00000100:00100000:3.0:1787826939.530504:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608660f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669226263680:192.168.203.114@tcp:36:rm.0 00010000:00080000:0.0:1787826939.530573:0:84122:0:(ldlm_request.c:2972:ldlm_lock_replay_thread()) lock replay thread lustre-MDT0000-mdc-ffff985602bc9800 to lustre-MDT0000_UUID@192.168.203.114@tcp 00010000:00010000:0.0:1787826939.530581:0:84122:0:(ldlm_request.c:2888:ldlm_cancel_unused_locks_for_replay()) Dropping as many unused locks as possible before replay for namespace lustre-MDT0000-mdc-ffff985602bc9800 (400) 00000080:00200000:0.0:1787826939.531078:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x320:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.531136:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084cd200/0xb3f83dad1b7e3578 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x320:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91ac3c6 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.531161:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x31f:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.531174:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084cda00/0xb3f83dad1b7e356a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x31f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91ac395 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.531193:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x31e:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.531203:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084cd800/0xb3f83dad1b7e355c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x31e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91ac364 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.531216:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x31d:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.531227:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084cdc00/0xb3f83dad1b7e354e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x31d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91ac333 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.531240:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x31c:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.531257:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084cd000/0xb3f83dad1b7e3540 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x31c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91ac302 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.531270:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x31b:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.531280:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608569400/0xb3f83dad1b7e3532 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x31b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91ac2d1 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.531292:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x31a:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.531304:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608568a00/0xb3f83dad1b7e3524 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x31a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91ac2a0 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.531316:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x319:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.531325:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608595800/0xb3f83dad1b7e3516 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x319:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91ac26f expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.531339:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x318:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.531349:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608594200/0xb3f83dad1b7e3508 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x318:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91ac23e expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.531366:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x317:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.531376:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608568600/0xb3f83dad1b7e34fa lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x317:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91ac20d expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.531399:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x316:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.531421:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608568400/0xb3f83dad1b7e34ec lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x316:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91ac1dc expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.531440:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x315:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.531451:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608568800/0xb3f83dad1b7e34de lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x315:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91ac1ab expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.531463:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x314:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.531475:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608568c00/0xb3f83dad1b7e34d0 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x314:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91ac17a expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.531487:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x313:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.531496:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608569a00/0xb3f83dad1b7e34c2 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x313:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91ac149 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.531508:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x312:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.531519:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608569800/0xb3f83dad1b7e34b4 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x312:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91ac118 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.531531:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x311:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.531541:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608569e00/0xb3f83dad1b7e34a6 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x311:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91ac0e7 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.531557:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x310:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.531567:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608569c00/0xb3f83dad1b7e3498 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x310:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91ac0b6 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.531580:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x30f:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.531592:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608569000/0xb3f83dad1b7e348a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x30f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91ac085 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.531604:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x30e:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.531614:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608569600/0xb3f83dad1b7e347c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x30e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91ac054 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.531627:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x30d:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.531637:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608568e00/0xb3f83dad1b7e346e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x30d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91ac023 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.531649:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x30c:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.531660:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608568200/0xb3f83dad1b7e3460 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x30c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91abff2 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.531674:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x30b:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.531684:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608546a00/0xb3f83dad1b7e3452 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x30b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91abfc1 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.531738:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x30a:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.531750:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608530000/0xb3f83dad1b7e3444 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x30a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91abf90 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.531773:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x309:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.531786:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084ea800/0xb3f83dad1b7e3436 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x309:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91abf5f expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.531800:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x308:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.531810:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608595200/0xb3f83dad1b7e3428 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x308:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91abf2e expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.531823:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x307:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.531833:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608595a00/0xb3f83dad1b7e341a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x307:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91abefd expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.531846:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x306:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.531855:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608595c00/0xb3f83dad1b7e340c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x306:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91abecc expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.531866:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x305:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.531876:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608594a00/0xb3f83dad1b7e33fe lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x305:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91abe9b expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.531888:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x304:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.531897:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608595000/0xb3f83dad1b7e33f0 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x304:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91abe6a expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.531908:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x303:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.531917:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608594c00/0xb3f83dad1b7e33e2 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x303:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91abe39 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.531935:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x302:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.531945:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608594e00/0xb3f83dad1b7e33d4 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x302:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91abe08 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.531956:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x301:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.531966:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608594600/0xb3f83dad1b7e33c6 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x301:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91abdd7 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.531978:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x300:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.531989:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608595600/0xb3f83dad1b7e33b8 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x300:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91abda6 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.532000:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2ff:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.532011:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608595e00/0xb3f83dad1b7e33aa lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2ff:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91abd75 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.532023:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2fe:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.532035:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608595400/0xb3f83dad1b7e339c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2fe:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91abd44 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.532048:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2fd:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.532058:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608594000/0xb3f83dad1b7e338e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2fd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91abd13 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.532071:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2fc:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.532102:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608594800/0xb3f83dad1b7e3380 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2fc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91abce2 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.532126:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2fb:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.532139:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084a9e00/0xb3f83dad1b7e3372 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2fb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91abcb1 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.532153:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2fa:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.532163:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608546e00/0xb3f83dad1b7e3364 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2fa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91abc80 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.532175:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2f9:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.532186:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608546c00/0xb3f83dad1b7e3356 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2f9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91abc4f expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.532200:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2f8:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.532214:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608547000/0xb3f83dad1b7e3348 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2f8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91abc1e expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.532227:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2f7:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.532237:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608546200/0xb3f83dad1b7e333a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2f7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91abbed expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.532251:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2f6:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.532261:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608547c00/0xb3f83dad1b7e332c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2f6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91abbbc expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.532274:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2f5:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.532285:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608547600/0xb3f83dad1b7e331e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2f5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91abb8b expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.532301:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2f4:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.532312:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608547200/0xb3f83dad1b7e3310 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2f4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91abb5a expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.532325:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2f3:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.532336:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608546400/0xb3f83dad1b7e3302 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2f3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91abb29 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.532349:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2f2:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.532360:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608546600/0xb3f83dad1b7e32f4 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2f2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91abaf8 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.532373:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2f1:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.532384:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608547e00/0xb3f83dad1b7e32e6 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2f1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91abac7 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.532398:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2f0:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.532427:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608547400/0xb3f83dad1b7e32d8 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2f0:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aba96 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.532442:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2ef:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.532452:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608546000/0xb3f83dad1b7e32ca lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2ef:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aba65 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.532465:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2ee:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.532476:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608546800/0xb3f83dad1b7e32bc lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2ee:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aba34 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.532493:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2ed:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.532504:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608547a00/0xb3f83dad1b7e32ae lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2ed:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aba03 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.532517:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2ec:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.532529:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608531e00/0xb3f83dad1b7e32a0 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2ec:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91ab9d2 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.532542:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2eb:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.532553:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560850c200/0xb3f83dad1b7e3292 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2eb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91ab9a1 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.532566:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2ea:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.532576:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608531200/0xb3f83dad1b7e3284 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2ea:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91ab970 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.532590:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2e9:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.532601:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608531a00/0xb3f83dad1b7e3276 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2e9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91ab93f expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.532613:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2e8:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.532624:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608530800/0xb3f83dad1b7e3268 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2e8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91ab90e expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.532638:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2e7:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.532650:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608531c00/0xb3f83dad1b7e325a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2e7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91ab8dd expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.532667:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2e6:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.532678:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608531400/0xb3f83dad1b7e324c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2e6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91ab8ac expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.532717:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2e5:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.532730:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608531800/0xb3f83dad1b7e323e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2e5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91ab87b expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.532743:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2e4:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.532756:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608530a00/0xb3f83dad1b7e3230 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2e4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91ab84a expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.532769:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2e3:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.532782:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608530c00/0xb3f83dad1b7e3222 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2e3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91ab819 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.532794:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2e2:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.532805:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608531000/0xb3f83dad1b7e3214 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2e2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91ab7e8 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.532818:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2e1:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.532828:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608530600/0xb3f83dad1b7e3206 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2e1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91ab7b7 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.532841:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2e0:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.532853:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608531600/0xb3f83dad1b7e31f8 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2e0:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91ab786 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.532870:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2df:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.532881:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608530e00/0xb3f83dad1b7e31ea lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2df:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91ab755 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.532893:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2de:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.532903:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608530200/0xb3f83dad1b7e31dc lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2de:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91ab724 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.532916:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2dd:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.532927:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608530400/0xb3f83dad1b7e31ce lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2dd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91ab6f3 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.532940:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2dc:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.532951:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560851f400/0xb3f83dad1b7e31c0 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2dc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91ab6c2 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.532963:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2db:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.532974:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560850d200/0xb3f83dad1b7e31b2 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2db:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91ab691 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.532986:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2da:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.532997:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560850dc00/0xb3f83dad1b7e31a4 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2da:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91ab660 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.533009:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2d9:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.533021:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560851e600/0xb3f83dad1b7e3196 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2d9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91ab62f expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.533039:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2d8:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.533050:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560851fe00/0xb3f83dad1b7e3188 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2d8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91ab5fe expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.533063:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2d7:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.533074:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560851e200/0xb3f83dad1b7e317a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2d7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91ab5cd expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.533087:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2d6:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.533099:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560851e000/0xb3f83dad1b7e316c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2d6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91ab59c expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.533112:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2d5:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.533124:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560851ee00/0xb3f83dad1b7e315e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2d5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91ab56b expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.533136:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2d4:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.533148:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560851e800/0xb3f83dad1b7e3150 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2d4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91ab53a expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.533161:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2d3:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.533172:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560851f200/0xb3f83dad1b7e3142 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2d3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91ab509 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.533185:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2d2:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.533196:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560851ea00/0xb3f83dad1b7e3134 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2d2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91ab4d8 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.533212:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2d1:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.533223:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560851e400/0xb3f83dad1b7e3126 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2d1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91ab4a7 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.533236:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2d0:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.533247:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560851ec00/0xb3f83dad1b7e3118 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2d0:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91ab476 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.533260:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2cf:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.533270:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560851fa00/0xb3f83dad1b7e310a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2cf:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91ab445 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.533283:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2ce:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.533294:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560851f800/0xb3f83dad1b7e30fc lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2ce:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91ab414 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.533307:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2cd:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.533317:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560851fc00/0xb3f83dad1b7e30ee lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2cd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91ab3e3 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.533329:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2cc:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.533338:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560851f000/0xb3f83dad1b7e30e0 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2cc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91ab3b2 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.533350:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2cb:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.533361:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560851f600/0xb3f83dad1b7e30d2 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2cb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91ab381 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.533378:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2ca:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.533389:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084eb800/0xb3f83dad1b7e30c4 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2ca:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91ab350 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.533401:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2c9:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.533432:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560850ce00/0xb3f83dad1b7e30b6 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2c9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91ab31f expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.533446:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2c8:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.533458:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608418400/0xb3f83dad1b7e30a8 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2c8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91ab2ee expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.533471:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2c7:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.533481:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560850c000/0xb3f83dad1b7e309a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2c7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91ab2bd expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.533493:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2c6:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.533502:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560850d000/0xb3f83dad1b7e308c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2c6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91ab28c expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.533515:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2c5:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.533526:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560850c400/0xb3f83dad1b7e307e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2c5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91ab25b expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.533537:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2c4:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.533549:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560850c800/0xb3f83dad1b7e3070 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2c4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91ab22a expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.533566:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2c3:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.533577:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560850da00/0xb3f83dad1b7e3062 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2c3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91ab1f9 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.533590:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2c2:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.533600:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560850d800/0xb3f83dad1b7e3054 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2c2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91ab1c8 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.533612:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2c1:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.533623:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560850ca00/0xb3f83dad1b7e3046 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2c1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91ab197 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.533636:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2c0:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.533646:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560850d600/0xb3f83dad1b7e3038 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2c0:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91ab166 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.533658:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2bf:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.533668:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560850cc00/0xb3f83dad1b7e302a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2bf:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91ab135 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.533682:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2be:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.533711:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560850c600/0xb3f83dad1b7e301c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2be:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91ab104 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.533725:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2bd:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.533736:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560850de00/0xb3f83dad1b7e300e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2bd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91ab0d3 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.533753:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2bc:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.533764:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560850d400/0xb3f83dad1b7e3000 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2bc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91ab0a2 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.533778:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2bb:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.533789:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608418800/0xb3f83dad1b7e2ff2 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2bb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91ab071 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.533801:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2ba:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.533811:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084eae00/0xb3f83dad1b7e2fe4 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2ba:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91ab040 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.533823:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2b9:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.533832:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084ea600/0xb3f83dad1b7e2fd6 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2b9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91ab00f expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.533843:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2b8:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.533854:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084eaa00/0xb3f83dad1b7e2fc8 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2b8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aafde expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.533867:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2b7:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.533878:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084eb200/0xb3f83dad1b7e2fba lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2b7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aafad expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.533891:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2b6:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.533903:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084eba00/0xb3f83dad1b7e2fac lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2b6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aaf7c expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.533919:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2b5:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.533930:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084eb000/0xb3f83dad1b7e2f9e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2b5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aaf4b expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.533943:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2b4:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.533954:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084ea400/0xb3f83dad1b7e2f90 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2b4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aaf1a expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.533965:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2b3:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.533975:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084ebc00/0xb3f83dad1b7e2f82 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2b3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aaee9 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.533986:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2b2:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.533998:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084eac00/0xb3f83dad1b7e2f74 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2b2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aaeb8 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.534010:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2b1:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.534022:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084ea200/0xb3f83dad1b7e2f66 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2b1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aae87 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.534035:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2b0:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.534045:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084eb600/0xb3f83dad1b7e2f58 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2b0:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aae56 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.534059:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2af:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.534069:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084ebe00/0xb3f83dad1b7e2f4a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2af:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aae25 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.534085:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2ae:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.534095:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084eb400/0xb3f83dad1b7e2f3c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2ae:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aadf4 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.534108:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2ad:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.534119:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084ea000/0xb3f83dad1b7e2f2e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2ad:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aadc3 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.534131:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2ac:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.534142:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608418200/0xb3f83dad1b7e2f20 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2ac:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aad92 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.534156:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2ab:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.534167:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084a8000/0xb3f83dad1b7e2f12 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2ab:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aad61 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.534180:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2aa:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.534192:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084b7400/0xb3f83dad1b7e2f04 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2aa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aad30 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.534205:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2a9:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.534217:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608419600/0xb3f83dad1b7e2ef6 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2a9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aacff expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.534230:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2a8:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.534242:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608419e00/0xb3f83dad1b7e2ee8 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2a8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aacce expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.534259:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2a7:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.534271:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608418e00/0xb3f83dad1b7e2eda lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2a7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aac9d expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.534284:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2a6:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.534294:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608419400/0xb3f83dad1b7e2ecc lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2a6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aac6c expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.534306:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2a5:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.534317:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608418000/0xb3f83dad1b7e2ebe lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2a5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aac3b expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.534330:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2a4:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.534343:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608419200/0xb3f83dad1b7e2eb0 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2a4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aac0a expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.534357:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2a3:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.534367:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608419a00/0xb3f83dad1b7e2ea2 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2a3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aabd9 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.534380:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2a2:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.534391:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608418600/0xb3f83dad1b7e2e94 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2a2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aaba8 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.534404:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2a1:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.534430:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608418a00/0xb3f83dad1b7e2e86 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2a1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aab77 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.534447:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x2a0:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.534459:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608418c00/0xb3f83dad1b7e2e78 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x2a0:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aab46 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.534473:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x29f:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.534484:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608419800/0xb3f83dad1b7e2e6a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x29f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aab15 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.534498:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x29e:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.534510:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608419c00/0xb3f83dad1b7e2e5c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x29e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aaae4 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.534523:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x29d:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.534534:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608419000/0xb3f83dad1b7e2e4e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x29d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aaab3 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.534546:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x29c:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.534557:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084fc200/0xb3f83dad1b7e2e40 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x29c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aaa82 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.534569:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x29b:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.534581:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084b7e00/0xb3f83dad1b7e2e32 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x29b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aaa51 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.534593:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x29a:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.534604:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084a8800/0xb3f83dad1b7e2e24 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x29a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aaa20 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.534622:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x299:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.534634:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084b7c00/0xb3f83dad1b7e2e16 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x299:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aa9ef expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.534648:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x298:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.534660:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084b6c00/0xb3f83dad1b7e2e08 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x298:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aa9be expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.534672:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x297:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.534684:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084fcc00/0xb3f83dad1b7e2dfa lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x297:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aa98d expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.534718:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x296:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.534730:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084b6600/0xb3f83dad1b7e2dec lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x296:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aa95c expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.534743:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x295:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.534755:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084b6e00/0xb3f83dad1b7e2dde lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x295:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aa92b expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.534768:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x294:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.534781:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084b7a00/0xb3f83dad1b7e2dd0 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x294:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aa8fa expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.534794:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x293:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.534806:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084b7000/0xb3f83dad1b7e2dc2 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x293:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aa8c9 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.534823:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x292:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.534836:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084b7600/0xb3f83dad1b7e2db4 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x292:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aa898 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.534852:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x291:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.534865:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084b6000/0xb3f83dad1b7e2da6 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x291:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aa867 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.534877:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x290:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.534888:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084b6800/0xb3f83dad1b7e2d98 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x290:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aa836 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.534900:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x28f:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.534912:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084b6a00/0xb3f83dad1b7e2d8a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x28f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aa805 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.534925:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x28e:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.534935:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084b6200/0xb3f83dad1b7e2d7c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x28e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aa7d4 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.534947:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x28d:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.534958:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084b6400/0xb3f83dad1b7e2d6e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x28d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aa7a3 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.534972:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x28c:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.534982:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084b7200/0xb3f83dad1b7e2d60 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x28c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aa772 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.535001:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x28b:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.535014:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084b7800/0xb3f83dad1b7e2d52 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x28b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aa741 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.535026:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x28a:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.535037:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084a9400/0xb3f83dad1b7e2d44 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x28a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aa710 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.535050:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x289:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.535061:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084fd200/0xb3f83dad1b7e2d36 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x289:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aa6df expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.535073:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x288:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.535083:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608483800/0xb3f83dad1b7e2d28 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x288:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aa6ae expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.535096:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x287:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.535106:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084a8a00/0xb3f83dad1b7e2d1a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x287:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aa67d expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.535120:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x286:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.535131:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084a9200/0xb3f83dad1b7e2d0c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x286:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aa64c expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.535145:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x285:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.535157:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084a9a00/0xb3f83dad1b7e2cfe lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x285:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aa61b expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.535176:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x284:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.535186:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084a9800/0xb3f83dad1b7e2cf0 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x284:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aa5ea expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.535200:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x283:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.535212:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084a9c00/0xb3f83dad1b7e2ce2 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x283:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aa5b9 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.535225:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x282:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.535235:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084a8c00/0xb3f83dad1b7e2cd4 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x282:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aa588 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.535248:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x281:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.535258:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084a9000/0xb3f83dad1b7e2cc6 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x281:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aa557 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.535271:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x280:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.535282:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084a8600/0xb3f83dad1b7e2cb8 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x280:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aa526 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.535298:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x27f:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.535309:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084a9600/0xb3f83dad1b7e2caa lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x27f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aa4f5 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.535322:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x27e:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.535333:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084a8e00/0xb3f83dad1b7e2c9c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x27e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aa4c4 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.535349:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x27d:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.535359:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084a8200/0xb3f83dad1b7e2c8e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x27d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aa493 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.535372:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x27c:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.535383:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084a8400/0xb3f83dad1b7e2c80 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x27c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aa462 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.535396:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x27b:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.535420:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608464a00/0xb3f83dad1b7e2c72 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x27b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aa431 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.535435:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x27a:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.535448:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084fd600/0xb3f83dad1b7e2c64 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x27a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aa400 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.535460:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x279:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.535471:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084fc600/0xb3f83dad1b7e2c56 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x279:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aa3cf expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.535484:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x278:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.535494:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608482a00/0xb3f83dad1b7e2c48 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x278:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aa39e expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.535508:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x277:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.535519:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608483600/0xb3f83dad1b7e2c3a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x277:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aa36d expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.535536:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x276:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.535547:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084fdc00/0xb3f83dad1b7e2c2c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x276:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aa33c expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.535561:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x275:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.535572:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084fd000/0xb3f83dad1b7e2c1e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x275:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aa30b expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.535585:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x274:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.535597:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084fd800/0xb3f83dad1b7e2c10 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x274:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aa2da expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.535611:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x273:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.535622:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084fc400/0xb3f83dad1b7e2c02 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x273:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aa2a9 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.535635:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x272:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.535647:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084fde00/0xb3f83dad1b7e2bf4 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x272:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aa278 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.535660:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x271:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.535671:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084fce00/0xb3f83dad1b7e2be6 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x271:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aa247 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.535684:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x270:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.535717:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084fd400/0xb3f83dad1b7e2bd8 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x270:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aa216 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.535737:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x26f:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.535750:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084fc000/0xb3f83dad1b7e2bca lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x26f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aa1e5 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.535770:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x26e:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.535784:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084fc800/0xb3f83dad1b7e2bbc lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x26e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aa1b4 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.535797:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x26d:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.535810:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084fda00/0xb3f83dad1b7e2bae lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x26d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aa183 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.535822:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x26c:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.535833:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084fca00/0xb3f83dad1b7e2ba0 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x26c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aa152 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.535848:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x26b:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.535858:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608482400/0xb3f83dad1b7e2b92 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x26b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aa121 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.535871:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x26a:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.535882:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608483200/0xb3f83dad1b7e2b84 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x26a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aa0f0 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.535896:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x269:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.535907:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608464000/0xb3f83dad1b7e2b76 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x269:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aa0bf expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.535924:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x268:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.535935:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608465a00/0xb3f83dad1b7e2b68 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x268:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aa08e expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.535964:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x267:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.535977:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560845e000/0xb3f83dad1b7e2b5a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x267:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aa05d expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.535991:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x266:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.536003:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608483a00/0xb3f83dad1b7e2b4c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x266:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91aa02c expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.536016:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x265:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.536028:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608482e00/0xb3f83dad1b7e2b3e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x265:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a9ffb expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.536040:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x264:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.536050:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608483c00/0xb3f83dad1b7e2b30 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x264:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a9fca expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.536062:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x263:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.536073:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608482c00/0xb3f83dad1b7e2b22 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x263:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a9f99 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.536086:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x262:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.536097:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608483000/0xb3f83dad1b7e2b14 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x262:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a9f68 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.536116:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x261:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.536128:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608482600/0xb3f83dad1b7e2b06 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x261:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a9f37 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.536143:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x260:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.536154:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608482200/0xb3f83dad1b7e2af8 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x260:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a9f06 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.536167:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x25f:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.536178:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608483e00/0xb3f83dad1b7e2aea lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x25f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a9ed5 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.536190:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x25e:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.536201:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608483400/0xb3f83dad1b7e2adc lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x25e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a9ea4 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.536215:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x25d:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.536227:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608482000/0xb3f83dad1b7e2ace lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x25d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a9e73 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.536240:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x25c:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.536251:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608482800/0xb3f83dad1b7e2ac0 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x25c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a9e42 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.536264:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x25b:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.536276:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608465e00/0xb3f83dad1b7e2ab2 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x25b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a9e11 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.536294:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x25a:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.536306:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560845ee00/0xb3f83dad1b7e2aa4 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x25a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a9de0 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.536318:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x259:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.536330:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560845fa00/0xb3f83dad1b7e2a96 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x259:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a9daf expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.536342:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x258:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.536353:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560842a400/0xb3f83dad1b7e2a88 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x258:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a9d7e expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.536366:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x257:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.536377:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084bcc00/0xb3f83dad1b7e2a7a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x257:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a9d4d expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.536392:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x256:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.536404:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608465c00/0xb3f83dad1b7e2a6c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x256:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a9d1c expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.536432:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x255:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.536446:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608464c00/0xb3f83dad1b7e2a5e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x255:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a9ceb expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.536459:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x254:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.536469:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608465000/0xb3f83dad1b7e2a50 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x254:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a9cba expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.536486:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x253:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.536497:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608465400/0xb3f83dad1b7e2a42 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x253:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a9c89 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.536516:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x252:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.536528:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608464800/0xb3f83dad1b7e2a34 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x252:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a9c58 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.536542:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x251:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.536554:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608464600/0xb3f83dad1b7e2a26 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x251:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a9c27 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.536567:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x250:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.536579:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608465600/0xb3f83dad1b7e2a18 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x250:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a9bf6 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.536593:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x24f:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.536604:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608464e00/0xb3f83dad1b7e2a0a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x24f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a9bc5 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.536619:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x24e:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.536629:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608464200/0xb3f83dad1b7e29fc lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x24e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a9b94 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.536642:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x24d:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.536653:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608464400/0xb3f83dad1b7e29ee lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x24d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a9b63 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.536672:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x24c:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.536683:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608465200/0xb3f83dad1b7e29e0 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x24c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a9b32 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.536715:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x24b:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.536728:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608465800/0xb3f83dad1b7e29d2 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x24b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a9b01 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.536743:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x24a:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.536756:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084bdc00/0xb3f83dad1b7e29c4 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x24a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a9ad0 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.536771:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x249:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.536783:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560845f600/0xb3f83dad1b7e29b6 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x249:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a9a9f expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.536796:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x248:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.536807:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f8ca00/0xb3f83dad1b7e29a8 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x248:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a9a6e expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.536820:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x247:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.536831:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560845f400/0xb3f83dad1b7e299a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x247:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a9a3d expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.536845:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x246:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.536856:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560845f000/0xb3f83dad1b7e298c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x246:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a9a0c expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.536874:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x245:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.536886:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560845e200/0xb3f83dad1b7e297e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x245:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a99db expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.536900:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x244:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.536912:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560845e400/0xb3f83dad1b7e2970 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x244:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a99aa expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.536927:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x243:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.536939:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560845e800/0xb3f83dad1b7e2962 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x243:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a9979 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.536953:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x242:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.536966:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560845f200/0xb3f83dad1b7e2954 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x242:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a9948 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.536983:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x241:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.536995:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560845f800/0xb3f83dad1b7e2946 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x241:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a9917 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.537010:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x240:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.537023:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560845ea00/0xb3f83dad1b7e2938 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x240:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a98e6 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.537037:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x23f:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.537049:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560845fc00/0xb3f83dad1b7e292a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x23f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a98b5 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.537068:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x23e:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.537080:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560845ec00/0xb3f83dad1b7e291c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x23e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a9884 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.537094:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x23d:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.537104:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560845e600/0xb3f83dad1b7e290e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x23d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a9853 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.537117:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x23c:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.537127:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560845fe00/0xb3f83dad1b7e2900 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x23c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a9822 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.537140:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x23b:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.537150:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560842bc00/0xb3f83dad1b7e28f2 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x23b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a97f1 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.537163:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x23a:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.537174:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084bc400/0xb3f83dad1b7e28e4 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x23a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a97c0 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.537186:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x239:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.537196:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607e5f600/0xb3f83dad1b7e28d6 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x239:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a978f expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.537209:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x238:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.537221:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084bd200/0xb3f83dad1b7e28c8 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x238:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a975e expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.537239:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x237:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.537250:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084bd400/0xb3f83dad1b7e28ba lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x237:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a972d expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.537263:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x236:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.537273:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084bd000/0xb3f83dad1b7e28ac lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x236:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a96fc expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.537287:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x235:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.537298:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084bc600/0xb3f83dad1b7e289e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x235:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a96cb expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.537311:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x234:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.537323:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084bc000/0xb3f83dad1b7e2890 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x234:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a969a expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.537337:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x233:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.537348:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084bd600/0xb3f83dad1b7e2882 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x233:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a9669 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.537359:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x232:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.537367:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084bde00/0xb3f83dad1b7e2874 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x232:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a9638 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.537379:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x231:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.537388:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084bce00/0xb3f83dad1b7e2866 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x231:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a9607 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.537419:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x230:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.537433:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084bc200/0xb3f83dad1b7e2858 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x230:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a95d6 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.537448:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x22f:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.537460:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084bd800/0xb3f83dad1b7e284a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x22f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a95a5 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.537473:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x22e:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.537484:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084bc800/0xb3f83dad1b7e283c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x22e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a9574 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.537498:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x22d:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.537508:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084bda00/0xb3f83dad1b7e282e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x22d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a9543 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.537520:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x22c:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.537530:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084bca00/0xb3f83dad1b7e2820 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x22c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a9512 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.537543:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x22b:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.537555:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f8ce00/0xb3f83dad1b7e2812 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x22b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a94e1 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.537567:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x22a:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.537577:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560842a000/0xb3f83dad1b7e2804 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x22a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a94b0 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.537593:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x229:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.537603:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560842ae00/0xb3f83dad1b7e27f6 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x229:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a947f expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.537617:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x228:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.537627:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560842b400/0xb3f83dad1b7e27e8 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x228:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a944e expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.537640:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x227:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.537650:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560842a200/0xb3f83dad1b7e27da lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x227:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a941d expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.537663:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x226:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.537676:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560842b000/0xb3f83dad1b7e27cc lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x226:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a93ec expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.537708:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x225:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.537723:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560842b200/0xb3f83dad1b7e27be lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x225:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a93bb expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.537738:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x224:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.537752:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560842b600/0xb3f83dad1b7e27b0 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x224:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a938a expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.537765:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x223:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.537777:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560842a800/0xb3f83dad1b7e27a2 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x223:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a9359 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.537796:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x222:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.537806:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560842ba00/0xb3f83dad1b7e2794 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x222:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a9328 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.537820:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x221:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.537830:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560842b800/0xb3f83dad1b7e2786 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x221:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a92f7 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.537842:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x220:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.537852:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560842aa00/0xb3f83dad1b7e2778 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x220:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a92c6 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.537864:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x21f:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.537874:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560842ac00/0xb3f83dad1b7e276a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x21f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a9295 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.537886:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x21e:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.537896:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560842a600/0xb3f83dad1b7e275c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x21e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a9264 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.537909:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x21d:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.537920:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560842be00/0xb3f83dad1b7e274e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x21d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a9233 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.537933:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x21c:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.537944:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608453c00/0xb3f83dad1b7e2740 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x21c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a9202 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.537964:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x21b:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.537975:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f8c800/0xb3f83dad1b7e2732 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x21b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a91d1 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.537989:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x21a:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.538001:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f8c400/0xb3f83dad1b7e2724 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x21a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a91a0 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.538014:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x219:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.538025:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f8d400/0xb3f83dad1b7e2716 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x219:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a916f expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.538039:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x218:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.538050:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f8c200/0xb3f83dad1b7e2708 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x218:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a913e expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.538064:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x217:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.538075:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f8c000/0xb3f83dad1b7e26fa lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x217:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a910d expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.538088:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x216:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.538100:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f8c600/0xb3f83dad1b7e26ec lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x216:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a90dc expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.538112:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x215:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.538123:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f8da00/0xb3f83dad1b7e26de lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x215:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a90ab expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.538141:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x214:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.538153:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f8cc00/0xb3f83dad1b7e26d0 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x214:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a907a expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.538166:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x213:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.538178:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f8de00/0xb3f83dad1b7e26c2 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x213:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a9049 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.538191:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x212:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.538202:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f8d200/0xb3f83dad1b7e26b4 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x212:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a9018 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.538216:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x211:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.538228:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f8d800/0xb3f83dad1b7e26a6 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x211:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a8fe7 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.538242:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x210:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.538253:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f8dc00/0xb3f83dad1b7e2698 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x210:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a8fb6 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.538266:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x20f:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.538275:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f8d000/0xb3f83dad1b7e268a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x20f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a8f85 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.538288:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x20e:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.538297:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f8d600/0xb3f83dad1b7e267c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x20e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a8f54 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.538315:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x20d:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.538324:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608452c00/0xb3f83dad1b7e266e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x20d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a8f23 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.538337:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x20c:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.538347:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607e5ee00/0xb3f83dad1b7e2660 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x20c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a8ef2 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.538359:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x20b:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.538369:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607e5e200/0xb3f83dad1b7e2652 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x20b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a8ec1 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.538382:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x20a:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.538391:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608452200/0xb3f83dad1b7e2644 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x20a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a8e90 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.538404:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x209:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.538429:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607e5e400/0xb3f83dad1b7e2636 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x209:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a8e5f expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.538445:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x208:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.538456:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607e5f800/0xb3f83dad1b7e2628 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x208:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a8e2e expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.538471:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x207:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.538481:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f81e00/0xb3f83dad1b7e261a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x207:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a8dfd expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.538499:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x206:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.538511:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607e5fe00/0xb3f83dad1b7e260c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x206:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a8dcc expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.538525:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x205:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.538535:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607e5fc00/0xb3f83dad1b7e25fe lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x205:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a8d9b expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.538549:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x204:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.538560:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607e5f400/0xb3f83dad1b7e25f0 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x204:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a8d6a expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.538574:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x203:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.538585:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607e5e000/0xb3f83dad1b7e25e2 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x203:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a8d39 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.538599:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x202:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.538612:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607e5f200/0xb3f83dad1b7e25d4 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x202:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a8d08 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.538624:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x201:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.538634:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607e5e800/0xb3f83dad1b7e25c6 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x201:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a8cd7 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.538647:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x200:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.538657:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607e5f000/0xb3f83dad1b7e25b8 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x200:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a8ca6 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.538674:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1ff:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.538685:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607e5fa00/0xb3f83dad1b7e25aa lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1ff:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a8c75 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.538745:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1fe:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.538761:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607e5ea00/0xb3f83dad1b7e259c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1fe:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a8c44 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.538777:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1fd:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.538788:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607e5ec00/0xb3f83dad1b7e258e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1fd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a8c13 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.538801:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1fc:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.538813:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607e5e600/0xb3f83dad1b7e2580 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1fc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a8be2 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.538830:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1fb:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.538842:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608452400/0xb3f83dad1b7e2572 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a8bb1 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.538855:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1fa:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.538865:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608453800/0xb3f83dad1b7e2564 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1fa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a8b80 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.538878:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1f9:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.538887:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608452600/0xb3f83dad1b7e2556 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1f9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a8b4f expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.538904:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1f8:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.538914:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f73000/0xb3f83dad1b7e2548 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1f8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a8b1e expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.538927:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1f7:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.538937:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607e45c00/0xb3f83dad1b7e253a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1f7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a8aed expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.538949:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1f6:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.538959:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608452a00/0xb3f83dad1b7e252c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1f6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a8abc expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.538971:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1f5:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.538982:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608453200/0xb3f83dad1b7e251e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1f5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a8a8b expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.538996:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1f4:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.539006:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608453000/0xb3f83dad1b7e2510 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1f4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a8a5a expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.539019:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1f3:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.539029:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608453600/0xb3f83dad1b7e2502 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1f3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a8a29 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.539043:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1f2:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.539052:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608453e00/0xb3f83dad1b7e24f4 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1f2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a89f8 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.539070:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1f1:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.539080:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608452e00/0xb3f83dad1b7e24e6 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1f1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a89c7 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.539093:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1f0:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.539103:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608453400/0xb3f83dad1b7e24d8 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1f0:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a8996 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.539117:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1ef:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.539127:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608452000/0xb3f83dad1b7e24ca lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1ef:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a8965 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.539139:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1ee:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.539148:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608452800/0xb3f83dad1b7e24bc lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1ee:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a8934 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.539160:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1ed:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.539171:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608453a00/0xb3f83dad1b7e24ae lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1ed:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a8903 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.539184:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1ec:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.539194:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607e44c00/0xb3f83dad1b7e24a0 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1ec:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a88d2 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.539207:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1eb:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.539217:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f72800/0xb3f83dad1b7e2492 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1eb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a88a1 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.539235:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1ea:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.539246:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607e45600/0xb3f83dad1b7e2484 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1ea:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a8870 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.539258:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1e9:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.539269:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607e44600/0xb3f83dad1b7e2476 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1e9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a883f expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.539281:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1e8:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.539291:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607e45e00/0xb3f83dad1b7e2468 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1e8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a880e expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.539303:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1e7:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.539314:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607e45000/0xb3f83dad1b7e245a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1e7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a87dd expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.539326:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1e6:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.539337:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607e45400/0xb3f83dad1b7e244c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1e6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a87ac expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.539351:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1e5:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.539381:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607e45200/0xb3f83dad1b7e243e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1e5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a877b expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.539398:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1e4:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.539425:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607e45800/0xb3f83dad1b7e2430 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1e4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a874a expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.539445:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1e3:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.539457:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607e44e00/0xb3f83dad1b7e2422 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1e3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a8719 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.539472:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1e2:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.539498:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607e44400/0xb3f83dad1b7e2414 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1e2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a86e8 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.539514:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1e1:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.539526:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607e44200/0xb3f83dad1b7e2406 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1e1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a86b7 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.539540:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1e0:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.539551:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607e44000/0xb3f83dad1b7e23f8 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1e0:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a8686 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.539565:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1df:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.539575:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607e44800/0xb3f83dad1b7e23ea lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1df:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a8655 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.539589:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1de:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.539601:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607e45a00/0xb3f83dad1b7e23dc lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1de:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a8624 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.539614:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1dd:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.539628:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607e44a00/0xb3f83dad1b7e23ce lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1dd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a85f3 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.539647:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1dc:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.539659:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f81800/0xb3f83dad1b7e23c0 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1dc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a85c2 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.539673:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1db:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.539685:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f73800/0xb3f83dad1b7e23b2 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1db:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a8591 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.539724:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1da:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.539741:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f72a00/0xb3f83dad1b7e23a4 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1da:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a8560 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.539757:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1d9:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.539774:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f81400/0xb3f83dad1b7e2396 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1d9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a852f expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.539789:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1d8:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.539801:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f81600/0xb3f83dad1b7e2388 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1d8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a84fe expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.539815:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1d7:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.539827:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f73400/0xb3f83dad1b7e237a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1d7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a84cd expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.539840:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1d6:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.539851:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57c00/0xb3f83dad1b7e236c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1d6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a849c expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.539869:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1d5:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.539881:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f73200/0xb3f83dad1b7e235e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1d5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a846b expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.539893:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1d4:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.539905:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f73a00/0xb3f83dad1b7e2350 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1d4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a843a expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.539918:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1d3:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.539929:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f72600/0xb3f83dad1b7e2342 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1d3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a8409 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.539942:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1d2:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.539954:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f73c00/0xb3f83dad1b7e2334 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1d2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a83d8 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.539968:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1d1:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.539979:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f72c00/0xb3f83dad1b7e2326 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1d1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a83a7 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.539993:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1d0:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.540004:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f73e00/0xb3f83dad1b7e2318 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1d0:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a8376 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.540017:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1cf:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.540027:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f72000/0xb3f83dad1b7e230a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1cf:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a8345 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.540045:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1ce:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.540055:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f73600/0xb3f83dad1b7e22fc lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1ce:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a8314 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.540068:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1cd:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.540080:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f72e00/0xb3f83dad1b7e22ee lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1cd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a82e3 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.540092:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1cc:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.540104:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f72200/0xb3f83dad1b7e22e0 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1cc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a82b2 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.540135:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1cb:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.540148:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f72400/0xb3f83dad1b7e22d2 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1cb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a8281 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.540163:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1ca:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.540174:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57400/0xb3f83dad1b7e22c4 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1ca:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a8250 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.540188:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1c9:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.540200:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605571800/0xb3f83dad1b7e22b6 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1c9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a821f expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.540213:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1c8:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.540223:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f80200/0xb3f83dad1b7e22a8 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1c8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a81ee expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.540240:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1c7:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.540249:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f80e00/0xb3f83dad1b7e229a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1c7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a81bd expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.540262:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1c6:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.540271:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f80000/0xb3f83dad1b7e228c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1c6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a818c expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.540283:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1c5:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.540292:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f80800/0xb3f83dad1b7e227e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1c5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a815b expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.540303:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1c4:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.540313:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f81000/0xb3f83dad1b7e2270 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1c4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a812a expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.540326:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1c3:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.540335:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f80400/0xb3f83dad1b7e2262 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1c3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a80f9 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.540347:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1c2:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.540355:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f81200/0xb3f83dad1b7e2254 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1c2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a80c8 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.540368:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1c1:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.540377:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f81c00/0xb3f83dad1b7e2246 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1c1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a8097 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.540393:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1c0:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.540403:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f81a00/0xb3f83dad1b7e2238 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1c0:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a8066 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.540430:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1bf:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.540440:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f80a00/0xb3f83dad1b7e222a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1bf:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a8035 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.540453:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1be:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.540462:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f80c00/0xb3f83dad1b7e221c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1be:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a8004 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.540475:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1bd:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.540484:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f80600/0xb3f83dad1b7e220e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1bd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a7fd3 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.540497:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1bc:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.540507:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57a00/0xb3f83dad1b7e2200 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1bc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a7fa2 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.540519:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1bb:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.540529:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607d56800/0xb3f83dad1b7e21f2 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1bb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a7f71 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.540541:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1ba:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.540553:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57000/0xb3f83dad1b7e21e4 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1ba:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a7f40 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.540570:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1b9:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.540580:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605571c00/0xb3f83dad1b7e21d6 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1b9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a7f0f expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.540592:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1b8:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.540602:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605570e00/0xb3f83dad1b7e21c8 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1b8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a7ede expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.540614:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1b7:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.540623:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607d51600/0xb3f83dad1b7e21ba lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1b7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a7ead expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.540637:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1b6:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.540647:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f56a00/0xb3f83dad1b7e21ac lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1b6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a7e7c expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.540659:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1b5:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.540669:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f56c00/0xb3f83dad1b7e219e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1b5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a7e4b expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.540684:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1b4:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.540715:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f56800/0xb3f83dad1b7e2190 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1b4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a7e1a expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.540730:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1b3:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.540740:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57e00/0xb3f83dad1b7e2182 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1b3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a7de9 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.540758:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1b2:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.540783:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4200/0xb3f83dad1b7e2174 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1b2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a7db8 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.540799:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1b1:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.540810:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5e00/0xb3f83dad1b7e2166 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1b1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a7d87 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.540823:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1b0:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.540833:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f56000/0xb3f83dad1b7e2158 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1b0:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a7d56 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.540845:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1af:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.540854:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f56600/0xb3f83dad1b7e214a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1af:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a7d25 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.540867:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1ae:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.540876:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57600/0xb3f83dad1b7e213c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1ae:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a7cf4 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.540889:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1ad:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.540900:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f56e00/0xb3f83dad1b7e212e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1ad:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a7cc3 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.540913:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1ac:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.540923:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f56200/0xb3f83dad1b7e2120 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1ac:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a7c92 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.540940:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1ab:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.540950:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f56400/0xb3f83dad1b7e2112 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1ab:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a7c61 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.540962:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1aa:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.540973:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57200/0xb3f83dad1b7e2104 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1aa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a7c30 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.540986:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1a9:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.540996:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57800/0xb3f83dad1b7e20f6 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1a9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a7bff expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.541009:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1a8:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.541020:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605571200/0xb3f83dad1b7e20e8 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1a8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a7bce expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.541033:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1a7:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.541043:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605689200/0xb3f83dad1b7e20da lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1a7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a7b9d expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.541058:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1a6:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.541070:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605571400/0xb3f83dad1b7e20cc lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1a6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a7b6c expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.541083:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1a5:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.541095:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605570200/0xb3f83dad1b7e20be lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1a5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a7b3b expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.541113:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1a4:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.541125:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605571000/0xb3f83dad1b7e20b0 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1a4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a7b0a expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.541139:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1a3:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.541150:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605570000/0xb3f83dad1b7e20a2 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1a3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a7ad9 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.541162:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1a2:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.541173:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605570400/0xb3f83dad1b7e2094 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1a2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a7aa8 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.541185:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1a1:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.541197:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605570800/0xb3f83dad1b7e2086 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1a1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a7a77 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.541210:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1a0:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.541222:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605571600/0xb3f83dad1b7e2078 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1a0:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a7a46 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.541236:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x19f:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.541246:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605571a00/0xb3f83dad1b7e206a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x19f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a7a15 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.541258:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x19e:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.541268:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605570a00/0xb3f83dad1b7e205c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x19e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a79e4 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.541286:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x19d:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.541296:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605570c00/0xb3f83dad1b7e204e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x19d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a79b3 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.541309:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x19c:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.541319:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605570600/0xb3f83dad1b7e2040 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x19c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a7982 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.541331:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x19b:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.541341:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605571e00/0xb3f83dad1b7e2032 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x19b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a7951 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.541354:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x19a:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.541363:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607d57800/0xb3f83dad1b7e2024 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x19a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a7920 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.541376:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x199:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.541387:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607d51000/0xb3f83dad1b7e2016 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x199:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a78ef expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.541430:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x198:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.541443:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4600/0xb3f83dad1b7e2008 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x198:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a78be expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.541457:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x197:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.541468:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607d56600/0xb3f83dad1b7e1ffa lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x197:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a788d expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.541485:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x196:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.541498:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607d57400/0xb3f83dad1b7e1fec lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x196:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a785c expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.541511:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x195:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.541522:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607d51200/0xb3f83dad1b7e1fde lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x195:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a782b expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.541535:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x194:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.541546:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607d1de00/0xb3f83dad1b7e1fd0 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x194:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a77fa expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.541560:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x193:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.541573:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607d57a00/0xb3f83dad1b7e1fc2 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x193:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a77c9 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.541586:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x192:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.541597:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607d57c00/0xb3f83dad1b7e1fb4 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x192:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a7798 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826939.541611:0:84122:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x191:0x0]: losing layout lock 00010000:00010000:0.0:1787826939.541621:0:84122:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607d56a00/0xb3f83dad1b7e1fa6 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x191:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91a7767 expref: -99 pid: 83443 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826939.541632:0:84122:0:(ldlm_request.c:2901:ldlm_cancel_unused_locks_for_replay()) Canceled 0 unused locks from namespace lustre-MDT0000-mdc-ffff985602bc9800 00000100:00080000:0.0:1787826939.542101:0:84122:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from REPLAY_LOCKS to REPLAY_WAIT 00000100:00100000:3.0:1787826939.542220:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856087730c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669226267392:192.168.203.114@tcp:400:ldlm_lock_repla.0 00000100:00100000:3.0:1787826939.545130:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 210453399506 gen 9 00000100:00080000:3.0:1787826939.545156:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from REPLAY_WAIT to RECOVER 00000100:00080000:3.0:1787826939.545169:0:2382:0:(client.c:3247:ptlrpc_resend_req()) @@@ going to resend req@ffff9856087730c0 x1874669226267392/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787826955 ref 1 fl Interpret:RQU/2c0/0 rc 0/0 job:'ldlm_lock_repla.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826939.545181:0:2382:0:(client.c:3256:ptlrpc_resend_req()) @@@ it has reply, so skip it req@ffff9856087730c0 x1874669226267392/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787826955 ref 1 fl Interpret:RQU/2c0/0 rc 0/0 job:'ldlm_lock_repla.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826939.545188:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from RECOVER to FULL 00000100:02000000:3.0:1787826939.545432:0:2382:0:(import.c:1743:ptlrpc_import_recovery_state_machine()) lustre-MDT0000-mdc-ffff985602bc9800: Connection restored to 192.168.203.114@tcp (at 192.168.203.114@tcp) 00000100:00080000:3.0:1787826939.545439:0:2382:0:(recover.c:191:ptlrpc_wake_delayed()) @@@ waking (set 000000004dcc3727): req@ffff985602ee0780 x1874669226266496/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 576/1152 e 0 to 0 dl 0 ref 2 fl Rpc:WQU/600/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:0 00000100:00100000:3.0:1787826939.545461:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856087730c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669226267392:192.168.203.114@tcp:400:ldlm_lock_repla.0 00000100:00100000:2.0:1787826939.545651:0:84106:0:(client.c:2715:ptlrpc_set_wait()) set ffff985608840900 going to sleep for 11 seconds 00000100:00100000:1.0:1787826939.547091:0:84106:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee0780 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:84106:1874669226266496:192.168.203.114@tcp:101:lfs.0 00010000:00010000:1.0:1787826939.547114:0:84106:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5400/0xb3f83dad1b7e4084 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91b1312 expref: -99 pid: 84106 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826939.547127:0:84106:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5400/0xb3f83dad1b7e4084 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91b1312 expref: -99 pid: 84106 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826939.547138:0:84106:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5400/0xb3f83dad1b7e4084 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91b1312 expref: -99 pid: 84106 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826939.547148:0:84106:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5400/0xb3f83dad1b7e4084 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91b1312 expref: -99 pid: 84106 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826939.547158:0:84106:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5400/0xb3f83dad1b7e4084 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91b1312 expref: -99 pid: 84106 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826939.547164:0:84106:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5400/0xb3f83dad1b7e4084 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91b1312 expref: -99 pid: 84106 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826939.547169:0:84106:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5400/0xb3f83dad1b7e4084 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91b1312 expref: -99 pid: 84106 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826939.547179:0:84106:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff985602ee0780 x1874669226266496/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826955 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826939.547194:0:84106:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5400/0xb3f83dad1b7e4084 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91b1312 expref: -99 pid: 84106 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826939.547211:0:84106:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:1.0:1787826939.547220:0:84106:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:1.0:1787826939.547248:0:84106:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 69632 00000080:00010000:1.0:1787826939.547258:0:84106:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e4084 00000080:00010000:1.0:1787826939.547265:0:84106:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826939.547269:0:84106:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e4084 from it ffffab3ac0d93cb8 00010000:00010000:1.0:1787826939.547275:0:84106:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5400/0xb3f83dad1b7e4084 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91b1312 expref: -99 pid: 84106 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826939.547285:0:84106:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5400/0xb3f83dad1b7e4084 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91b1312 expref: -99 pid: 84106 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:1.0:1787826939.548198:0:84106:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 0, flags 0, foreign 0 00000080:00200000:1.0:1787826939.548206:0:84106:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826939.548224:0:84106:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826939.548237:0:84106:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5400/0xb3f83dad1b7e4084 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91b1312 expref: -99 pid: 84106 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826939.548246:0:84106:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5400/0xb3f83dad1b7e4084 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91b1312 expref: -99 pid: 84106 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826939.548252:0:84106:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826939.548257:0:84106:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826939.548261:0:84106:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e4084 00000080:00010000:1.0:1787826939.548266:0:84106:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e4084 from it ffffab3ac0d93cb8 00010000:00010000:1.0:1787826939.548270:0:84106:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5400/0xb3f83dad1b7e4084 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91b1312 expref: -99 pid: 84106 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826939.548279:0:84106:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5400/0xb3f83dad1b7e4084 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91b1312 expref: -99 pid: 84106 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826939.548403:0:84106:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826939.548429:0:84106:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826939.548440:0:84106:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5400/0xb3f83dad1b7e4084 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91b1312 expref: -99 pid: 84106 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826939.548450:0:84106:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5400/0xb3f83dad1b7e4084 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91b1312 expref: -99 pid: 84106 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826939.548456:0:84106:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826939.548465:0:84106:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826939.548468:0:84106:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e4084 00000080:00010000:1.0:1787826939.548474:0:84106:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e4084 from it ffffab3ac0d93bb0 00010000:00010000:1.0:1787826939.548477:0:84106:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5400/0xb3f83dad1b7e4084 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91b1312 expref: -99 pid: 84106 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826939.548487:0:84106:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5400/0xb3f83dad1b7e4084 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91b1312 expref: -99 pid: 84106 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787826939.548500:0:84106:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 44 00000080:00200008:1.0:1787826939.548510:0:84106:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 100000 00000080:00200080:1.0:1787826939.548605:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00800000:00000080:1.0:1787826939.548620:0:84106:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00800000:00000004:1.0:1787826939.548627:0:84106:0:(obd_class.h:1091:obd_statfs()) lustre-MDT0000-mdc-ffff985602bc9800: age 3067, max_age 3089 00000100:00100000:1.0:1787826939.548668:0:84106:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856086630c0 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:84106:1874669226267520:192.168.203.114@tcp:41:lfs.0 00000100:00100000:1.0:1787826939.548772:0:84106:0:(client.c:2715:ptlrpc_set_wait()) set ffff9856084cfd80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826939.550385:0:84106:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856086630c0 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:84106:1874669226267520:192.168.203.114@tcp:41:lfs.0 00800000:00000004:1.0:1787826939.550424:0:84106:0:(obd_class.h:1117:obd_statfs()) lustre-MDT0000-mdc-ffff985602bc9800: update ffff985606a337f0 cache blocks 317321/353529 objects 1023286/1024000 00000080:00200080:1.0:1787826939.550484:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00800000:00000080:1.0:1787826939.550496:0:84106:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00800000:00000004:1.0:1787826939.550503:0:84106:0:(obd_class.h:1091:obd_statfs()) lustre-MDT0001-mdc-ffff985602bc9800: age 3067, max_age 3089 00000100:00100000:1.0:1787826939.550528:0:84106:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856086630c0 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:84106:1874669226267648:192.168.203.114@tcp:41:lfs.0 00000100:00100000:1.0:1787826939.550596:0:84106:0:(client.c:2715:ptlrpc_set_wait()) set ffff9856084cfd80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826939.551520:0:84106:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856086630c0 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:84106:1874669226267648:192.168.203.114@tcp:41:lfs.0 00800000:00000004:1.0:1787826939.551551:0:84106:0:(obd_class.h:1117:obd_statfs()) lustre-MDT0001-mdc-ffff985602bc9800: update ffff98560952b7f0 cache blocks 317401/353529 objects 1023737/1024000 00000080:00200080:1.0:1787826939.551610:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00800000:00000080:1.0:1787826939.551626:0:84106:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.551634:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00800000:00000080:1.0:1787826939.551639:0:84106:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.551643:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00800000:00000080:1.0:1787826939.551647:0:84106:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.551650:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00800000:00000080:1.0:1787826939.551654:0:84106:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.551658:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00800000:00000080:1.0:1787826939.551661:0:84106:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.551665:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00800000:00000080:1.0:1787826939.551669:0:84106:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.551673:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00800000:00000080:1.0:1787826939.551677:0:84106:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.551680:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00800000:00000080:1.0:1787826939.551682:0:84106:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.551686:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00800000:00000080:1.0:1787826939.551688:0:84106:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.551706:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00800000:00000080:1.0:1787826939.551712:0:84106:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.551716:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00800000:00000080:1.0:1787826939.551720:0:84106:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.551725:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00800000:00000080:1.0:1787826939.551728:0:84106:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.551732:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00800000:00000080:1.0:1787826939.551735:0:84106:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.551739:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00800000:00000080:1.0:1787826939.551742:0:84106:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.551746:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00800000:00000080:1.0:1787826939.551750:0:84106:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.551754:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00800000:00000080:1.0:1787826939.551757:0:84106:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.551761:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00800000:00000080:1.0:1787826939.551764:0:84106:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.551773:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00800000:00000080:1.0:1787826939.551777:0:84106:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.551780:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00800000:00000080:1.0:1787826939.551784:0:84106:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.551787:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00800000:00000080:1.0:1787826939.551790:0:84106:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.551794:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00800000:00000080:1.0:1787826939.551798:0:84106:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.551803:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00800000:00000080:1.0:1787826939.551806:0:84106:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.551810:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00800000:00000080:1.0:1787826939.551814:0:84106:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.551819:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00800000:00000080:1.0:1787826939.551823:0:84106:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.551828:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00800000:00000080:1.0:1787826939.551832:0:84106:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.551836:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00800000:00000080:1.0:1787826939.551840:0:84106:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.551844:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00800000:00000080:1.0:1787826939.551861:0:84106:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.551867:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00800000:00000080:1.0:1787826939.551872:0:84106:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.551876:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00800000:00000080:1.0:1787826939.551881:0:84106:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.551886:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00800000:00000080:1.0:1787826939.551890:0:84106:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.551895:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00800000:00000080:1.0:1787826939.551899:0:84106:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.551904:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00800000:00000080:1.0:1787826939.551908:0:84106:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.551913:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00800000:00000080:1.0:1787826939.551917:0:84106:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.551922:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00800000:00000080:1.0:1787826939.551935:0:84106:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.551940:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00800000:00000080:1.0:1787826939.551946:0:84106:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.551950:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00800000:00000080:1.0:1787826939.551955:0:84106:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.551959:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00800000:00000080:1.0:1787826939.551962:0:84106:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.551968:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00800000:00000080:1.0:1787826939.551973:0:84106:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.551978:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00800000:00000080:1.0:1787826939.551990:0:84106:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.551998:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00800000:00000080:1.0:1787826939.552004:0:84106:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.552011:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00800000:00000080:1.0:1787826939.552016:0:84106:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.552021:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00800000:00000080:1.0:1787826939.552025:0:84106:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.552031:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00800000:00000080:1.0:1787826939.552035:0:84106:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.552041:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00800000:00000080:1.0:1787826939.552046:0:84106:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.552051:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00800000:00000080:1.0:1787826939.552064:0:84106:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.552070:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00800000:00000080:1.0:1787826939.552075:0:84106:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.552080:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00800000:00000080:1.0:1787826939.552084:0:84106:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.552088:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00800000:00000080:1.0:1787826939.552091:0:84106:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.552095:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00800000:00000080:1.0:1787826939.552099:0:84106:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.552103:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00800000:00000080:1.0:1787826939.552107:0:84106:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.552111:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00800000:00000080:1.0:1787826939.552116:0:84106:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.552119:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00800000:00000080:1.0:1787826939.552121:0:84106:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.552123:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00800000:00000080:1.0:1787826939.552125:0:84106:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.552127:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00800000:00000080:1.0:1787826939.552129:0:84106:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.552131:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00800000:00000080:1.0:1787826939.552132:0:84106:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.552135:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00800000:00000080:1.0:1787826939.552136:0:84106:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.552138:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00800000:00000080:1.0:1787826939.552140:0:84106:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.552142:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00800000:00000080:1.0:1787826939.552144:0:84106:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.552147:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00800000:00000080:1.0:1787826939.552149:0:84106:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.552151:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00800000:00000080:1.0:1787826939.552153:0:84106:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.552155:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00800000:00000080:1.0:1787826939.552157:0:84106:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.552159:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00800000:00000080:1.0:1787826939.552160:0:84106:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.552162:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00800000:00000080:1.0:1787826939.552164:0:84106:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.552166:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00020000:00000080:1.0:1787826939.552171:0:84106:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00020000:00000004:1.0:1787826939.552178:0:84106:0:(obd_class.h:1091:obd_statfs()) lustre-OST0000-osc-ffff985602bc9800: age 3067, max_age 3089 00000100:00100000:1.0:1787826939.552212:0:84106:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856086630c0 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:84106:1874669226267776:192.168.203.114@tcp:13:lfs.0 00000100:00100000:1.0:1787826939.552298:0:84106:0:(client.c:2715:ptlrpc_set_wait()) set ffff9856084cfd80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826939.555067:0:84106:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856086630c0 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:84106:1874669226267776:192.168.203.114@tcp:13:lfs.0 00020000:00000004:1.0:1787826939.555109:0:84106:0:(obd_class.h:1117:obd_statfs()) lustre-OST0000-osc-ffff985602bc9800: update ffff985609528450 cache blocks 901357/958279 objects 261544/262144 00000080:00200080:1.0:1787826939.555179:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00020000:00000080:1.0:1787826939.555197:0:84106:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00020000:00000004:1.0:1787826939.555207:0:84106:0:(obd_class.h:1091:obd_statfs()) lustre-OST0001-osc-ffff985602bc9800: age 3067, max_age 3089 00000100:00100000:1.0:1787826939.555246:0:84106:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856086630c0 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:84106:1874669226267904:192.168.203.114@tcp:13:lfs.0 00000100:00100000:1.0:1787826939.555345:0:84106:0:(client.c:2715:ptlrpc_set_wait()) set ffff9856084cfd80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826939.556401:0:84106:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856086630c0 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:84106:1874669226267904:192.168.203.114@tcp:13:lfs.0 00020000:00000004:1.0:1787826939.556430:0:84106:0:(obd_class.h:1117:obd_statfs()) lustre-OST0001-osc-ffff985602bc9800: update ffff98560952d1c0 cache blocks 901363/958279 objects 261551/262144 00000080:00200080:1.0:1787826939.556469:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00020000:00000080:1.0:1787826939.556481:0:84106:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.556489:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00020000:00000080:1.0:1787826939.556494:0:84106:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.556500:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00020000:00000080:1.0:1787826939.556504:0:84106:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.556509:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00020000:00000080:1.0:1787826939.556513:0:84106:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.556518:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00020000:00000080:1.0:1787826939.556523:0:84106:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.556529:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00020000:00000080:1.0:1787826939.556534:0:84106:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.556539:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00020000:00000080:1.0:1787826939.556544:0:84106:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.556549:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00020000:00000080:1.0:1787826939.556554:0:84106:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.556560:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00020000:00000080:1.0:1787826939.556565:0:84106:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.556570:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00020000:00000080:1.0:1787826939.556575:0:84106:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.556580:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00020000:00000080:1.0:1787826939.556585:0:84106:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.556591:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00020000:00000080:1.0:1787826939.556595:0:84106:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.556601:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00020000:00000080:1.0:1787826939.556606:0:84106:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.556611:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00020000:00000080:1.0:1787826939.556616:0:84106:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.556620:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00020000:00000080:1.0:1787826939.556635:0:84106:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.556641:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00020000:00000080:1.0:1787826939.556646:0:84106:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.556651:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00020000:00000080:1.0:1787826939.556656:0:84106:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.556661:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00020000:00000080:1.0:1787826939.556665:0:84106:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.556671:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00020000:00000080:1.0:1787826939.556676:0:84106:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.556682:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00020000:00000080:1.0:1787826939.556688:0:84106:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.556729:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00020000:00000080:1.0:1787826939.556735:0:84106:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.556740:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00020000:00000080:1.0:1787826939.556746:0:84106:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.556751:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00020000:00000080:1.0:1787826939.556756:0:84106:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.556762:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00020000:00000080:1.0:1787826939.556767:0:84106:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.556772:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00020000:00000080:1.0:1787826939.556777:0:84106:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.556784:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00020000:00000080:1.0:1787826939.556789:0:84106:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.556795:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00020000:00000080:1.0:1787826939.556799:0:84106:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.556804:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00020000:00000080:1.0:1787826939.556809:0:84106:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.556821:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00020000:00000080:1.0:1787826939.556826:0:84106:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.556831:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00020000:00000080:1.0:1787826939.556837:0:84106:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.556842:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00020000:00000080:1.0:1787826939.556847:0:84106:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.556864:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00020000:00000080:1.0:1787826939.556871:0:84106:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.556878:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00020000:00000080:1.0:1787826939.556883:0:84106:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.556888:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00020000:00000080:1.0:1787826939.556892:0:84106:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.556897:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00020000:00000080:1.0:1787826939.556901:0:84106:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.556905:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00020000:00000080:1.0:1787826939.556910:0:84106:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.556914:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00020000:00000080:1.0:1787826939.556918:0:84106:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.556923:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00020000:00000080:1.0:1787826939.556928:0:84106:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.556932:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00020000:00000080:1.0:1787826939.556937:0:84106:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.556941:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00020000:00000080:1.0:1787826939.556945:0:84106:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.556949:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00020000:00000080:1.0:1787826939.556953:0:84106:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.556958:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00020000:00000080:1.0:1787826939.556962:0:84106:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.556966:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00020000:00000080:1.0:1787826939.556970:0:84106:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.556974:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00020000:00000080:1.0:1787826939.556978:0:84106:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.556982:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00020000:00000080:1.0:1787826939.556986:0:84106:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.556991:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00020000:00000080:1.0:1787826939.556995:0:84106:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.557000:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00020000:00000080:1.0:1787826939.557003:0:84106:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.557008:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00020000:00000080:1.0:1787826939.557011:0:84106:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.557016:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00020000:00000080:1.0:1787826939.557020:0:84106:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.557024:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00020000:00000080:1.0:1787826939.557029:0:84106:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.557034:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00020000:00000080:1.0:1787826939.557037:0:84106:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.557042:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00020000:00000080:1.0:1787826939.557046:0:84106:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.557050:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00020000:00000080:1.0:1787826939.557054:0:84106:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.557059:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00020000:00000080:1.0:1787826939.557063:0:84106:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.557067:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00020000:00000080:1.0:1787826939.557071:0:84106:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.557075:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00020000:00000080:1.0:1787826939.557079:0:84106:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.557084:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00020000:00000080:1.0:1787826939.557088:0:84106:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.557093:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00020000:00000080:1.0:1787826939.557097:0:84106:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.557101:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00020000:00000080:1.0:1787826939.557105:0:84106:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.557110:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00020000:00000080:1.0:1787826939.557113:0:84106:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.557118:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00020000:00000080:1.0:1787826939.557132:0:84106:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.557138:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00020000:00000080:1.0:1787826939.557142:0:84106:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200080:1.0:1787826939.557147:0:84106:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffdd094e230 00020000:00000080:1.0:1787826939.557152:0:84106:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560863c000 uarg=0000000000000000 00000080:00200008:1.0:1787826939.557168:0:84106:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 100000 00010000:00010000:1.0:1787826939.700809:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826940.724971:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:1.0:1787826940.980743:0:15166:0:(mgc_request.c:611:do_requeue()) updating log lustre-cliir 10000000:01000000:1.0:1787826940.980753:0:15166:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-cliir-ffff985602bc9800 from 200 10000000:01000000:1.0:1787826940.980759:0:15166:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-cliir (res 0x65727473756c) 00010000:00010000:1.0:1787826940.980809:0:15166:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.114@tcp lock: ffff985610f40c00/0xb3f83dad1b7e408b lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826940.980821:0:15166:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.114@tcp lock: ffff985610f40c00/0xb3f83dad1b7e408b lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826940.980832:0:15166:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.114@tcp lock: ffff985610f40c00/0xb3f83dad1b7e408b lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826940.980860:0:15166:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856086603c0 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669226268032:192.168.203.114@tcp:101:ll_cfg_requeue.0 00000100:00100000:1.0:1787826940.980956:0:15166:0:(client.c:2715:ptlrpc_set_wait()) set ffff9856084cfd80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826940.982873:0:15166:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856086603c0 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669226268032:192.168.203.114@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787826940.982900:0:15166:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.114@tcp lock: ffff985610f40c00/0xb3f83dad1b7e408b lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x5aa6d504d91b13f9 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826940.982910:0:15166:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.114@tcp lock: ffff985610f40c00/0xb3f83dad1b7e408b lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x5aa6d504d91b13f9 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826940.982974:0:15166:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856086603c0 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669226268160:192.168.203.114@tcp:256:ll_cfg_requeue.0 00000100:00100000:1.0:1787826940.983040:0:15166:0:(client.c:2715:ptlrpc_set_wait()) set ffff9856084cfd80 going to sleep for 11 seconds 00000100:00100000:0.0:1787826940.985552:0:15166:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856086603c0 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669226268160:192.168.203.114@tcp:256:ll_cfg_requeue.0 10000000:01000000:0.0:1787826940.985649:0:15166:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.114@tcp: configuration from log 'lustre-cliir' succeeded (0). 00010000:00010000:0.0:1787826940.985660:0:15166:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.114@tcp lock: ffff985610f40c00/0xb3f83dad1b7e408b lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x5aa6d504d91b13f9 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826940.985669:0:15166:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.114@tcp lock: ffff985610f40c00/0xb3f83dad1b7e408b lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x5aa6d504d91b13f9 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787826940.985679:0:15166:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 10000000:01000000:0.0:1787826940.985683:0:15166:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff985602bc9800 from 52 10000000:01000000:0.0:1787826940.985686:0:15166:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00010000:00010000:0.0:1787826940.985756:0:15166:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.114@tcp lock: ffff985609eb1c00/0xb3f83dad1b7e4092 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826940.985764:0:15166:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.114@tcp lock: ffff985609eb1c00/0xb3f83dad1b7e4092 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826940.985773:0:15166:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.114@tcp lock: ffff985609eb1c00/0xb3f83dad1b7e4092 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826940.985792:0:15166:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608770780 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669226268288:192.168.203.114@tcp:101:ll_cfg_requeue.0 00000100:00100000:0.0:1787826940.985852:0:15166:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607ff0240 going to sleep for 11 seconds 00000100:00100000:0.0:1787826940.988531:0:15166:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608770780 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669226268288:192.168.203.114@tcp:101:ll_cfg_requeue.0 00010000:00010000:0.0:1787826940.988557:0:15166:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.114@tcp lock: ffff985609eb1c00/0xb3f83dad1b7e4092 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x5aa6d504d91b1400 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826940.988569:0:15166:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.114@tcp lock: ffff985609eb1c00/0xb3f83dad1b7e4092 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x5aa6d504d91b1400 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826940.988632:0:15166:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608770780 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669226268416:192.168.203.114@tcp:501:ll_cfg_requeue.0 00000100:00100000:0.0:1787826940.989031:0:15166:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607ff0240 going to sleep for 11 seconds 00000100:00100000:3.0:1787826940.992309:0:15166:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608770780 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669226268416:192.168.203.114@tcp:501:ll_cfg_requeue.0 00000100:00100000:3.0:1787826940.992368:0:15166:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e82580 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669226268544:192.168.203.114@tcp:503:ll_cfg_requeue.0 00000100:00100000:3.0:1787826940.992458:0:15166:0:(client.c:2715:ptlrpc_set_wait()) set ffff985605788d80 going to sleep for 11 seconds 00000100:00100000:0.0:1787826940.995012:0:15166:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e82580 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669226268544:192.168.203.114@tcp:503:ll_cfg_requeue.0 00000040:00080000:0.0:1787826940.995939:0:84239:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x3:0x0] index 64768 count 51 00000020:01000000:1.0:1787826940.996060:0:15166:0:(obd_config.c:2145:class_config_parse_llog()) Processed log lustre-client gen 52-51 (rc=0) 10000000:01000000:1.0:1787826940.996078:0:15166:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.114@tcp: configuration from log 'lustre-client' succeeded (0). 00010000:00010000:1.0:1787826940.996090:0:15166:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.114@tcp lock: ffff985609eb1c00/0xb3f83dad1b7e4092 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x5aa6d504d91b1400 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826940.996102:0:15166:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.114@tcp lock: ffff985609eb1c00/0xb3f83dad1b7e4092 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x5aa6d504d91b1400 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787826940.996113:0:15166:0:(mgc_request.c:611:do_requeue()) updating log params 10000000:01000000:1.0:1787826940.996116:0:15166:0:(mgc_request.c:1802:mgc_process_log()) Process log params-ffff985602bc9800 from 7 10000000:01000000:1.0:1787826940.996119:0:15166:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for params (res 0x736d61726170) 00010000:00010000:1.0:1787826940.996154:0:15166:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.114@tcp lock: ffff985610f41400/0xb3f83dad1b7e4099 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826940.996163:0:15166:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.114@tcp lock: ffff985610f41400/0xb3f83dad1b7e4099 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826940.996172:0:15166:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.114@tcp lock: ffff985610f41400/0xb3f83dad1b7e4099 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826940.996191:0:15166:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608661680 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669226268672:192.168.203.114@tcp:101:ll_cfg_requeue.0 00000100:00100000:1.0:1787826940.996280:0:15166:0:(client.c:2715:ptlrpc_set_wait()) set ffff9856084cfc00 going to sleep for 11 seconds 00000100:00100000:1.0:1787826940.997969:0:15166:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608661680 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669226268672:192.168.203.114@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787826940.997992:0:15166:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.114@tcp lock: ffff985610f41400/0xb3f83dad1b7e4099 lrc: 4/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x5aa6d504d91b1407 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826940.998004:0:15166:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.114@tcp lock: ffff985610f41400/0xb3f83dad1b7e4099 lrc: 4/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x5aa6d504d91b1407 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826940.998071:0:15166:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608661680 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669226268800:192.168.203.114@tcp:501:ll_cfg_requeue.0 00000100:00100000:1.0:1787826940.998138:0:15166:0:(client.c:2715:ptlrpc_set_wait()) set ffff9856084cfc00 going to sleep for 11 seconds 00000100:00100000:1.0:1787826940.999360:0:15166:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608661680 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669226268800:192.168.203.114@tcp:501:ll_cfg_requeue.0 00000100:00100000:1.0:1787826940.999408:0:15166:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608661680 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669226268928:192.168.203.114@tcp:503:ll_cfg_requeue.0 00000100:00100000:1.0:1787826940.999466:0:15166:0:(client.c:2715:ptlrpc_set_wait()) set ffff9856084cfc00 going to sleep for 11 seconds 00000100:00100000:1.0:1787826941.001942:0:15166:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608661680 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669226268928:192.168.203.114@tcp:503:ll_cfg_requeue.0 00000040:00080000:1.0:1787826941.003661:0:84243:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x2:0x0] index 64768 count 7 00000020:01000000:3.0:1787826941.003771:0:15166:0:(obd_config.c:2145:class_config_parse_llog()) Processed log params gen 7-6 (rc=0) 10000000:01000000:3.0:1787826941.003798:0:15166:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.114@tcp: configuration from log 'params' succeeded (0). 00010000:00010000:3.0:1787826941.003810:0:15166:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.114@tcp lock: ffff985610f41400/0xb3f83dad1b7e4099 lrc: 3/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x5aa6d504d91b1407 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826941.003822:0:15166:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.114@tcp lock: ffff985610f41400/0xb3f83dad1b7e4099 lrc: 2/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x5aa6d504d91b1407 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1787826941.003832:0:15166:0:(mgc_request.c:611:do_requeue()) updating log lustre-sptlrpc 10000000:01000000:3.0:1787826941.003835:0:15166:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-sptlrpc-ffff985606a30000 from 1 10000000:01000000:3.0:1787826941.003839:0:15166:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-sptlrpc (res 0x65727473756c) 00010000:00010000:3.0:1787826941.003871:0:15166:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.114@tcp lock: ffff985610e07200/0xb3f83dad1b7e40a0 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826941.003880:0:15166:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.114@tcp lock: ffff985610e07200/0xb3f83dad1b7e40a0 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826941.003890:0:15166:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.114@tcp lock: ffff985610e07200/0xb3f83dad1b7e40a0 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826941.003912:0:15166:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e80b40 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669226269056:192.168.203.114@tcp:101:ll_cfg_requeue.0 00000100:00100000:3.0:1787826941.003985:0:15166:0:(client.c:2715:ptlrpc_set_wait()) set ffff985605788840 going to sleep for 11 seconds 00000100:00100000:0.0:1787826941.009539:0:15166:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e80b40 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669226269056:192.168.203.114@tcp:101:ll_cfg_requeue.0 00010000:00010000:0.0:1787826941.009570:0:15166:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.114@tcp lock: ffff985610e07200/0xb3f83dad1b7e40a0 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0x5aa6d504d91b140e expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826941.009584:0:15166:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.114@tcp lock: ffff985610e07200/0xb3f83dad1b7e40a0 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1000000000000 nid: local remote: 0x5aa6d504d91b140e expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826941.009656:0:15166:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608771a40 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669226269184:192.168.203.114@tcp:501:ll_cfg_requeue.0 00000100:00100000:0.0:1787826941.009819:0:15166:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607ff00c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826941.013446:0:15166:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608771a40 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669226269184:192.168.203.114@tcp:501:ll_cfg_requeue.0 10000000:01000000:0.0:1787826941.013491:0:15166:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.114@tcp: configuration from log 'lustre-sptlrpc' failed (-2). 00010000:00010000:0.0:1787826941.013502:0:15166:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.114@tcp lock: ffff985610e07200/0xb3f83dad1b7e40a0 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0x5aa6d504d91b140e expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826941.013513:0:15166:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.114@tcp lock: ffff985610e07200/0xb3f83dad1b7e40a0 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0x5aa6d504d91b140e expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:02000400:3.0:1787826941.640550:0:84393:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: oleg314-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 00010000:00010000:3.0:1787826941.750060:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:0.0:1787826942.256268:0:84541:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec 00010000:00010000:2.0:1787826942.772889:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200008:2.0:1787826943.618070:0:84579:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:2.0:1787826943.618086:0:84579:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:2.0:1787826943.618110:0:84579:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826943.618129:0:84579:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5400/0xb3f83dad1b7e4084 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91b1312 expref: -99 pid: 84106 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:2.0:1787826943.618145:0:84579:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5400/0xb3f83dad1b7e4084 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91b1312 expref: -99 pid: 84106 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:2.0:1787826943.618154:0:84579:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787826943.618162:0:84579:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:2.0:1787826943.618166:0:84579:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e4084 00000080:00010000:2.0:1787826943.618174:0:84579:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e4084 from it ffffab3ac151bcb8 00010000:00010000:2.0:1787826943.618178:0:84579:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5400/0xb3f83dad1b7e4084 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91b1312 expref: -99 pid: 84106 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:2.0:1787826943.618187:0:84579:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5400/0xb3f83dad1b7e4084 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91b1312 expref: -99 pid: 84106 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00000080:00200008:2.0:1787826943.618824:0:84579:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 0, flags 0, foreign 0 00000080:00200000:2.0:1787826943.618829:0:84579:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:2.0:1787826943.618839:0:84579:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826943.618848:0:84579:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5400/0xb3f83dad1b7e4084 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91b1312 expref: -99 pid: 84106 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00010000:00010000:2.0:1787826943.618857:0:84579:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5400/0xb3f83dad1b7e4084 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91b1312 expref: -99 pid: 84106 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00000080:00200000:2.0:1787826943.618863:0:84579:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787826943.618868:0:84579:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:2.0:1787826943.618870:0:84579:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e4084 00000080:00010000:2.0:1787826943.618875:0:84579:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e4084 from it ffffab3ac151bcb8 00010000:00010000:2.0:1787826943.618879:0:84579:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5400/0xb3f83dad1b7e4084 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91b1312 expref: -99 pid: 84106 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00010000:00010000:2.0:1787826943.618886:0:84579:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5400/0xb3f83dad1b7e4084 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91b1312 expref: -99 pid: 84106 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00000080:00200000:2.0:1787826943.618964:0:84579:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:2.0:1787826943.618972:0:84579:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826943.618979:0:84579:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5400/0xb3f83dad1b7e4084 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91b1312 expref: -99 pid: 84106 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00010000:00010000:2.0:1787826943.618988:0:84579:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5400/0xb3f83dad1b7e4084 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91b1312 expref: -99 pid: 84106 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00000080:00200000:2.0:1787826943.618995:0:84579:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787826943.618999:0:84579:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:2.0:1787826943.619002:0:84579:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e4084 00000080:00010000:2.0:1787826943.619005:0:84579:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e4084 from it ffffab3ac151bbb0 00010000:00010000:2.0:1787826943.619008:0:84579:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5400/0xb3f83dad1b7e4084 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91b1312 expref: -99 pid: 84106 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00010000:00010000:2.0:1787826943.619016:0:84579:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5400/0xb3f83dad1b7e4084 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91b1312 expref: -99 pid: 84106 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00000080:00200000:2.0:1787826943.619024:0:84579:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 44 00000080:00200008:2.0:1787826943.619031:0:84579:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 100000 00000080:00200080:2.0:1787826943.619093:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00800000:00000080:2.0:1787826943.619108:0:84579:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00800000:00000004:2.0:1787826943.619115:0:84579:0:(obd_class.h:1091:obd_statfs()) lustre-MDT0000-mdc-ffff985602bc9800: age 3090, max_age 3093 00000100:00100000:2.0:1787826943.619159:0:84579:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee0f00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:84579:1874669226269312:192.168.203.114@tcp:41:lfs.0 00000100:00100000:2.0:1787826943.619286:0:84579:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c436c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826943.620767:0:84579:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee0f00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:84579:1874669226269312:192.168.203.114@tcp:41:lfs.0 00800000:00000004:2.0:1787826943.620789:0:84579:0:(obd_class.h:1117:obd_statfs()) lustre-MDT0000-mdc-ffff985602bc9800: update ffff985606a337f0 cache blocks 317333/353529 objects 1023289/1024000 00000080:00200080:2.0:1787826943.620833:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00800000:00000080:2.0:1787826943.620844:0:84579:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00800000:00000004:2.0:1787826943.620850:0:84579:0:(obd_class.h:1091:obd_statfs()) lustre-MDT0001-mdc-ffff985602bc9800: age 3090, max_age 3093 00000100:00100000:2.0:1787826943.620872:0:84579:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee0f00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:84579:1874669226269440:192.168.203.114@tcp:41:lfs.0 00000100:00100000:2.0:1787826943.620920:0:84579:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c436c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826943.622755:0:84579:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee0f00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:84579:1874669226269440:192.168.203.114@tcp:41:lfs.0 00800000:00000004:2.0:1787826943.622786:0:84579:0:(obd_class.h:1117:obd_statfs()) lustre-MDT0001-mdc-ffff985602bc9800: update ffff98560952b7f0 cache blocks 317401/353529 objects 1023737/1024000 00000080:00200080:2.0:1787826943.622836:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00800000:00000080:2.0:1787826943.622853:0:84579:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.622860:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00800000:00000080:2.0:1787826943.622864:0:84579:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.622868:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00800000:00000080:2.0:1787826943.622872:0:84579:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.622876:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00800000:00000080:2.0:1787826943.622879:0:84579:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.622884:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00800000:00000080:2.0:1787826943.622888:0:84579:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.622892:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00800000:00000080:2.0:1787826943.622897:0:84579:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.622902:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00800000:00000080:2.0:1787826943.622907:0:84579:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.622911:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00800000:00000080:2.0:1787826943.622915:0:84579:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.622919:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00800000:00000080:2.0:1787826943.622923:0:84579:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.622928:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00800000:00000080:2.0:1787826943.622932:0:84579:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.622936:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00800000:00000080:2.0:1787826943.622939:0:84579:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.622943:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00800000:00000080:2.0:1787826943.622946:0:84579:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.622950:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00800000:00000080:2.0:1787826943.622954:0:84579:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.622957:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00800000:00000080:2.0:1787826943.622961:0:84579:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.622965:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00800000:00000080:2.0:1787826943.622969:0:84579:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.622973:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00800000:00000080:2.0:1787826943.622977:0:84579:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.622981:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00800000:00000080:2.0:1787826943.622986:0:84579:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.622990:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00800000:00000080:2.0:1787826943.622993:0:84579:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.622998:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00800000:00000080:2.0:1787826943.623002:0:84579:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.623006:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00800000:00000080:2.0:1787826943.623011:0:84579:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.623015:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00800000:00000080:2.0:1787826943.623019:0:84579:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.623024:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00800000:00000080:2.0:1787826943.623028:0:84579:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.623032:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00800000:00000080:2.0:1787826943.623036:0:84579:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.623040:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00800000:00000080:2.0:1787826943.623043:0:84579:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.623048:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00800000:00000080:2.0:1787826943.623052:0:84579:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.623057:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00800000:00000080:2.0:1787826943.623061:0:84579:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.623066:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00800000:00000080:2.0:1787826943.623070:0:84579:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.623075:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00800000:00000080:2.0:1787826943.623079:0:84579:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.623084:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00800000:00000080:2.0:1787826943.623088:0:84579:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.623092:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00800000:00000080:2.0:1787826943.623096:0:84579:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.623100:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00800000:00000080:2.0:1787826943.623106:0:84579:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.623111:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00800000:00000080:2.0:1787826943.623115:0:84579:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.623119:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00800000:00000080:2.0:1787826943.623123:0:84579:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.623127:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00800000:00000080:2.0:1787826943.623131:0:84579:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.623136:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00800000:00000080:2.0:1787826943.623140:0:84579:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.623153:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00800000:00000080:2.0:1787826943.623156:0:84579:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.623160:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00800000:00000080:2.0:1787826943.623164:0:84579:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.623167:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00800000:00000080:2.0:1787826943.623171:0:84579:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.623174:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00800000:00000080:2.0:1787826943.623178:0:84579:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.623182:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00800000:00000080:2.0:1787826943.623184:0:84579:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.623188:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00800000:00000080:2.0:1787826943.623191:0:84579:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.623195:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00800000:00000080:2.0:1787826943.623198:0:84579:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.623203:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00800000:00000080:2.0:1787826943.623207:0:84579:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.623212:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00800000:00000080:2.0:1787826943.623215:0:84579:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.623220:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00800000:00000080:2.0:1787826943.623224:0:84579:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.623229:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00800000:00000080:2.0:1787826943.623232:0:84579:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.623236:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00800000:00000080:2.0:1787826943.623240:0:84579:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.623244:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00800000:00000080:2.0:1787826943.623248:0:84579:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.623252:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00800000:00000080:2.0:1787826943.623256:0:84579:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.623260:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00800000:00000080:2.0:1787826943.623263:0:84579:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.623267:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00800000:00000080:2.0:1787826943.623270:0:84579:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.623274:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00800000:00000080:2.0:1787826943.623278:0:84579:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.623281:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00800000:00000080:2.0:1787826943.623284:0:84579:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.623288:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00800000:00000080:2.0:1787826943.623291:0:84579:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.623296:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00800000:00000080:2.0:1787826943.623299:0:84579:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.623303:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00800000:00000080:2.0:1787826943.623307:0:84579:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.623311:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00800000:00000080:2.0:1787826943.623314:0:84579:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.623318:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00800000:00000080:2.0:1787826943.623321:0:84579:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.623325:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00800000:00000080:2.0:1787826943.623328:0:84579:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.623332:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00800000:00000080:2.0:1787826943.623336:0:84579:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.623339:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00800000:00000080:2.0:1787826943.623342:0:84579:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.623346:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00800000:00000080:2.0:1787826943.623349:0:84579:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.623353:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00800000:00000080:2.0:1787826943.623357:0:84579:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.623361:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00020000:00000080:2.0:1787826943.623376:0:84579:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00020000:00000004:2.0:1787826943.623400:0:84579:0:(obd_class.h:1091:obd_statfs()) lustre-OST0000-osc-ffff985602bc9800: age 3090, max_age 3093 00000100:00100000:2.0:1787826943.623442:0:84579:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee0f00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:84579:1874669226269568:192.168.203.114@tcp:13:lfs.0 00000100:00100000:2.0:1787826943.623525:0:84579:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c436c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826943.629626:0:84579:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee0f00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:84579:1874669226269568:192.168.203.114@tcp:13:lfs.0 00020000:00000004:2.0:1787826943.629658:0:84579:0:(obd_class.h:1117:obd_statfs()) lustre-OST0000-osc-ffff985602bc9800: update ffff985609528450 cache blocks 901357/958279 objects 261528/262144 00000080:00200080:2.0:1787826943.629756:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00020000:00000080:2.0:1787826943.629774:0:84579:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00020000:00000004:2.0:1787826943.629784:0:84579:0:(obd_class.h:1091:obd_statfs()) lustre-OST0001-osc-ffff985602bc9800: age 3090, max_age 3093 00000100:00100000:2.0:1787826943.629821:0:84579:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee0f00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:84579:1874669226269696:192.168.203.114@tcp:13:lfs.0 00000100:00100000:2.0:1787826943.629905:0:84579:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c436c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826943.631395:0:84579:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee0f00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:84579:1874669226269696:192.168.203.114@tcp:13:lfs.0 00020000:00000004:2.0:1787826943.631413:0:84579:0:(obd_class.h:1117:obd_statfs()) lustre-OST0001-osc-ffff985602bc9800: update ffff98560952d1c0 cache blocks 901363/958279 objects 261535/262144 00000080:00200080:2.0:1787826943.631452:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00020000:00000080:2.0:1787826943.631466:0:84579:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.631473:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00020000:00000080:2.0:1787826943.631478:0:84579:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.631481:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00020000:00000080:2.0:1787826943.631484:0:84579:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.631488:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00020000:00000080:2.0:1787826943.631491:0:84579:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.631495:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00020000:00000080:2.0:1787826943.631498:0:84579:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.631502:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00020000:00000080:2.0:1787826943.631505:0:84579:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.631509:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00020000:00000080:2.0:1787826943.631512:0:84579:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.631516:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00020000:00000080:2.0:1787826943.631519:0:84579:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.631523:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00020000:00000080:2.0:1787826943.631527:0:84579:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.631530:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00020000:00000080:2.0:1787826943.631534:0:84579:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.631538:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00020000:00000080:2.0:1787826943.631541:0:84579:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.631544:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00020000:00000080:2.0:1787826943.631547:0:84579:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.631551:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00020000:00000080:2.0:1787826943.631554:0:84579:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.631570:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00020000:00000080:2.0:1787826943.631575:0:84579:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.631580:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00020000:00000080:2.0:1787826943.631583:0:84579:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.631587:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00020000:00000080:2.0:1787826943.631591:0:84579:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.631595:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00020000:00000080:2.0:1787826943.631598:0:84579:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.631602:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00020000:00000080:2.0:1787826943.631612:0:84579:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.631620:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00020000:00000080:2.0:1787826943.631624:0:84579:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.631628:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00020000:00000080:2.0:1787826943.631631:0:84579:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.631635:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00020000:00000080:2.0:1787826943.631638:0:84579:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.631642:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00020000:00000080:2.0:1787826943.631644:0:84579:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.631649:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00020000:00000080:2.0:1787826943.631652:0:84579:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.631656:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00020000:00000080:2.0:1787826943.631659:0:84579:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.631663:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00020000:00000080:2.0:1787826943.631666:0:84579:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.631671:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00020000:00000080:2.0:1787826943.631678:0:84579:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.631712:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00020000:00000080:2.0:1787826943.631719:0:84579:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.631724:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00020000:00000080:2.0:1787826943.631728:0:84579:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.631732:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00020000:00000080:2.0:1787826943.631735:0:84579:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.631739:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00020000:00000080:2.0:1787826943.631743:0:84579:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.631747:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00020000:00000080:2.0:1787826943.631751:0:84579:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.631755:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00020000:00000080:2.0:1787826943.631758:0:84579:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.631763:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00020000:00000080:2.0:1787826943.631766:0:84579:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.631770:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00020000:00000080:2.0:1787826943.631774:0:84579:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.631778:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00020000:00000080:2.0:1787826943.631782:0:84579:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.631786:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00020000:00000080:2.0:1787826943.631790:0:84579:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.631793:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00020000:00000080:2.0:1787826943.631797:0:84579:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.631801:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00020000:00000080:2.0:1787826943.631805:0:84579:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.631809:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00020000:00000080:2.0:1787826943.631812:0:84579:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.631816:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00020000:00000080:2.0:1787826943.631820:0:84579:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.631835:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00020000:00000080:2.0:1787826943.631839:0:84579:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.631844:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00020000:00000080:2.0:1787826943.631847:0:84579:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.631851:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00020000:00000080:2.0:1787826943.631854:0:84579:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.631859:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00020000:00000080:2.0:1787826943.631862:0:84579:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.631865:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00020000:00000080:2.0:1787826943.631868:0:84579:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.631872:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00020000:00000080:2.0:1787826943.631875:0:84579:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.631879:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00020000:00000080:2.0:1787826943.631882:0:84579:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.631886:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00020000:00000080:2.0:1787826943.631889:0:84579:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.631893:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00020000:00000080:2.0:1787826943.631896:0:84579:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.631901:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00020000:00000080:2.0:1787826943.631904:0:84579:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.631908:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00020000:00000080:2.0:1787826943.631912:0:84579:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.631916:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00020000:00000080:2.0:1787826943.631919:0:84579:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.631923:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00020000:00000080:2.0:1787826943.631927:0:84579:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.631931:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00020000:00000080:2.0:1787826943.631934:0:84579:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.631938:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00020000:00000080:2.0:1787826943.631941:0:84579:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.631945:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00020000:00000080:2.0:1787826943.631948:0:84579:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.631952:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00020000:00000080:2.0:1787826943.631955:0:84579:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.631959:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00020000:00000080:2.0:1787826943.631968:0:84579:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.631972:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00020000:00000080:2.0:1787826943.631976:0:84579:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.631980:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00020000:00000080:2.0:1787826943.631984:0:84579:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.631988:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00020000:00000080:2.0:1787826943.631992:0:84579:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.631995:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00020000:00000080:2.0:1787826943.631999:0:84579:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200080:2.0:1787826943.632002:0:84579:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe411c8b00 00020000:00000080:2.0:1787826943.632006:0:84579:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985607c4c800 uarg=0000000000000000 00000080:00200008:2.0:1787826943.632020:0:84579:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 100000 00010000:00010000:3.0:1787826943.796918:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826944.245924:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:3c946890-15ca-497a-b838-fe1cba61f4dd:2386:1874669226269824:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787826944.246085:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee0f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669226270080:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787826944.246128:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669226270336:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787826944.246162:0:2386:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:2.0:1787826944.246169:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669226269952:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787826944.246192:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669226270208:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787826944.248102:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669226269952:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787826944.248116:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:3c946890-15ca-497a-b838-fe1cba61f4dd:2386:1874669226269824:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787826944.248126:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee0f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669226270080:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787826944.248729:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669226270208:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787826944.249830:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669226270336:192.168.203.114@tcp:400:kworker.0 00010000:00010000:2.0:1787826947.893399:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826948.916867:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826949.365012:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2385:1874669226475136:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787826949.365027:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669226475264:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787826949.365125:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669226475392:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787826949.365143:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee2d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669226475520:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787826949.365154:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669226475648:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787826949.368071:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2385:1874669226475136:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787826949.368073:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669226475264:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787826949.368097:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee2d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669226475520:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787826949.368103:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669226475392:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787826949.368120:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669226475648:192.168.203.114@tcp:400:kworker.0 00010000:00010000:1.0:1787826949.940876:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200000:2.0:1787826949.944931:0:83182:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:2.0:1787826949.944957:0:83182:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826949.945012:0:83182:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4200/0xb3f83dad1b7e5680 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 83182 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826949.945021:0:83182:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4200/0xb3f83dad1b7e5680 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 83182 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826949.945030:0:83182:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4200/0xb3f83dad1b7e5680 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 83182 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826949.945054:0:83182:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee2d00 pname:cluuid:pid:xid:nid:opc:job bash:7b4e8535-9db7-485d-abf6-bda966b63320:83182:1874669226475776:192.168.203.114@tcp:101:bash.0 00000100:00100000:2.0:1787826949.945150:0:83182:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c43c00 going to sleep for 11 seconds 00000100:00100000:1.0:1787826949.946843:0:83182:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748365200 gen 9 00000100:00100000:1.0:1787826949.947616:0:83182:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee2d00 pname:cluuid:pid:xid:nid:opc:job bash:7b4e8535-9db7-485d-abf6-bda966b63320:83182:1874669226475776:192.168.203.114@tcp:101:bash.0 00010000:00010000:1.0:1787826949.947632:0:83182:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4200/0xb3f83dad1b7e5680 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91b8ac8 expref: -99 pid: 83182 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826949.947647:0:83182:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4200/0xb3f83dad1b7e5680 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91b8ac8 expref: -99 pid: 83182 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826949.947659:0:83182:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4200/0xb3f83dad1b7e5680 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91b8ac8 expref: -99 pid: 83182 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826949.947667:0:83182:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4200/0xb3f83dad1b7e5680 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91b8ac8 expref: -99 pid: 83182 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826949.947677:0:83182:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4200/0xb3f83dad1b7e5680 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91b8ac8 expref: -99 pid: 83182 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826949.947681:0:83182:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4200/0xb3f83dad1b7e5680 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91b8ac8 expref: -99 pid: 83182 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826949.947723:0:83182:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4200/0xb3f83dad1b7e5680 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91b8ac8 expref: -99 pid: 83182 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826949.947733:0:83182:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff985602ee2d00 x1874669226475776/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826965 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'bash.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826949.947744:0:83182:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4200/0xb3f83dad1b7e5680 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91b8ac8 expref: -99 pid: 83182 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826949.947752:0:83182:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:1.0:1787826949.947757:0:83182:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:1.0:1787826949.947778:0:83182:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 69632 00000080:00010000:1.0:1787826949.947782:0:83182:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e5680 00000080:00010000:1.0:1787826949.947785:0:83182:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826949.947788:0:83182:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e5680 from it ffffab3ac1c5fbb0 00010000:00010000:1.0:1787826949.947790:0:83182:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4200/0xb3f83dad1b7e5680 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91b8ac8 expref: -99 pid: 83182 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826949.947796:0:83182:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4200/0xb3f83dad1b7e5680 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91b8ac8 expref: -99 pid: 83182 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826949.947807:0:83182:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 44 00000080:00200008:1.0:1787826949.947816:0:83182:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 304000 00000080:00200008:1.0:1787826949.947832:0:83182:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 2047, flags 0, foreign 0 00000002:00010000:1.0:1787826949.947847:0:83182:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: readdir flags 0 00010000:00010000:1.0:1787826949.947855:0:83182:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4200/0xb3f83dad1b7e5680 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91b8ac8 expref: -99 pid: 83182 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826949.947870:0:83182:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4200/0xb3f83dad1b7e5680 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91b8ac8 expref: -99 pid: 83182 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000100:00100000:1.0:1787826949.948475:0:83182:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610fdc000 pname:cluuid:pid:xid:nid:opc:job bash:7b4e8535-9db7-485d-abf6-bda966b63320:83182:1874669226475904:192.168.203.114@tcp:37:bash.0 00000100:00100000:1.0:1787826949.948548:0:83182:0:(client.c:2715:ptlrpc_set_wait()) set ffff9856084cf180 going to sleep for 11 seconds 00000100:00100000:1.0:1787826949.951494:0:83182:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610fdc000 pname:cluuid:pid:xid:nid:opc:job bash:7b4e8535-9db7-485d-abf6-bda966b63320:83182:1874669226475904:192.168.203.114@tcp:37:bash.0 00010000:00010000:1.0:1787826949.952109:0:83182:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4200/0xb3f83dad1b7e5680 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91b8ac8 expref: -99 pid: 83182 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826949.952124:0:83182:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4200/0xb3f83dad1b7e5680 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91b8ac8 expref: -99 pid: 83182 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826949.952148:0:83182:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826949.952166:0:83182:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826949.952174:0:83182:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4200/0xb3f83dad1b7e5680 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91b8ac8 expref: -99 pid: 83182 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826949.952181:0:83182:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4200/0xb3f83dad1b7e5680 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91b8ac8 expref: -99 pid: 83182 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826949.952186:0:83182:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826949.952196:0:83182:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826949.952198:0:83182:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e5680 00000080:00010000:1.0:1787826949.952201:0:83182:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e5680 from it ffffab3ac1c5fcd8 00010000:00010000:1.0:1787826949.952204:0:83182:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4200/0xb3f83dad1b7e5680 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91b8ac8 expref: -99 pid: 83182 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826949.952209:0:83182:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4200/0xb3f83dad1b7e5680 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91b8ac8 expref: -99 pid: 83182 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 1 00000080:00200000:1.0:1787826949.952457:0:83182:0:(dir.c:332:ll_iterate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) pos/size0/69632 32bit_api 0 00000080:00200000:1.0:1787826949.952511:0:83182:0:(dir.c:332:ll_iterate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) pos/size18446744073709551614/69632 32bit_api 0 00000080:00200008:1.0:1787826949.952524:0:83182:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 304000 00000080:00200000:3.0:1787826949.955495:0:84796:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:3.0:1787826949.955517:0:84796:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826949.955532:0:84796:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4200/0xb3f83dad1b7e5680 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91b8ac8 expref: -99 pid: 83182 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 1 00010000:00010000:3.0:1787826949.955545:0:84796:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4200/0xb3f83dad1b7e5680 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91b8ac8 expref: -99 pid: 83182 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 1 00000080:00200000:3.0:1787826949.955552:0:84796:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826949.955558:0:84796:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:3.0:1787826949.955561:0:84796:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e5680 00000080:00010000:3.0:1787826949.955566:0:84796:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e5680 from it ffffab3ac1d07b58 00010000:00010000:3.0:1787826949.955570:0:84796:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4200/0xb3f83dad1b7e5680 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91b8ac8 expref: -99 pid: 83182 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 1 00010000:00010000:3.0:1787826949.955583:0:84796:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4200/0xb3f83dad1b7e5680 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91b8ac8 expref: -99 pid: 83182 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00000080:00200000:3.0:1787826949.955594:0:84796:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:3.0:1787826949.955603:0:84796:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f38.replay-single-*, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), flags=8 00000080:00200000:3.0:1787826949.955607:0:84796:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f38.replay-single-*, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=getattr 00000002:00010000:3.0:1787826949.955618:0:84796:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f38.replay-single-*,[0x0:0x0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 01 00010000:00010000:3.0:1787826949.955654:0:84796:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06000/0xb3f83dad1b7e5687 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 84796 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826949.955662:0:84796:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06000/0xb3f83dad1b7e5687 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 84796 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826949.955671:0:84796:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06000/0xb3f83dad1b7e5687 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 84796 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826949.955716:0:84796:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e81680 pname:cluuid:pid:xid:nid:opc:job checkstat:7b4e8535-9db7-485d-abf6-bda966b63320:84796:1874669226476032:192.168.203.114@tcp:101:checkstat.0 00000100:00100000:3.0:1787826949.955785:0:84796:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607ff0240 going to sleep for 11 seconds 00000100:00100000:1.0:1787826949.957308:0:84796:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e81680 pname:cluuid:pid:xid:nid:opc:job checkstat:7b4e8535-9db7-485d-abf6-bda966b63320:84796:1874669226476032:192.168.203.114@tcp:101:checkstat.0 00010000:00010000:1.0:1787826949.957324:0:84796:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06000/0xb3f83dad1b7e5687 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 84796 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826949.957352:0:84796:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06000/0xb3f83dad1b7e5687 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 84796 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826949.957362:0:84796:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06000/0xb3f83dad1b7e5687 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 84796 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826949.957368:0:84796:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06000/0xb3f83dad1b7e5687 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 84796 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826949.957376:0:84796:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06000/0xb3f83dad1b7e5687 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 84796 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826949.957382:0:84796:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985610e06000 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:1.0:1787826949.957386:0:84796:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06000/0xb3f83dad1b7e5687 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 84796 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826949.957393:0:84796:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06000/0xb3f83dad1b7e5687 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 84796 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826949.957403:0:84796:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06000/0xb3f83dad1b7e5687 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x20e400000000 pid: 84796 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826949.957409:0:84796:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787826949.957415:0:84796:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06000/0xb3f83dad1b7e5687 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 84796 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826949.958119:0:84796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06000/0xb3f83dad1b7e5687 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 84796 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826949.958136:0:84796:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff985610e81680 x1874669226476032/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/584 e 0 to 0 dl 1787826965 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'checkstat.0' uid:0 gid:0 projid:0 00000080:00200000:1.0:1787826949.958150:0:84796:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00010000:00010000:1.0:1787826949.958171:0:84796:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4200/0xb3f83dad1b7e5680 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91b8ac8 expref: -99 pid: 83182 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00010000:00010000:1.0:1787826949.958181:0:84796:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4200/0xb3f83dad1b7e5680 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91b8ac8 expref: -99 pid: 83182 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00000080:00010000:1.0:1787826949.958189:0:84796:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e5680 from it ffffab3ac1d07930 00010000:00010000:1.0:1787826949.958192:0:84796:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4200/0xb3f83dad1b7e5680 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91b8ac8 expref: -99 pid: 83182 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00010000:00010000:1.0:1787826949.958199:0:84796:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4200/0xb3f83dad1b7e5680 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91b8ac8 expref: -99 pid: 83182 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00010000:00010000:2.0:1787826950.964933:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826951.989837:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:0.0:1787826952.350119:0:85360:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == replay-single test 39: test recovery from unlink llog (test llog_gen_rec) ========================================================== 06:35:51 (1787826951) 00010000:00010000:1.0:1787826953.012997:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826958.133101:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200008:0.0:1787826958.502834:0:85570:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:0.0:1787826958.502845:0:85570:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:0.0:1787826958.502865:0:85570:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826958.502926:0:85570:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57c00/0xb3f83dad1b7e8247 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 85570 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826958.502937:0:85570:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57c00/0xb3f83dad1b7e8247 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 85570 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826958.502947:0:85570:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57c00/0xb3f83dad1b7e8247 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 85570 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826958.502972:0:85570:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610cea1c0 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:85570:1874669226988672:192.168.203.114@tcp:101:lfs.0 00000100:00100000:0.0:1787826958.503048:0:85570:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607ff0900 going to sleep for 11 seconds 00000100:00100000:0.0:1787826958.506164:0:85570:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1787826958.508916:0:85570:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610cea1c0 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:85570:1874669226988672:192.168.203.114@tcp:101:lfs.0 00010000:00010000:0.0:1787826958.508937:0:85570:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57c00/0xb3f83dad1b7e8247 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91c2593 expref: -99 pid: 85570 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826958.508946:0:85570:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57c00/0xb3f83dad1b7e8247 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91c2593 expref: -99 pid: 85570 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826958.508954:0:85570:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57c00/0xb3f83dad1b7e8247 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91c2593 expref: -99 pid: 85570 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826958.508962:0:85570:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57c00/0xb3f83dad1b7e8247 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91c2593 expref: -99 pid: 85570 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826958.508970:0:85570:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57c00/0xb3f83dad1b7e8247 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91c2593 expref: -99 pid: 85570 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826958.508979:0:85570:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57c00/0xb3f83dad1b7e8247 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91c2593 expref: -99 pid: 85570 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826958.508984:0:85570:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57c00/0xb3f83dad1b7e8247 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91c2593 expref: -99 pid: 85570 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787826958.508991:0:85570:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff985610cea1c0 x1874669226988672/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826974 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787826958.509001:0:85570:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57c00/0xb3f83dad1b7e8247 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91c2593 expref: -99 pid: 85570 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826958.509010:0:85570:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:0.0:1787826958.509015:0:85570:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:0.0:1787826958.509078:0:85570:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 69632 00000080:00010000:0.0:1787826958.509082:0:85570:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e8247 00000080:00010000:0.0:1787826958.509086:0:85570:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:0.0:1787826958.509088:0:85570:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e8247 from it ffffab3ac1d57cb8 00010000:00010000:0.0:1787826958.509090:0:85570:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57c00/0xb3f83dad1b7e8247 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91c2593 expref: -99 pid: 85570 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826958.509096:0:85570:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57c00/0xb3f83dad1b7e8247 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91c2593 expref: -99 pid: 85570 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:0.0:1787826958.509624:0:85570:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 0, flags 0, foreign 0 00000080:00200000:0.0:1787826958.509630:0:85570:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:0.0:1787826958.509647:0:85570:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826958.509655:0:85570:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57c00/0xb3f83dad1b7e8247 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91c2593 expref: -99 pid: 85570 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826958.509664:0:85570:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57c00/0xb3f83dad1b7e8247 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91c2593 expref: -99 pid: 85570 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826958.509672:0:85570:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826958.509676:0:85570:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:0.0:1787826958.509679:0:85570:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e8247 00000080:00010000:0.0:1787826958.509683:0:85570:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e8247 from it ffffab3ac1d57cb8 00010000:00010000:0.0:1787826958.509686:0:85570:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57c00/0xb3f83dad1b7e8247 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91c2593 expref: -99 pid: 85570 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826958.509725:0:85570:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57c00/0xb3f83dad1b7e8247 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91c2593 expref: -99 pid: 85570 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1787826958.509816:0:85570:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:0.0:1787826958.509825:0:85570:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826958.509833:0:85570:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57c00/0xb3f83dad1b7e8247 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91c2593 expref: -99 pid: 85570 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1787826958.509841:0:85570:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57c00/0xb3f83dad1b7e8247 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91c2593 expref: -99 pid: 85570 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1787826958.509846:0:85570:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826958.509849:0:85570:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:0.0:1787826958.509850:0:85570:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e8247 00000080:00010000:0.0:1787826958.509854:0:85570:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e8247 from it ffffab3ac1d57bb0 00010000:00010000:0.0:1787826958.509858:0:85570:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57c00/0xb3f83dad1b7e8247 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91c2593 expref: -99 pid: 85570 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1787826958.509864:0:85570:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57c00/0xb3f83dad1b7e8247 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91c2593 expref: -99 pid: 85570 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 1 00000080:00200000:0.0:1787826958.509874:0:85570:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 44 00000080:00200008:0.0:1787826958.509880:0:85570:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 100000 00000080:00200080:0.0:1787826958.509962:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00800000:00000080:0.0:1787826958.509973:0:85570:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00800000:00000004:0.0:1787826958.509978:0:85570:0:(obd_class.h:1091:obd_statfs()) lustre-MDT0000-mdc-ffff985602bc9800: age 3094, max_age 3108 00000100:00100000:0.0:1787826958.510005:0:85570:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610cea1c0 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:85570:1874669226988800:192.168.203.114@tcp:41:lfs.0 00000100:00100000:0.0:1787826958.510089:0:85570:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607ff0900 going to sleep for 11 seconds 00000100:00100000:0.0:1787826958.511362:0:85570:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610cea1c0 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:85570:1874669226988800:192.168.203.114@tcp:41:lfs.0 00800000:00000004:0.0:1787826958.511377:0:85570:0:(obd_class.h:1117:obd_statfs()) lustre-MDT0000-mdc-ffff985602bc9800: update ffff985606a337f0 cache blocks 317325/353529 objects 1022889/1024000 00000080:00200080:0.0:1787826958.511434:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00800000:00000080:0.0:1787826958.511442:0:85570:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00800000:00000004:0.0:1787826958.511447:0:85570:0:(obd_class.h:1091:obd_statfs()) lustre-MDT0001-mdc-ffff985602bc9800: age 3094, max_age 3108 00000100:00100000:0.0:1787826958.511462:0:85570:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610cea1c0 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:85570:1874669226988928:192.168.203.114@tcp:41:lfs.0 00000100:00100000:0.0:1787826958.511503:0:85570:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607ff0900 going to sleep for 11 seconds 00000100:00100000:0.0:1787826958.514393:0:85570:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610cea1c0 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:85570:1874669226988928:192.168.203.114@tcp:41:lfs.0 00800000:00000004:0.0:1787826958.514415:0:85570:0:(obd_class.h:1117:obd_statfs()) lustre-MDT0001-mdc-ffff985602bc9800: update ffff98560952b7f0 cache blocks 317401/353529 objects 1023737/1024000 00000080:00200080:0.0:1787826958.514466:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00800000:00000080:0.0:1787826958.514481:0:85570:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.514488:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00800000:00000080:0.0:1787826958.514492:0:85570:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.514496:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00800000:00000080:0.0:1787826958.514500:0:85570:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.514504:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00800000:00000080:0.0:1787826958.514507:0:85570:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.514511:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00800000:00000080:0.0:1787826958.514514:0:85570:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.514518:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00800000:00000080:0.0:1787826958.514521:0:85570:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.514525:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00800000:00000080:0.0:1787826958.514528:0:85570:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.514532:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00800000:00000080:0.0:1787826958.514535:0:85570:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.514539:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00800000:00000080:0.0:1787826958.514542:0:85570:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.514546:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00800000:00000080:0.0:1787826958.514549:0:85570:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.514552:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00800000:00000080:0.0:1787826958.514555:0:85570:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.514558:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00800000:00000080:0.0:1787826958.514561:0:85570:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.514565:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00800000:00000080:0.0:1787826958.514568:0:85570:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.514578:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00800000:00000080:0.0:1787826958.514582:0:85570:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.514586:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00800000:00000080:0.0:1787826958.514589:0:85570:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.514593:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00800000:00000080:0.0:1787826958.514597:0:85570:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.514601:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00800000:00000080:0.0:1787826958.514603:0:85570:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.514607:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00800000:00000080:0.0:1787826958.514610:0:85570:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.514614:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00800000:00000080:0.0:1787826958.514616:0:85570:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.514620:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00800000:00000080:0.0:1787826958.514623:0:85570:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.514626:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00800000:00000080:0.0:1787826958.514629:0:85570:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.514633:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00800000:00000080:0.0:1787826958.514636:0:85570:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.514640:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00800000:00000080:0.0:1787826958.514642:0:85570:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.514646:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00800000:00000080:0.0:1787826958.514649:0:85570:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.514653:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00800000:00000080:0.0:1787826958.514656:0:85570:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.514659:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00800000:00000080:0.0:1787826958.514672:0:85570:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.514679:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00800000:00000080:0.0:1787826958.514683:0:85570:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.514710:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00800000:00000080:0.0:1787826958.514716:0:85570:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.514722:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00800000:00000080:0.0:1787826958.514728:0:85570:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.514734:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00800000:00000080:0.0:1787826958.514738:0:85570:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.514744:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00800000:00000080:0.0:1787826958.514749:0:85570:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.514754:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00800000:00000080:0.0:1787826958.514759:0:85570:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.514765:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00800000:00000080:0.0:1787826958.514769:0:85570:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.514774:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00800000:00000080:0.0:1787826958.514778:0:85570:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.514784:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00800000:00000080:0.0:1787826958.514788:0:85570:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.514793:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00800000:00000080:0.0:1787826958.514797:0:85570:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.514802:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00800000:00000080:0.0:1787826958.514806:0:85570:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.514810:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00800000:00000080:0.0:1787826958.514813:0:85570:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.514817:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00800000:00000080:0.0:1787826958.514821:0:85570:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.514826:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00800000:00000080:0.0:1787826958.514829:0:85570:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.514834:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00800000:00000080:0.0:1787826958.514837:0:85570:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.514841:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00800000:00000080:0.0:1787826958.514845:0:85570:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.514849:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00800000:00000080:0.0:1787826958.514853:0:85570:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.514873:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00800000:00000080:0.0:1787826958.514878:0:85570:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.514883:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00800000:00000080:0.0:1787826958.514887:0:85570:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.514891:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00800000:00000080:0.0:1787826958.514895:0:85570:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.514899:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00800000:00000080:0.0:1787826958.514902:0:85570:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.514907:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00800000:00000080:0.0:1787826958.514910:0:85570:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.514915:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00800000:00000080:0.0:1787826958.514918:0:85570:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.514922:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00800000:00000080:0.0:1787826958.514926:0:85570:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.514930:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00800000:00000080:0.0:1787826958.514934:0:85570:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.514938:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00800000:00000080:0.0:1787826958.514942:0:85570:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.514946:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00800000:00000080:0.0:1787826958.514950:0:85570:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.514953:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00800000:00000080:0.0:1787826958.514957:0:85570:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.514961:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00800000:00000080:0.0:1787826958.514965:0:85570:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.514969:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00800000:00000080:0.0:1787826958.514973:0:85570:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.514977:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00800000:00000080:0.0:1787826958.514980:0:85570:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.514985:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00800000:00000080:0.0:1787826958.514988:0:85570:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.514992:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00800000:00000080:0.0:1787826958.514996:0:85570:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.514999:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00800000:00000080:0.0:1787826958.515003:0:85570:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.515007:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00800000:00000080:0.0:1787826958.515010:0:85570:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.515015:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00800000:00000080:0.0:1787826958.515018:0:85570:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.515022:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00800000:00000080:0.0:1787826958.515026:0:85570:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.515030:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00020000:00000080:0.0:1787826958.515037:0:85570:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00020000:00000004:0.0:1787826958.515045:0:85570:0:(obd_class.h:1091:obd_statfs()) lustre-OST0000-osc-ffff985602bc9800: age 3094, max_age 3108 00000100:00100000:0.0:1787826958.515076:0:85570:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610cea1c0 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:85570:1874669226989056:192.168.203.114@tcp:13:lfs.0 00000100:00100000:0.0:1787826958.515148:0:85570:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607ff0900 going to sleep for 11 seconds 00000100:00100000:0.0:1787826958.516518:0:85570:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610cea1c0 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:85570:1874669226989056:192.168.203.114@tcp:13:lfs.0 00020000:00000004:0.0:1787826958.516537:0:85570:0:(obd_class.h:1117:obd_statfs()) lustre-OST0000-osc-ffff985602bc9800: update ffff985609528450 cache blocks 901357/958279 objects 261376/262144 00000080:00200080:0.0:1787826958.516582:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00020000:00000080:0.0:1787826958.516593:0:85570:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00020000:00000004:0.0:1787826958.516599:0:85570:0:(obd_class.h:1091:obd_statfs()) lustre-OST0001-osc-ffff985602bc9800: age 3094, max_age 3108 00000100:00100000:0.0:1787826958.516620:0:85570:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610cea1c0 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:85570:1874669226989184:192.168.203.114@tcp:13:lfs.0 00000100:00100000:0.0:1787826958.516671:0:85570:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607ff0900 going to sleep for 11 seconds 00000100:00100000:0.0:1787826958.517808:0:85570:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610cea1c0 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:85570:1874669226989184:192.168.203.114@tcp:13:lfs.0 00020000:00000004:0.0:1787826958.517842:0:85570:0:(obd_class.h:1117:obd_statfs()) lustre-OST0001-osc-ffff985602bc9800: update ffff98560952d1c0 cache blocks 901363/958279 objects 261383/262144 00000080:00200080:0.0:1787826958.517906:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00020000:00000080:0.0:1787826958.517924:0:85570:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.517932:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00020000:00000080:0.0:1787826958.517938:0:85570:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.517943:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00020000:00000080:0.0:1787826958.517947:0:85570:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.517966:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00020000:00000080:0.0:1787826958.517971:0:85570:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.517976:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00020000:00000080:0.0:1787826958.517981:0:85570:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.517986:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00020000:00000080:0.0:1787826958.517990:0:85570:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.517996:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00020000:00000080:0.0:1787826958.518000:0:85570:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.518004:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00020000:00000080:0.0:1787826958.518008:0:85570:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.518021:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00020000:00000080:0.0:1787826958.518026:0:85570:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.518030:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00020000:00000080:0.0:1787826958.518034:0:85570:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.518039:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00020000:00000080:0.0:1787826958.518042:0:85570:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.518047:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00020000:00000080:0.0:1787826958.518050:0:85570:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.518055:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00020000:00000080:0.0:1787826958.518059:0:85570:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.518063:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00020000:00000080:0.0:1787826958.518067:0:85570:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.518072:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00020000:00000080:0.0:1787826958.518076:0:85570:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.518080:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00020000:00000080:0.0:1787826958.518084:0:85570:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.518088:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00020000:00000080:0.0:1787826958.518091:0:85570:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.518096:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00020000:00000080:0.0:1787826958.518100:0:85570:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.518104:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00020000:00000080:0.0:1787826958.518109:0:85570:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.518113:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00020000:00000080:0.0:1787826958.518117:0:85570:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.518121:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00020000:00000080:0.0:1787826958.518126:0:85570:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.518130:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00020000:00000080:0.0:1787826958.518134:0:85570:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.518138:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00020000:00000080:0.0:1787826958.518142:0:85570:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.518146:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00020000:00000080:0.0:1787826958.518151:0:85570:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.518155:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00020000:00000080:0.0:1787826958.518158:0:85570:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.518162:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00020000:00000080:0.0:1787826958.518166:0:85570:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.518170:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00020000:00000080:0.0:1787826958.518174:0:85570:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.518178:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00020000:00000080:0.0:1787826958.518182:0:85570:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.518186:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00020000:00000080:0.0:1787826958.518190:0:85570:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.518194:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00020000:00000080:0.0:1787826958.518198:0:85570:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.518202:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00020000:00000080:0.0:1787826958.518206:0:85570:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.518211:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00020000:00000080:0.0:1787826958.518214:0:85570:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.518218:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00020000:00000080:0.0:1787826958.518221:0:85570:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.518225:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00020000:00000080:0.0:1787826958.518229:0:85570:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.518233:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00020000:00000080:0.0:1787826958.518236:0:85570:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.518241:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00020000:00000080:0.0:1787826958.518244:0:85570:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.518248:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00020000:00000080:0.0:1787826958.518252:0:85570:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.518255:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00020000:00000080:0.0:1787826958.518259:0:85570:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.518262:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00020000:00000080:0.0:1787826958.518266:0:85570:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.518270:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00020000:00000080:0.0:1787826958.518287:0:85570:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.518293:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00020000:00000080:0.0:1787826958.518297:0:85570:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.518301:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00020000:00000080:0.0:1787826958.518304:0:85570:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.518308:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00020000:00000080:0.0:1787826958.518311:0:85570:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.518315:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00020000:00000080:0.0:1787826958.518319:0:85570:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.518323:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00020000:00000080:0.0:1787826958.518327:0:85570:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.518331:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00020000:00000080:0.0:1787826958.518334:0:85570:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.518338:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00020000:00000080:0.0:1787826958.518342:0:85570:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.518346:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00020000:00000080:0.0:1787826958.518349:0:85570:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.518353:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00020000:00000080:0.0:1787826958.518357:0:85570:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.518360:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00020000:00000080:0.0:1787826958.518364:0:85570:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.518368:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00020000:00000080:0.0:1787826958.518371:0:85570:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.518375:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00020000:00000080:0.0:1787826958.518378:0:85570:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.518382:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00020000:00000080:0.0:1787826958.518385:0:85570:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.518389:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00020000:00000080:0.0:1787826958.518402:0:85570:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.518407:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00020000:00000080:0.0:1787826958.518411:0:85570:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.518415:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00020000:00000080:0.0:1787826958.518418:0:85570:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.518423:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00020000:00000080:0.0:1787826958.518427:0:85570:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.518430:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00020000:00000080:0.0:1787826958.518434:0:85570:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.518437:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00020000:00000080:0.0:1787826958.518441:0:85570:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.518444:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00020000:00000080:0.0:1787826958.518448:0:85570:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.518452:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00020000:00000080:0.0:1787826958.518455:0:85570:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.518458:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00020000:00000080:0.0:1787826958.518462:0:85570:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200080:0.0:1787826958.518466:0:85570:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc38a23850 00020000:00000080:0.0:1787826958.518469:0:85570:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560543f000 uarg=0000000000000000 00000080:00200008:0.0:1787826958.518485:0:85570:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 100000 00000080:00200000:0.0:1787826958.537582:0:85577:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:0.0:1787826958.537607:0:85577:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826958.537623:0:85577:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57c00/0xb3f83dad1b7e8247 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91c2593 expref: -99 pid: 85570 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 1 00010000:00010000:0.0:1787826958.537637:0:85577:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57c00/0xb3f83dad1b7e8247 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91c2593 expref: -99 pid: 85570 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 1 00000080:00200000:0.0:1787826958.537645:0:85577:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826958.537652:0:85577:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:0.0:1787826958.537655:0:85577:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e8247 00000080:00010000:0.0:1787826958.537662:0:85577:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e8247 from it ffffab3ac1cefbd0 00010000:00010000:0.0:1787826958.537667:0:85577:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57c00/0xb3f83dad1b7e8247 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91c2593 expref: -99 pid: 85570 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 1 00010000:00010000:0.0:1787826958.537676:0:85577:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57c00/0xb3f83dad1b7e8247 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91c2593 expref: -99 pid: 85570 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00000080:00200000:0.0:1787826958.537733:0:85577:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:0.0:1787826958.537746:0:85577:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=fsa-oleg314-client.virtnet, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), flags=1536 00000080:00200000:0.0:1787826958.537750:0:85577:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:0.0:1787826958.537757:0:85577:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826958.537764:0:85577:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57c00/0xb3f83dad1b7e8247 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91c2593 expref: -99 pid: 85570 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00010000:00010000:0.0:1787826958.537773:0:85577:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57c00/0xb3f83dad1b7e8247 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91c2593 expref: -99 pid: 85570 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00000080:00200000:0.0:1787826958.537779:0:85577:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826958.537783:0:85577:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:0.0:1787826958.537786:0:85577:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e8247 00000080:00010000:0.0:1787826958.537790:0:85577:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e8247 from it ffffab3ac1cefcd0 00010000:00010000:0.0:1787826958.537793:0:85577:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57c00/0xb3f83dad1b7e8247 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91c2593 expref: -99 pid: 85570 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00010000:00010000:0.0:1787826958.537800:0:85577:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57c00/0xb3f83dad1b7e8247 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91c2593 expref: -99 pid: 85570 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00000080:00200000:0.0:1787826958.537809:0:85577:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 3 00000080:00200000:0.0:1787826958.537814:0:85577:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:0.0:1787826958.537821:0:85577:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826958.537827:0:85577:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57c00/0xb3f83dad1b7e8247 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91c2593 expref: -99 pid: 85570 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00010000:00010000:0.0:1787826958.537832:0:85577:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57c00/0xb3f83dad1b7e8247 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91c2593 expref: -99 pid: 85570 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00000080:00200000:0.0:1787826958.537836:0:85577:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826958.537839:0:85577:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:0.0:1787826958.537840:0:85577:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e8247 00000080:00010000:0.0:1787826958.537843:0:85577:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e8247 from it ffffab3ac1cefdb0 00010000:00010000:0.0:1787826958.537845:0:85577:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57c00/0xb3f83dad1b7e8247 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91c2593 expref: -99 pid: 85570 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00010000:00010000:0.0:1787826958.537849:0:85577:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57c00/0xb3f83dad1b7e8247 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91c2593 expref: -99 pid: 85570 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00000080:00200000:0.0:1787826958.537853:0:85577:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 3 00000080:00200000:0.0:1787826958.537856:0:85577:0:(namei.c:2129:ll_create_nd()) VFS Op:name=fsa-oleg314-client.virtnet, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), flags=33188, excl=1 00000080:00200000:0.0:1787826958.537859:0:85577:0:(namei.c:2084:ll_mknod()) VFS Op:name=fsa-oleg314-client.virtnet, dir=[0x200000007:0x1:0x0](ffff98561158e3c8) mode 100644 dev 0 00010000:00010000:0.0:1787826958.537876:0:85577:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57c00/0xb3f83dad1b7e8247 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x5aa6d504d91c2593 expref: -99 pid: 85570 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00010000:00010000:0.0:1787826958.537999:0:85577:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57c00/0xb3f83dad1b7e8247 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91c2593 expref: -99 pid: 85570 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00010000:00010000:0.0:1787826958.538004:0:85577:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826958.538006:0:85577:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57c00/0xb3f83dad1b7e8247 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91c2593 expref: -99 pid: 85570 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00000020:00100000:0.0:1787826958.538029:0:85577:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:0.0:1787826958.538040:0:85577:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610cea1c0 x1874669226989440/t0(0) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 528/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826958.538062:0:85577:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610cea1c0 pname:cluuid:pid:xid:nid:opc:job mcreate:7b4e8535-9db7-485d-abf6-bda966b63320:85577:1874669226989440:192.168.203.114@tcp:36:mcreate.0 00000100:00100000:0.0:1787826958.538133:0:85577:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607ff0900 going to sleep for 11 seconds 00000100:00100000:0.0:1787826958.540314:0:85577:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610cea1c0 pname:cluuid:pid:xid:nid:opc:job mcreate:7b4e8535-9db7-485d-abf6-bda966b63320:85577:1874669226989440:192.168.203.114@tcp:36:mcreate.0 00000080:00200000:0.0:1787826958.540344:0:85577:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x2000182d1:0x642:0x0](ffff9856102a5348) 00000080:00200000:0.0:1787826958.540369:0:85577:0:(namei.c:2137:ll_create_nd()) VFS Op:name=fsa-oleg314-client.virtnet, unhashed 1 00000080:00200000:3.0:1787826958.548009:0:85580:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:3.0:1787826958.548048:0:85580:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826958.548126:0:85580:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7e824e lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 85580 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826958.548138:0:85580:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7e824e lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 85580 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826958.548149:0:85580:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7e824e lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 85580 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826958.548182:0:85580:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e81e00 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:85580:1874669226989568:192.168.203.114@tcp:101:rm.0 00000100:00100000:3.0:1787826958.548305:0:85580:0:(client.c:2715:ptlrpc_set_wait()) set ffff985611f30cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826958.550081:0:85580:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e81e00 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:85580:1874669226989568:192.168.203.114@tcp:101:rm.0 00010000:00010000:0.0:1787826958.550106:0:85580:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7e824e lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91c25af expref: -99 pid: 85580 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826958.550118:0:85580:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7e824e lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91c25af expref: -99 pid: 85580 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826958.550133:0:85580:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7e824e lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91c25af expref: -99 pid: 85580 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826958.550147:0:85580:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7e824e lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91c25af expref: -99 pid: 85580 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826958.550160:0:85580:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7e824e lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91c25af expref: -99 pid: 85580 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826958.550169:0:85580:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7e824e lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91c25af expref: -99 pid: 85580 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826958.550178:0:85580:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7e824e lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91c25af expref: -99 pid: 85580 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787826958.550191:0:85580:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff985610e81e00 x1874669226989568/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826974 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787826958.550205:0:85580:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7e824e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91c25af expref: -99 pid: 85580 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826958.550218:0:85580:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:0.0:1787826958.550225:0:85580:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:0.0:1787826958.550256:0:85580:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 69632 00000080:00010000:0.0:1787826958.550263:0:85580:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e824e 00000080:00010000:0.0:1787826958.550270:0:85580:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:0.0:1787826958.550286:0:85580:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e824e from it ffffab3ac1d57b58 00010000:00010000:0.0:1787826958.550290:0:85580:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7e824e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91c25af expref: -99 pid: 85580 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826958.550300:0:85580:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7e824e lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91c25af expref: -99 pid: 85580 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826958.550326:0:85580:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:0.0:1787826958.550345:0:85580:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=fsa-oleg314-client.virtnet, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), flags=8 00000080:00200000:0.0:1787826958.550349:0:85580:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=fsa-oleg314-client.virtnet, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=getattr 00000002:00010000:0.0:1787826958.550377:0:85580:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: fsa-oleg314-client.virtnet,[0x0:0x0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787826958.550414:0:85580:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57c00/0xb3f83dad1b7e8255 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 85580 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826958.550423:0:85580:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57c00/0xb3f83dad1b7e8255 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 85580 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826958.550431:0:85580:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57c00/0xb3f83dad1b7e8255 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 85580 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826958.550450:0:85580:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610ce8b40 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:85580:1874669226989696:192.168.203.114@tcp:101:rm.0 00000100:00100000:0.0:1787826958.550529:0:85580:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607ff0900 going to sleep for 11 seconds 00000100:00100000:0.0:1787826958.552265:0:85580:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610ce8b40 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:85580:1874669226989696:192.168.203.114@tcp:101:rm.0 00010000:00010000:0.0:1787826958.552291:0:85580:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57c00/0xb3f83dad1b7e8255 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91c25cb expref: -99 pid: 85580 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826958.552307:0:85580:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57c00/0xb3f83dad1b7e8255 lrc: 4/1,0 mode: --/PR res: [0x2000182d1:0x642:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91c25cb expref: -99 pid: 85580 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826958.552313:0:85580:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57c00/0xb3f83dad1b7e8255 lrc: 4/1,0 mode: --/PR res: [0x2000182d1:0x642:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91c25cb expref: -99 pid: 85580 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826958.552319:0:85580:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57c00/0xb3f83dad1b7e8255 lrc: 4/1,0 mode: PR/PR res: [0x2000182d1:0x642:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91c25cb expref: -99 pid: 85580 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826958.552325:0:85580:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57c00/0xb3f83dad1b7e8255 lrc: 4/1,0 mode: PR/PR res: [0x2000182d1:0x642:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91c25cb expref: -99 pid: 85580 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826958.552333:0:85580:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57c00/0xb3f83dad1b7e8255 lrc: 5/2,0 mode: PR/PR res: [0x2000182d1:0x642:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91c25cb expref: -99 pid: 85580 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826958.552341:0:85580:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57c00/0xb3f83dad1b7e8255 lrc: 5/2,0 mode: PR/PR res: [0x2000182d1:0x642:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91c25cb expref: -99 pid: 85580 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826958.552346:0:85580:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57c00/0xb3f83dad1b7e8255 lrc: 4/1,0 mode: PR/PR res: [0x2000182d1:0x642:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91c25cb expref: -99 pid: 85580 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787826958.552353:0:85580:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff985610ce8b40 x1874669226989696/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 600/584 e 0 to 0 dl 1787826974 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787826958.552360:0:85580:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57c00/0xb3f83dad1b7e8255 lrc: 3/1,0 mode: PR/PR res: [0x2000182d1:0x642:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91c25cb expref: -99 pid: 85580 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826958.552366:0:85580:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8590033617/1602 (0 0) 00000080:00200000:0.0:1787826958.552370:0:85580:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:0.0:1787826958.552387:0:85580:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x2000182d1:0x642:0x0], updating i_size 0 00000080:00200000:0.0:1787826958.552389:0:85580:0:(namei.c:147:ll_iget()) got inode: [0x2000182d1:0x642:0x0](ffff9856102a5348): rc = 0 00000080:00200000:0.0:1787826958.552405:0:85580:0:(file.c:6854:ll_layout_conf()) [0x2000182d1:0x642:0x0]: layout version change: 4294967294 -> 4294967295 00000080:00010000:0.0:1787826958.552410:0:85580:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x2000182d1:0x642:0x0](ffff9856102a5348) for lock 0xb3f83dad1b7e8255 00000080:00010000:0.0:1787826958.552418:0:85580:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x2000182d1:0x642:0x0](ffff9856102a5348) 00000080:00010000:0.0:1787826958.552420:0:85580:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e8255 from it ffffab3ac1d57b20 00010000:00010000:0.0:1787826958.552422:0:85580:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57c00/0xb3f83dad1b7e8255 lrc: 3/1,0 mode: PR/PR res: [0x2000182d1:0x642:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91c25cb expref: -99 pid: 85580 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826958.552428:0:85580:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57c00/0xb3f83dad1b7e8255 lrc: 2/0,0 mode: PR/PR res: [0x2000182d1:0x642:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91c25cb expref: -99 pid: 85580 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:0.0:1787826958.552454:0:85580:0:(file.c:6313:ll_getattr_dentry()) START file fsa-oleg314-client.virtnet:[0x2000182d1:0x642:0x0](ffff9856102a5348), request_mask 2047, flags 0, foreign 0 00000080:00200000:0.0:1787826958.552461:0:85580:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x2000182d1:0x642:0x0](ffff9856102a5348),name=fsa-oleg314-client.virtnet 00000002:00010000:0.0:1787826958.552483:0:85580:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x2000182d1:0x642:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826958.552494:0:85580:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57c00/0xb3f83dad1b7e8255 lrc: 2/1,0 mode: PR/PR res: [0x2000182d1:0x642:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91c25cb expref: -99 pid: 85580 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826958.552504:0:85580:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57c00/0xb3f83dad1b7e8255 lrc: 2/1,0 mode: PR/PR res: [0x2000182d1:0x642:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91c25cb expref: -99 pid: 85580 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826958.552511:0:85580:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826958.552516:0:85580:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x2000182d1:0x642:0x0](ffff9856102a5348) 00000080:00010000:0.0:1787826958.552523:0:85580:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x2000182d1:0x642:0x0](ffff9856102a5348) for lock 0xb3f83dad1b7e8255 00000080:00010000:0.0:1787826958.552527:0:85580:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e8255 from it ffffab3ac1d57cb8 00010000:00010000:0.0:1787826958.552529:0:85580:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57c00/0xb3f83dad1b7e8255 lrc: 3/1,0 mode: PR/PR res: [0x2000182d1:0x642:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91c25cb expref: -99 pid: 85580 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826958.552534:0:85580:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57c00/0xb3f83dad1b7e8255 lrc: 2/0,0 mode: PR/PR res: [0x2000182d1:0x642:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91c25cb expref: -99 pid: 85580 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1787826958.552586:0:85580:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:0.0:1787826958.552592:0:85580:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826958.552597:0:85580:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7e824e lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91c25af expref: -99 pid: 85580 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826958.552602:0:85580:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7e824e lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91c25af expref: -99 pid: 85580 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826958.552607:0:85580:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826958.552609:0:85580:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:0.0:1787826958.552611:0:85580:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e824e 00000080:00010000:0.0:1787826958.552613:0:85580:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e824e from it ffffab3ac1d57c10 00010000:00010000:0.0:1787826958.552615:0:85580:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7e824e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91c25af expref: -99 pid: 85580 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826958.552619:0:85580:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7e824e lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91c25af expref: -99 pid: 85580 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1787826958.552623:0:85580:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:0.0:1787826958.552630:0:85580:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=fsa-oleg314-client.virtnet, flags=0 00000080:00200000:0.0:1787826958.552633:0:85580:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:0.0:1787826958.552636:0:85580:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826958.552639:0:85580:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7e824e lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91c25af expref: -99 pid: 85580 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1787826958.552643:0:85580:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7e824e lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91c25af expref: -99 pid: 85580 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1787826958.552647:0:85580:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826958.552648:0:85580:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:0.0:1787826958.552650:0:85580:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e824e 00000080:00010000:0.0:1787826958.552651:0:85580:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e824e from it ffffab3ac1d57d68 00010000:00010000:0.0:1787826958.552653:0:85580:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7e824e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91c25af expref: -99 pid: 85580 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1787826958.552657:0:85580:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7e824e lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91c25af expref: -99 pid: 85580 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:0.0:1787826958.552664:0:85580:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 3 00000080:00200000:0.0:1787826958.552668:0:85580:0:(namei.c:2418:ll_unlink()) VFS Op:name=fsa-oleg314-client.virtnet, dir=[0x200000007:0x1:0x0](ffff98561158e3c8) 00010000:00010000:0.0:1787826958.552683:0:85580:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7e824e lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x5aa6d504d91c25af expref: -99 pid: 85580 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787826958.552855:0:85580:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57c00/0xb3f83dad1b7e8255 lrc: 2/0,0 mode: PR/PR res: [0x2000182d1:0x642:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x5aa6d504d91c25cb expref: -99 pid: 85580 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1787826958.552862:0:85580:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x642:0x0]: losing layout lock 00010000:00010000:0.0:1787826958.552891:0:85580:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57c00/0xb3f83dad1b7e8255 lrc: 1/0,0 mode: --/PR res: [0x2000182d1:0x642:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x5aa6d504d91c25cb expref: -99 pid: 85580 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1787826958.552896:0:85580:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7e824e lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91c25af expref: -99 pid: 85580 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787826958.552900:0:85580:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 2 locks packed 00010000:00010000:0.0:1787826958.552902:0:85580:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57c00/0xb3f83dad1b7e8255 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x642:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x5aa6d504d91c25cb expref: -99 pid: 85580 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1787826958.552914:0:85580:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7e824e lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91c25af expref: -99 pid: 85580 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000020:00100000:0.0:1787826958.552926:0:85580:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:0.0:1787826958.552932:0:85580:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610ce8b40 x1874669226989952/t0(0) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826958.552947:0:85580:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610ce8b40 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:85580:1874669226989952:192.168.203.114@tcp:36:rm.0 00000100:00100000:0.0:1787826958.552999:0:85580:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607ff0900 going to sleep for 11 seconds 00000100:00100000:0.0:1787826958.555361:0:85580:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610ce8b40 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:85580:1874669226989952:192.168.203.114@tcp:36:rm.0 00000080:00200000:0.0:1787826958.555411:0:85580:0:(vvp_io.c:1885:vvp_io_init()) [0x2000182d1:0x642:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:0.0:1787826958.555420:0:85580:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x642:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787826958.555426:0:85580:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x642:0x0](ffff9856102a5348) 00010000:00010000:3.0:1787826959.157107:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826959.606554:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98560841de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669226990208:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787826959.606629:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98560841e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669226990464:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787826959.606657:0:2385:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [1->0] 00000100:00100000:2.0:1787826959.606663:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98560841cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2385:1874669226990080:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787826959.606688:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98560841d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669226990336:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787826959.606738:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98560841e1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669226990592:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787826959.608359:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98560841d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669226990336:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787826959.608373:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98560841e1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669226990592:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787826959.608383:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98560841de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669226990208:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787826959.608392:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98560841e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669226990464:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787826959.608421:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98560841cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2385:1874669226990080:192.168.203.114@tcp:400:kworker.0 00010000:00010000:1.0:1787826960.181069:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:0.0:1787826961.103894:0:85646:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: local REPLAY BARRIER on lustre-MDT0000 00010000:00010000:2.0:1787826961.204911:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826964.276964:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826964.724958:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608401e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:3c946890-15ca-497a-b838-fe1cba61f4dd:2384:1874669227144320:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787826964.725643:0:2384:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1787826964.725652:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856084012c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669227144448:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787826964.725834:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085512c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669227144704:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787826964.725871:0:2383:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1787826964.725877:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608551e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669227144576:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787826964.725902:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608552d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669227144832:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787826964.728156:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608401e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:3c946890-15ca-497a-b838-fe1cba61f4dd:2384:1874669227144320:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787826964.728919:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085512c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669227144704:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787826964.729208:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856084012c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669227144448:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787826964.729223:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608551e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669227144576:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787826964.729247:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608552d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669227144832:192.168.203.114@tcp:400:kworker.0 00010000:00010000:1.0:1787826965.300910:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826966.324834:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826967.348874:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826968.372860:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826969.396965:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826969.844931:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e821c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669227145088:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787826969.844932:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2385:1874669227144960:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787826969.845025:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669227145216:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787826969.845026:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e83c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669227145344:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787826969.845058:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669227145472:192.168.203.114@tcp:400:kworker.0 00000100:00080000:2.0:1787826969.846458:0:2386:0:(recover.c:205:ptlrpc_request_handle_notconn()) import lustre-MDT0000-mdc-ffff985602bc9800 of lustre-MDT0000_UUID@192.168.203.114@tcp abruptly disconnected: reconnecting 00000100:00100000:3.0:1787826969.846460:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2385:1874669227144960:192.168.203.114@tcp:400:kworker.0 00000100:00080000:2.0:1787826969.846467:0:2386:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from FULL to DISCONN 00000100:02000400:2.0:1787826969.846475:0:2386:0:(import.c:182:ptlrpc_set_import_discon()) lustre-MDT0000-mdc-ffff985602bc9800: Connection to lustre-MDT0000 (at 192.168.203.114@tcp) was lost; in progress operations using this service will wait for recovery to complete 00000100:00080000:2.0:1787826969.846480:0:2386:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826969.846487:0:2386:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3089 00000100:00100000:3.0:1787826969.846487:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669227145216:192.168.203.114@tcp:400:kworker.0 00000100:00080000:2.0:1787826969.846501:0:2386:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787826969.846506:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669227145472:192.168.203.114@tcp:400:kworker.0 00000008:00100000:2.0:1787826969.846520:0:2386:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787826969.846544:0:2386:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8f00 x1874669227145600/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826969.846577:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e821c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669227145088:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787826969.846600:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e83c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669227145344:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787826969.846607:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227145600:192.168.203.114@tcp:38:ptlrpcd_01_01.0 00000100:00080000:3.0:1787826969.847721:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826969.847727:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-19) 00000100:00100000:3.0:1787826969.847731:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227145600:192.168.203.114@tcp:38:ptlrpcd_01_01.0 00010000:00010000:1.0:1787826970.421360:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826971.444948:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826972.468869:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826973.492896:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826974.516856:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787826974.964890:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826974.964898:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787826974.964901:0:38:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787826974.964910:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3120 00000100:00080000:1.0:1787826974.964919:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 5s 00000100:00080000:1.0:1787826974.964924:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787826974.964936:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608552d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2383:1874669227145728:192.168.203.114@tcp:400:kworker.0 00000008:00100000:1.0:1787826974.964936:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787826974.964956:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985608551e00 x1874669227145856/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826974.965010:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085512c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669227145984:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787826974.965017:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608551e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227145856:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787826974.965045:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608551680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669227146112:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787826974.965076:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085503c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669227146240:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787826974.966768:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608551680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669227146112:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787826974.966772:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085512c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669227145984:192.168.203.114@tcp:400:kworker.0 00000100:00080000:3.0:1787826974.966788:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00100000:1.0:1787826974.966801:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085503c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669227146240:192.168.203.114@tcp:400:kworker.0 00000100:00080000:3.0:1787826974.966804:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-19) 00000100:00100000:3.0:1787826974.966819:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608551e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227145856:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787826975.540899:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826976.564951:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826977.589315:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826978.612878:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826979.636862:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787826980.084871:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826980.084878:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787826980.084882:0:38:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826980.084892:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3125 00000100:00080000:2.0:1787826980.084903:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 10s 00000100:00080000:2.0:1787826980.084911:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787826980.084921:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787826980.084936:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d83c0 x1874669227146496/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826980.084961:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:3c946890-15ca-497a-b838-fe1cba61f4dd:2386:1874669227146368:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787826980.085004:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227146496:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787826980.085017:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669227146624:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787826980.085058:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669227146880:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787826980.085066:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669227146752:192.168.203.114@tcp:400:kworker.0 00000100:00080000:0.0:1787826980.086176:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826980.086183:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-19) 00000100:00100000:0.0:1787826980.086189:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227146496:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787826980.086205:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669227146752:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787826980.086245:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669227146624:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787826980.086258:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669227146880:192.168.203.114@tcp:400:kworker.0 00010000:00010000:1.0:1787826980.660868:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826981.684870:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826982.709259:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826983.732897:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826984.758267:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826985.205013:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608550780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2383:1874669227147008:192.168.203.114@tcp:400:kworker.0 00000100:00080000:1.0:1787826985.205180:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826985.205186:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787826985.205190:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787826985.205197:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3130 00000100:00080000:1.0:1787826985.205204:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 15s 00000100:00080000:1.0:1787826985.205210:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787826985.205222:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787826985.205252:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985608550000 x1874669227147136/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826985.205314:0:2383:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1787826985.205317:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608550000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227147136:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787826985.205325:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608553840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669227147264:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787826985.205557:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608550b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669227147520:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787826985.205560:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608553c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669227147392:192.168.203.114@tcp:400:kworker.0 00000100:00080000:1.0:1787826985.206336:0:2383:0:(recover.c:205:ptlrpc_request_handle_notconn()) import MGC192.168.203.114@tcp of MGS@192.168.203.114@tcp abruptly disconnected: reconnecting 00000100:00080000:1.0:1787826985.206341:0:2383:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from FULL to DISCONN 00000100:02020000:1.0:1787826985.206345:0:2383:0:(import.c:186:ptlrpc_set_import_discon()) MGC192.168.203.114@tcp: Connection to MGS (at 192.168.203.114@tcp) was lost; in progress operations using this service will fail 00000100:00080000:1.0:1787826985.206349:0:2383:0:(import.c:192:ptlrpc_set_import_discon()) import MGS@192.168.203.114@tcp for MGC192.168.203.114@tcp not replayable, auto-deactivating 00000100:00080000:1.0:1787826985.206352:0:2383:0:(import.c:138:ptlrpc_deactivate_import_nolock()) setting import MGS INVALID 00000100:00100000:1.0:1787826985.206359:0:2383:0:(client.c:3589:ptlrpc_abort_inflight()) @@@ inflight req@ffff985608552d00 x1874669227145728/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787826990 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787826985.206369:0:2383:0:(client.c:3589:ptlrpc_abort_inflight()) @@@ inflight req@ffff9856140da940 x1874669227146368/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787826996 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787826985.206393:0:2383:0:(client.c:3589:ptlrpc_abort_inflight()) @@@ inflight req@ffff985608550780 x1874669227147008/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827001 ref 1 fl Rpc:RNQU/200/0 rc 0/-107 job:'kworker.0' uid:0 gid:0 projid:4294967295 10000000:01000000:1.0:1787826985.206403:0:2383:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808001 00000100:00080000:1.0:1787826985.206405:0:2383:0:(pinger.c:193:ptlrpc_pinger_ir_down()) IR down 10000000:01000000:1.0:1787826985.206406:0:2383:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808002 00000100:00080000:1.0:1787826985.206409:0:2383:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787826985.206413:0:2383:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3089 00000100:00080000:1.0:1787826985.206420:0:2383:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787826985.206437:0:2383:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985608552940 x1874669227147648/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826985.206464:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608550780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2383:1874669227147008:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787826985.206475:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608552d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2383:1874669227145728:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787826985.206555:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:3c946890-15ca-497a-b838-fe1cba61f4dd:2386:1874669227146368:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787826985.206626:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608552940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669227147648:192.168.203.114@tcp:250:ptlrpcd_00_00.0 00000100:00080000:0.0:1787826985.207971:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826985.207982:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-19) 00000100:00100000:0.0:1787826985.207986:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608550000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227147136:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787826985.208019:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608553840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669227147264:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787826985.208583:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608550b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669227147520:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787826985.208659:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608553c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669227147392:192.168.203.114@tcp:400:kworker.0 00000100:00080000:0.0:1787826985.210246:0:2382:0:(import.c:1132:ptlrpc_connect_interpret()) MGC192.168.203.114@tcp: connect to target with instance 0 10000000:01000000:0.0:1787826985.210259:0:2382:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808005 00000100:02000400:0.0:1787826985.210266:0:2382:0:(import.c:1284:ptlrpc_connect_interpret()) Evicted from MGS (at 192.168.203.114@tcp) after server handle changed from 0x5aa6d504d91b12c5 to 0x5aa6d504d91c6e64 00000100:00080000:0.0:1787826985.210270:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to EVICTED 00000100:00080000:0.0:1787826985.210276:0:2382:0:(import.c:1674:ptlrpc_import_recovery_state_machine()) evicted from MGS@192.168.203.114@tcp; invalidating 00000100:00100000:0.0:1787826985.210455:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608552940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669227147648:192.168.203.114@tcp:250:ptlrpcd_00_00.0 00000100:00080000:2.0:1787826985.210508:0:86057:0:(import.c:1598:ptlrpc_invalidate_import_thread()) thread invalidate import MGC192.168.203.114@tcp to MGS@192.168.203.114@tcp 00000100:00100000:2.0:1787826985.210513:0:86057:0:(import.c:323:ptlrpc_invalidate_import()) Sleeping 20s for inflight to error out 10000000:01000000:2.0:1787826985.210516:0:86057:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808003 00010000:00010000:2.0:1787826985.210530:0:86057:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.114@tcp lock: ffff985609eb1c00/0xb3f83dad1b7e4092 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x110e400000000 nid: local remote: 0x5aa6d504d91b1400 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:2.0:1787826985.210542:0:86057:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.114@tcp lock: ffff985609eb1c00/0xb3f83dad1b7e4092 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x110f400000000 nid: local remote: 0x5aa6d504d91b1400 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1787826985.210549:0:86057:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:2.0:1787826985.210568:0:86057:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1787826985.210577:0:86057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.114@tcp lock: ffff985609eb1c00/0xb3f83dad1b7e4092 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x590f400000000 nid: local remote: 0x5aa6d504d91b1400 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826985.210590:0:86057:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.114@tcp lock: ffff985610e07200/0xb3f83dad1b7e40a0 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x110e400000000 nid: local remote: 0x5aa6d504d91b140e expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:2.0:1787826985.210735:0:86057:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.114@tcp lock: ffff985610e07200/0xb3f83dad1b7e40a0 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x110f400000000 nid: local remote: 0x5aa6d504d91b140e expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1787826985.210743:0:86057:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:2.0:1787826985.210751:0:86057:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1787826985.210756:0:86057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.114@tcp lock: ffff985610e07200/0xb3f83dad1b7e40a0 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x590f400000000 nid: local remote: 0x5aa6d504d91b140e expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826985.210773:0:86057:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.114@tcp lock: ffff985610f40c00/0xb3f83dad1b7e408b lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x110e400000000 nid: local remote: 0x5aa6d504d91b13f9 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:2.0:1787826985.210780:0:86057:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.114@tcp lock: ffff985610f40c00/0xb3f83dad1b7e408b lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x110f400000000 nid: local remote: 0x5aa6d504d91b13f9 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1787826985.210786:0:86057:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x2:0x0].0x0 (lustre) 00010000:00010000:2.0:1787826985.210800:0:86057:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1787826985.210804:0:86057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.114@tcp lock: ffff985610f40c00/0xb3f83dad1b7e408b lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x590f400000000 nid: local remote: 0x5aa6d504d91b13f9 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826985.210817:0:86057:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.114@tcp lock: ffff985610f41400/0xb3f83dad1b7e4099 lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x110e400000000 nid: local remote: 0x5aa6d504d91b1407 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:2.0:1787826985.210824:0:86057:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.114@tcp lock: ffff985610f41400/0xb3f83dad1b7e4099 lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x110f400000000 nid: local remote: 0x5aa6d504d91b1407 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1787826985.210830:0:86057:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x736d61726170:0x3:0x0].0x0 (params) 00010000:00010000:2.0:1787826985.210836:0:86057:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1787826985.210841:0:86057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.114@tcp lock: ffff985610f41400/0xb3f83dad1b7e4099 lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x590f400000000 nid: local remote: 0x5aa6d504d91b1407 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787826985.210856:0:86057:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from EVICTED to RECOVER 00000100:00080000:2.0:1787826985.210860:0:86057:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from RECOVER to FULL 10000000:01000000:2.0:1787826985.210862:0:86057:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808004 00000100:00080000:2.0:1787826985.210870:0:86057:0:(pinger.c:186:ptlrpc_pinger_ir_up()) IR up 00000100:02000000:2.0:1787826985.210932:0:86057:0:(import.c:1743:ptlrpc_import_recovery_state_machine()) MGC192.168.203.114@tcp: Connection restored to 192.168.203.114@tcp (at 192.168.203.114@tcp) 00010000:00010000:1.0:1787826985.780990:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:0.0:1787826986.420972:0:15166:0:(mgc_request.c:611:do_requeue()) updating log lustre-cliir 10000000:01000000:0.0:1787826986.420979:0:15166:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-cliir-ffff985602bc9800 from 204 10000000:01000000:0.0:1787826986.420985:0:15166:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-cliir (res 0x65727473756c) 00010000:00010000:0.0:1787826986.421034:0:15166:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.114@tcp lock: ffff985607f56e00/0xb3f83dad1b7e8d4c lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826986.421045:0:15166:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.114@tcp lock: ffff985607f56e00/0xb3f83dad1b7e8d4c lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826986.421061:0:15166:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.114@tcp lock: ffff985607f56e00/0xb3f83dad1b7e8d4c lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826986.421102:0:15166:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607ce4f00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669227147776:192.168.203.114@tcp:101:ll_cfg_requeue.0 00000100:00100000:0.0:1787826986.421193:0:15166:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607ff0300 going to sleep for 11 seconds 00000100:00100000:0.0:1787826986.422993:0:15166:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607ce4f00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669227147776:192.168.203.114@tcp:101:ll_cfg_requeue.0 00010000:00010000:0.0:1787826986.423019:0:15166:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.114@tcp lock: ffff985607f56e00/0xb3f83dad1b7e8d4c lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x5aa6d504d91c6f21 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826986.423031:0:15166:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.114@tcp lock: ffff985607f56e00/0xb3f83dad1b7e8d4c lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x5aa6d504d91c6f21 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826986.423121:0:15166:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607ce4f00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669227147904:192.168.203.114@tcp:256:ll_cfg_requeue.0 00000100:00100000:0.0:1787826986.423176:0:15166:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607ff0300 going to sleep for 11 seconds 00000100:00100000:0.0:1787826986.425155:0:15166:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607ce4f00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669227147904:192.168.203.114@tcp:256:ll_cfg_requeue.0 00000100:00080000:0.0:1787826986.425211:0:15166:0:(import.c:214:ptlrpc_set_import_discon()) mdc: import ffff985611182000 already not connected (conn 294, was 0): DISCONN 00010000:00080000:0.0:1787826986.425223:0:15166:0:(ldlm_lib.c:85:import_set_conn()) imp ffff985611182000@lustre-MDT0000-mdc-ffff985602bc9800: found existing conn 192.168.203.114@tcp, moved to head 00000100:00080000:0.0:1787826986.425228:0:15166:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787826986.425232:0:15166:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 0 00000100:00080000:0.0:1787826986.425239:0:15166:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787826986.425250:0:15166:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787826986.425273:0:15166:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607ce5e00 x1874669227148032/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 10000000:01000000:0.0:1787826986.425330:0:15166:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.114@tcp: configuration from log 'lustre-cliir' succeeded (0). 00000100:00100000:2.0:1787826986.425332:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607ce5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227148032:192.168.203.114@tcp:38:ll_cfg_requeue.0 00010000:00010000:0.0:1787826986.425339:0:15166:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.114@tcp lock: ffff985607f56e00/0xb3f83dad1b7e8d4c lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x5aa6d504d91c6f21 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826986.425347:0:15166:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.114@tcp lock: ffff985607f56e00/0xb3f83dad1b7e8d4c lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x5aa6d504d91c6f21 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787826986.425356:0:15166:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 10000000:01000000:0.0:1787826986.425358:0:15166:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff985602bc9800 from 52 10000000:01000000:0.0:1787826986.425361:0:15166:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00010000:00010000:0.0:1787826986.425384:0:15166:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.114@tcp lock: ffff985607f56200/0xb3f83dad1b7e8d53 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826986.425391:0:15166:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.114@tcp lock: ffff985607f56200/0xb3f83dad1b7e8d53 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826986.425401:0:15166:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.114@tcp lock: ffff985607f56200/0xb3f83dad1b7e8d53 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826986.425416:0:15166:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607ce4f00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669227148160:192.168.203.114@tcp:101:ll_cfg_requeue.0 00000100:00100000:0.0:1787826986.425447:0:15166:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607ff0300 going to sleep for 11 seconds 00010000:00080000:2.0:1787826986.426623:0:2382:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=46162) req@ffff985607ce5e00 x1874669227148032/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/416 e 0 to 0 dl 1787826996 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826986.426641:0:2382:0:(import.c:1132:ptlrpc_connect_interpret()) lustre-MDT0000-mdc-ffff985602bc9800: connect to target with instance 51 00000100:00080000:2.0:1787826986.426658:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to REPLAY 00000100:00080000:2.0:1787826986.426779:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:2.0:1787826986.426782:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366800 gen 9 00000100:00100000:2.0:1787826986.426789:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610cea1c0 x1874669226989440/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 528/448 e 0 to 0 dl 1787826974 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826986.426798:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366800 last 0 00000100:00080000:2.0:1787826986.426805:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985610cea1c0 x1874669226989440/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 528/448 e 0 to 0 dl 1787826974 ref 1 fl New:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826986.426817:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607ce5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227148032:192.168.203.114@tcp:38:ll_cfg_requeue.0 00000100:00100000:2.0:1787826986.426839:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610cea1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669226989440:192.168.203.114@tcp:36:mcreate.0 00000100:00100000:0.0:1787826986.427144:0:15166:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607ce4f00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669227148160:192.168.203.114@tcp:101:ll_cfg_requeue.0 00010000:00010000:0.0:1787826986.427163:0:15166:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.114@tcp lock: ffff985607f56200/0xb3f83dad1b7e8d53 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x5aa6d504d91c6f28 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826986.427174:0:15166:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.114@tcp lock: ffff985607f56200/0xb3f83dad1b7e8d53 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x5aa6d504d91c6f28 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826986.427235:0:15166:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607ce4f00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669227148288:192.168.203.114@tcp:501:ll_cfg_requeue.0 00000100:00100000:0.0:1787826986.427273:0:15166:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607ff0300 going to sleep for 11 seconds 00000100:00100000:0.0:1787826986.431528:0:15166:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607ce4f00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669227148288:192.168.203.114@tcp:501:ll_cfg_requeue.0 00000100:00100000:0.0:1787826986.431594:0:15166:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607ce4f00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669227148416:192.168.203.114@tcp:503:ll_cfg_requeue.0 00000100:00100000:0.0:1787826986.431732:0:15166:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607ff0300 going to sleep for 11 seconds 00000100:00100000:0.0:1787826986.435186:0:15166:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607ce4f00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669227148416:192.168.203.114@tcp:503:ll_cfg_requeue.0 00000040:00080000:3.0:1787826986.436918:0:86103:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x3:0x0] index 64768 count 51 00000020:01000000:0.0:1787826986.437028:0:15166:0:(obd_config.c:2145:class_config_parse_llog()) Processed log lustre-client gen 52-51 (rc=0) 10000000:01000000:0.0:1787826986.437052:0:15166:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.114@tcp: configuration from log 'lustre-client' succeeded (0). 00010000:00010000:0.0:1787826986.437062:0:15166:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.114@tcp lock: ffff985607f56200/0xb3f83dad1b7e8d53 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x5aa6d504d91c6f28 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826986.437071:0:15166:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.114@tcp lock: ffff985607f56200/0xb3f83dad1b7e8d53 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x5aa6d504d91c6f28 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787826986.437088:0:15166:0:(mgc_request.c:611:do_requeue()) updating log params 10000000:01000000:0.0:1787826986.437091:0:15166:0:(mgc_request.c:1802:mgc_process_log()) Process log params-ffff985602bc9800 from 7 10000000:01000000:0.0:1787826986.437094:0:15166:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for params (res 0x736d61726170) 00010000:00010000:0.0:1787826986.437128:0:15166:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.114@tcp lock: ffff985607f56600/0xb3f83dad1b7e8d5a lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826986.437135:0:15166:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.114@tcp lock: ffff985607f56600/0xb3f83dad1b7e8d5a lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826986.437144:0:15166:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.114@tcp lock: ffff985607f56600/0xb3f83dad1b7e8d5a lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826986.437171:0:15166:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607ce4f00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669227148544:192.168.203.114@tcp:101:ll_cfg_requeue.0 00000100:00100000:0.0:1787826986.437243:0:15166:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607ff0300 going to sleep for 11 seconds 00000100:00100000:0.0:1787826986.438918:0:15166:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607ce4f00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669227148544:192.168.203.114@tcp:101:ll_cfg_requeue.0 00010000:00010000:0.0:1787826986.438938:0:15166:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.114@tcp lock: ffff985607f56600/0xb3f83dad1b7e8d5a lrc: 4/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x5aa6d504d91c6f2f expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826986.438949:0:15166:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.114@tcp lock: ffff985607f56600/0xb3f83dad1b7e8d5a lrc: 4/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x5aa6d504d91c6f2f expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826986.439068:0:15166:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607ce4f00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669227148672:192.168.203.114@tcp:501:ll_cfg_requeue.0 00000100:00100000:0.0:1787826986.439136:0:15166:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607ff0300 going to sleep for 11 seconds 00000100:00100000:0.0:1787826986.440502:0:15166:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607ce4f00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669227148672:192.168.203.114@tcp:501:ll_cfg_requeue.0 00000100:00100000:0.0:1787826986.440544:0:15166:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607ce4f00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669227148800:192.168.203.114@tcp:503:ll_cfg_requeue.0 00000100:00100000:0.0:1787826986.440609:0:15166:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607ff0300 going to sleep for 11 seconds 00000100:00100000:0.0:1787826986.442156:0:15166:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607ce4f00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669227148800:192.168.203.114@tcp:503:ll_cfg_requeue.0 00000040:00080000:3.0:1787826986.442946:0:86104:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x2:0x0] index 64768 count 7 00000020:01000000:0.0:1787826986.443010:0:15166:0:(obd_config.c:2145:class_config_parse_llog()) Processed log params gen 7-6 (rc=0) 10000000:01000000:0.0:1787826986.443033:0:15166:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.114@tcp: configuration from log 'params' succeeded (0). 00010000:00010000:0.0:1787826986.443042:0:15166:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.114@tcp lock: ffff985607f56600/0xb3f83dad1b7e8d5a lrc: 3/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x5aa6d504d91c6f2f expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826986.443052:0:15166:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.114@tcp lock: ffff985607f56600/0xb3f83dad1b7e8d5a lrc: 2/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x5aa6d504d91c6f2f expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787826986.443066:0:15166:0:(mgc_request.c:611:do_requeue()) updating log lustre-sptlrpc 10000000:01000000:0.0:1787826986.443069:0:15166:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-sptlrpc-ffff985606a30000 from 1 10000000:01000000:0.0:1787826986.443072:0:15166:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-sptlrpc (res 0x65727473756c) 00010000:00010000:0.0:1787826986.443108:0:15166:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.114@tcp lock: ffff985607f57e00/0xb3f83dad1b7e8d61 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826986.443116:0:15166:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.114@tcp lock: ffff985607f57e00/0xb3f83dad1b7e8d61 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826986.443125:0:15166:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.114@tcp lock: ffff985607f57e00/0xb3f83dad1b7e8d61 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826986.443142:0:15166:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607ce4f00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669227148928:192.168.203.114@tcp:101:ll_cfg_requeue.0 00000100:00100000:0.0:1787826986.443210:0:15166:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607ff0300 going to sleep for 11 seconds 00000100:00100000:0.0:1787826986.444196:0:15166:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607ce4f00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669227148928:192.168.203.114@tcp:101:ll_cfg_requeue.0 00010000:00010000:0.0:1787826986.444214:0:15166:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.114@tcp lock: ffff985607f57e00/0xb3f83dad1b7e8d61 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0x5aa6d504d91c6f36 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826986.444224:0:15166:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.114@tcp lock: ffff985607f57e00/0xb3f83dad1b7e8d61 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1000000000000 nid: local remote: 0x5aa6d504d91c6f36 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826986.444273:0:15166:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607ce4f00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669227149056:192.168.203.114@tcp:501:ll_cfg_requeue.0 00000100:00100000:0.0:1787826986.444318:0:15166:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607ff0300 going to sleep for 11 seconds 00000100:00100000:0.0:1787826986.446153:0:15166:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607ce4f00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669227149056:192.168.203.114@tcp:501:ll_cfg_requeue.0 10000000:01000000:0.0:1787826986.446185:0:15166:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.114@tcp: configuration from log 'lustre-sptlrpc' failed (-2). 00010000:00010000:0.0:1787826986.446194:0:15166:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.114@tcp lock: ffff985607f57e00/0xb3f83dad1b7e8d61 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0x5aa6d504d91c6f36 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826986.446203:0:15166:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.114@tcp lock: ffff985607f57e00/0xb3f83dad1b7e8d61 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0x5aa6d504d91c6f36 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826986.806374:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826987.828830:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826988.853390:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200008:1.0:1787826988.980582:0:86156:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:1.0:1787826988.980595:0:86156:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826988.980615:0:86156:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826988.980662:0:86156:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe6a00/0xb3f83dad1b7e8d68 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 86156 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826988.980670:0:86156:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe6a00/0xb3f83dad1b7e8d68 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 86156 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826988.980677:0:86156:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe6a00/0xb3f83dad1b7e8d68 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 86156 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1787826988.980763:0:86156:0:(client.c:1845:ptlrpc_send_new_req()) @@@ req waiting for recovery: (FULL != REPLAY) req@ffff985608550780 x1874669227149184/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 576/1152 e 0 to 0 dl 0 ref 2 fl Rpc:WQU/600/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:0 00000100:00100000:1.0:1787826988.980774:0:86156:0:(client.c:2715:ptlrpc_set_wait()) set ffff985612993b40 going to sleep for 0 seconds 00010000:00010000:3.0:1787826989.876837:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826990.004833:0:86156:0:(client.c:2715:ptlrpc_set_wait()) set ffff985612993b40 going to sleep for 0 seconds 00000100:00080000:0.0:1787826990.325535:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level REPLAY/5 force 0 force_next 1 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787826990.325542:0:37:0:(pinger.c:247:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: REPLAY) 00000100:00100000:0.0:1787826990.325590:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607ce5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669227149440:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787826990.325683:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607ce7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669227149696:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787826990.325735:0:2384:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1787826990.325746:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607ce6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:3c946890-15ca-497a-b838-fe1cba61f4dd:2384:1874669227149312:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787826990.325777:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607ce43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669227149568:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787826990.327539:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607ce6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:3c946890-15ca-497a-b838-fe1cba61f4dd:2384:1874669227149312:192.168.203.114@tcp:400:kworker.0 00010000:00010000:0.0:1787826990.327584:0:2384:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 0/0 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1787826990.327592:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607ce43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669227149568:192.168.203.114@tcp:400:kworker.0 00010000:00010000:0.0:1787826990.327615:0:2384:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 0/0 locks to regular blp list, 2 blwis in pool 00000100:00100000:0.0:1787826990.327619:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607ce5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669227149440:192.168.203.114@tcp:400:kworker.0 00010000:00010000:0.0:1787826990.327786:0:2384:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 0/0 locks to regular blp list, 3 blwis in pool 00000100:00100000:0.0:1787826990.327792:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607ce7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669227149696:192.168.203.114@tcp:400:kworker.0 00010000:00010000:3.0:1787826990.327976:0:79887:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 0 locks of 0 total in blp. (2 blwis in pool) 00010000:00010000:1.0:1787826990.327987:0:33342:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 0 locks of 0 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1787826990.328008:0:79887:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 0 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787826990.328010:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826990.328020:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826990.328023:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826990.328025:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826990.328028:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826990.328037:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826990.328038:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787826990.728442:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985610cea1c0 x1874669226989440/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 528/448 e 0 to 0 dl 1787827002 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826990.728469:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:2.0:1787826990.728472:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366800 gen 9 00000100:00100000:2.0:1787826990.728478:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610cea1c0 x1874669226989440/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 528/448 e 0 to 0 dl 1787827002 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826990.728486:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366800 last 214748366801 00000100:00080000:2.0:1787826990.728495:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787826974 ref 1 fl New:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826990.728505:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610cea1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669226989440:192.168.203.114@tcp:36:mcreate.0 00000100:00100000:2.0:1787826990.728520:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610ce8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669226989952:192.168.203.114@tcp:36:rm.0 00000100:00100000:2.0:1787826990.730020:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:2.0:1787826990.730064:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 2 fl Rpc:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826990.730081:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826990.730088:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:2.0:1787826990.730093:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:2.0:1787826990.730097:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826990.730103:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366802 00000100:00080000:2.0:1787826990.730112:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9856084870c0 x1874669226990976/t214748366803(214748366803) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826990.730123:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610ce8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669226989952:192.168.203.114@tcp:36:rm.0 00000100:00100000:2.0:1787826990.730150:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856084870c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669226990976:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:2.0:1787826990.731922:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff9856084870c0 x1874669226990976/t214748366803(214748366803) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826990.731933:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:2.0:1787826990.731935:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:2.0:1787826990.731940:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826990.731947:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366803 00000100:00080000:2.0:1787826990.731954:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985608484780 x1874669226991360/t214748366804(214748366804) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826990.731970:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856084870c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669226990976:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:2.0:1787826990.731989:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608484780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669226991360:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:2.0:1787826990.733629:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985608484780 x1874669226991360/t214748366804(214748366804) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826990.733641:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:2.0:1787826990.733644:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:2.0:1787826990.733650:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826990.733657:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366804 00000100:00080000:2.0:1787826990.733663:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985608485a40 x1874669226991744/t214748366805(214748366805) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826990.733673:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608484780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669226991360:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:2.0:1787826990.733689:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608485a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669226991744:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:2.0:1787826990.735288:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985608485a40 x1874669226991744/t214748366805(214748366805) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826990.735300:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:2.0:1787826990.735302:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:2.0:1787826990.735307:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826990.735315:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366805 00000100:00080000:2.0:1787826990.735322:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985608487480 x1874669226992128/t214748366806(214748366806) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826990.735331:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608485a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669226991744:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:2.0:1787826990.735345:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608487480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669226992128:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:2.0:1787826990.736985:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985608487480 x1874669226992128/t214748366806(214748366806) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826990.736998:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:2.0:1787826990.737001:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:2.0:1787826990.737006:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826990.737021:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366806 00000100:00080000:2.0:1787826990.737028:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9856084843c0 x1874669226992512/t214748366807(214748366807) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826990.737038:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608487480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669226992128:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:2.0:1787826990.737066:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856084843c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669226992512:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:2.0:1787826990.738771:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff9856084843c0 x1874669226992512/t214748366807(214748366807) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826990.738783:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:2.0:1787826990.738785:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:2.0:1787826990.738791:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826990.738798:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366807 00000100:00080000:2.0:1787826990.738805:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985608485680 x1874669226992896/t214748366808(214748366808) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826990.738822:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856084843c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669226992512:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:2.0:1787826990.738838:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608485680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669226992896:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:2.0:1787826990.740372:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985608485680 x1874669226992896/t214748366808(214748366808) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826990.740384:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:2.0:1787826990.740387:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:2.0:1787826990.740392:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826990.740409:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366808 00000100:00080000:2.0:1787826990.740419:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985608486d00 x1874669226993280/t214748366809(214748366809) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826990.740429:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608485680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669226992896:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:2.0:1787826990.740444:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608486d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669226993280:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:2.0:1787826990.741947:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985608486d00 x1874669226993280/t214748366809(214748366809) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826990.741959:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:2.0:1787826990.741961:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:2.0:1787826990.741975:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826990.741984:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366809 00000100:00080000:2.0:1787826990.741992:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985610ce9e00 x1874669226993664/t214748366810(214748366810) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826990.742002:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608486d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669226993280:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:2.0:1787826990.742016:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610ce9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669226993664:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:2.0:1787826990.743932:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985610ce9e00 x1874669226993664/t214748366810(214748366810) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826990.743945:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:2.0:1787826990.743949:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:2.0:1787826990.743955:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826990.743962:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366810 00000100:00080000:2.0:1787826990.743971:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985610ce92c0 x1874669226994048/t214748366811(214748366811) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826990.743981:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610ce9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669226993664:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:2.0:1787826990.743998:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610ce92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669226994048:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:2.0:1787826990.745515:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985610ce92c0 x1874669226994048/t214748366811(214748366811) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826990.745524:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:2.0:1787826990.745527:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:2.0:1787826990.745533:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826990.745539:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366811 00000100:00080000:2.0:1787826990.745548:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985610ce9680 x1874669226994432/t214748366812(214748366812) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826990.745558:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610ce92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669226994048:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:2.0:1787826990.745580:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610ce9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669226994432:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:2.0:1787826990.746867:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985610ce9680 x1874669226994432/t214748366812(214748366812) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826990.746876:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:2.0:1787826990.746879:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:2.0:1787826990.746883:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826990.746888:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366812 00000100:00080000:2.0:1787826990.746893:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985610ce83c0 x1874669226994816/t214748366813(214748366813) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826990.746899:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610ce9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669226994432:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:2.0:1787826990.746909:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610ce83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669226994816:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:2.0:1787826990.748735:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985610ce83c0 x1874669226994816/t214748366813(214748366813) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826990.748751:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:2.0:1787826990.748754:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:2.0:1787826990.748760:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826990.748769:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366813 00000100:00080000:2.0:1787826990.748779:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98560841e1c0 x1874669226995200/t214748366814(214748366814) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826990.748791:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610ce83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669226994816:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:2.0:1787826990.748810:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98560841e1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669226995200:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:2.0:1787826990.751459:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98560841e1c0 x1874669226995200/t214748366814(214748366814) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826990.751476:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:2.0:1787826990.751480:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:2.0:1787826990.751487:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826990.751495:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366814 00000100:00080000:2.0:1787826990.751514:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98560841d2c0 x1874669226995584/t214748366815(214748366815) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826990.751526:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98560841e1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669226995200:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:2.0:1787826990.751548:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98560841d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669226995584:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:2.0:1787826990.753390:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98560841d2c0 x1874669226995584/t214748366815(214748366815) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826990.753400:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:2.0:1787826990.753403:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:2.0:1787826990.753408:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826990.753415:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366815 00000100:00080000:2.0:1787826990.753424:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98560841e940 x1874669226995968/t214748366816(214748366816) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826990.753434:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98560841d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669226995584:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:2.0:1787826990.753449:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98560841e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669226995968:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:2.0:1787826990.754892:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98560841e940 x1874669226995968/t214748366816(214748366816) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826990.754903:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:2.0:1787826990.754906:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:2.0:1787826990.754911:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826990.754918:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366816 00000100:00080000:2.0:1787826990.754927:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98560841cb40 x1874669226996352/t214748366817(214748366817) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826990.754937:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98560841e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669226995968:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:2.0:1787826990.754950:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98560841cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669226996352:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:2.0:1787826990.757138:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98560841cb40 x1874669226996352/t214748366817(214748366817) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826990.757153:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:2.0:1787826990.757156:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:2.0:1787826990.757162:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826990.757170:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366817 00000100:00080000:2.0:1787826990.757179:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98560841fc00 x1874669226996736/t214748366818(214748366818) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826990.757191:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98560841cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669226996352:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:2.0:1787826990.757210:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98560841fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669226996736:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:2.0:1787826990.758960:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98560841fc00 x1874669226996736/t214748366818(214748366818) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826990.758975:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:2.0:1787826990.758979:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:2.0:1787826990.758985:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826990.758994:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366818 00000100:00080000:2.0:1787826990.759006:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98560841f840 x1874669226997120/t214748366819(214748366819) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826990.759019:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98560841fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669226996736:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:2.0:1787826990.759038:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98560841f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669226997120:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:2.0:1787826990.760751:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98560841f840 x1874669226997120/t214748366819(214748366819) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826990.760765:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:2.0:1787826990.760768:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:2.0:1787826990.760773:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826990.760781:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366819 00000100:00080000:2.0:1787826990.760790:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98560841c000 x1874669226997504/t214748366820(214748366820) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826990.760802:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98560841f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669226997120:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:2.0:1787826990.760819:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98560841c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669226997504:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:2.0:1787826990.762553:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98560841c000 x1874669226997504/t214748366820(214748366820) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826990.762565:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:2.0:1787826990.762569:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:2.0:1787826990.762575:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826990.762583:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366820 00000100:00080000:2.0:1787826990.762592:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98560841c780 x1874669226997888/t214748366821(214748366821) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826990.762602:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98560841c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669226997504:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:2.0:1787826990.762618:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98560841c780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669226997888:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:2.0:1787826990.764338:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98560841c780 x1874669226997888/t214748366821(214748366821) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826990.764361:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:2.0:1787826990.764364:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:2.0:1787826990.764371:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826990.764377:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366821 00000100:00080000:2.0:1787826990.764387:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98560841f0c0 x1874669226998272/t214748366822(214748366822) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826990.764397:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98560841c780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669226997888:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:2.0:1787826990.764416:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98560841f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669226998272:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:2.0:1787826990.766222:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98560841f0c0 x1874669226998272/t214748366822(214748366822) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826990.766233:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:2.0:1787826990.766235:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:2.0:1787826990.766239:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826990.766244:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366822 00000100:00080000:2.0:1787826990.766251:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98560841da40 x1874669226998656/t214748366823(214748366823) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826990.766258:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98560841f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669226998272:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:2.0:1787826990.766272:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98560841da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669226998656:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:2.0:1787826990.767935:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98560841da40 x1874669226998656/t214748366823(214748366823) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826990.767947:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:2.0:1787826990.767950:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:2.0:1787826990.767955:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826990.767963:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366823 00000100:00080000:2.0:1787826990.767972:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98560841f480 x1874669226999040/t214748366824(214748366824) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826990.767982:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98560841da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669226998656:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:2.0:1787826990.767996:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98560841f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669226999040:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:2.0:1787826990.772407:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98560841f480 x1874669226999040/t214748366824(214748366824) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826990.772421:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:2.0:1787826990.772424:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:2.0:1787826990.772431:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826990.772439:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366824 00000100:00080000:2.0:1787826990.772455:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98560841c3c0 x1874669226999424/t214748366825(214748366825) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826990.772464:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98560841f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669226999040:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:2.0:1787826990.772479:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98560841c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669226999424:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:2.0:1787826990.774979:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98560841c3c0 x1874669226999424/t214748366825(214748366825) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826990.774990:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:2.0:1787826990.774993:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:2.0:1787826990.774998:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826990.775005:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366825 00000100:00080000:2.0:1787826990.775014:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98560841d680 x1874669226999808/t214748366826(214748366826) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826990.775024:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98560841c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669226999424:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:2.0:1787826990.775037:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98560841d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669226999808:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:2.0:1787826990.776779:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98560841d680 x1874669226999808/t214748366826(214748366826) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826990.776797:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:2.0:1787826990.776801:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:2.0:1787826990.776809:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826990.776817:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366826 00000100:00080000:2.0:1787826990.776830:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98560841ed00 x1874669227000192/t214748366827(214748366827) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826990.776845:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98560841d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669226999808:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:2.0:1787826990.776866:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98560841ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227000192:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:2.0:1787826990.778752:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98560841ed00 x1874669227000192/t214748366827(214748366827) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826990.778766:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:2.0:1787826990.778769:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:2.0:1787826990.778775:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826990.778783:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366827 00000100:00080000:2.0:1787826990.778792:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98560841e580 x1874669227000576/t214748366828(214748366828) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826990.778803:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98560841ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227000192:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:2.0:1787826990.778817:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98560841e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227000576:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:2.0:1787826990.780507:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98560841e580 x1874669227000576/t214748366828(214748366828) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826990.780518:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:2.0:1787826990.780521:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:2.0:1787826990.780526:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826990.780534:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366828 00000100:00080000:2.0:1787826990.780546:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98560841de00 x1874669227000960/t214748366829(214748366829) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826990.780556:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98560841e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227000576:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:2.0:1787826990.780569:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98560841de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227000960:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.782246:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98560841de00 x1874669227000960/t214748366829(214748366829) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.782258:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.782261:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.782267:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.782274:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366829 00000100:00080000:0.0:1787826990.782284:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98560841cf00 x1874669227001344/t214748366830(214748366830) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.782294:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98560841de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227000960:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.782305:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98560841cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227001344:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.784746:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98560841cf00 x1874669227001344/t214748366830(214748366830) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.784759:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.784761:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.784766:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.784775:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366830 00000100:00080000:0.0:1787826990.784784:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9856140db840 x1874669227001728/t214748366831(214748366831) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.784795:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98560841cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227001344:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.784810:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227001728:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.786829:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff9856140db840 x1874669227001728/t214748366831(214748366831) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.786840:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.786843:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.786848:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.786855:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366831 00000100:00080000:0.0:1787826990.786864:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9856140da1c0 x1874669227002112/t214748366832(214748366832) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.786880:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227001728:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.786896:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227002112:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.788689:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff9856140da1c0 x1874669227002112/t214748366832(214748366832) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.788730:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.788744:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.788750:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.788756:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366832 00000100:00080000:0.0:1787826990.788765:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9856140dad00 x1874669227002496/t214748366833(214748366833) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.788775:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227002112:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.788792:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227002496:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.790841:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff9856140dad00 x1874669227002496/t214748366833(214748366833) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.790853:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.790857:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.790863:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.790870:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366833 00000100:00080000:0.0:1787826990.790879:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985610ce9a40 x1874669227002880/t214748366834(214748366834) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.790890:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227002496:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.790906:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610ce9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227002880:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.793268:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985610ce9a40 x1874669227002880/t214748366834(214748366834) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.793281:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.793283:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.793288:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.793295:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366834 00000100:00080000:0.0:1787826990.793306:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985610ceb0c0 x1874669227003264/t214748366835(214748366835) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.793316:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610ce9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227002880:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.793331:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610ceb0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227003264:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.795836:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985610ceb0c0 x1874669227003264/t214748366835(214748366835) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.795849:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.795853:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.795857:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.795863:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366835 00000100:00080000:0.0:1787826990.795872:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985610ce8780 x1874669227003648/t214748366836(214748366836) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.795880:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610ceb0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227003264:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.795892:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610ce8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227003648:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.798100:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985610ce8780 x1874669227003648/t214748366836(214748366836) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.798113:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.798116:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.798121:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.798129:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366836 00000100:00080000:0.0:1787826990.798139:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985610ce8000 x1874669227004032/t214748366837(214748366837) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.798150:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610ce8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227003648:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.798165:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610ce8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227004032:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.800132:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985610ce8000 x1874669227004032/t214748366837(214748366837) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.800145:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.800149:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.800155:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.800162:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366837 00000100:00080000:0.0:1787826990.800173:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985610cead00 x1874669227004416/t214748366838(214748366838) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.800183:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610ce8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227004032:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.800198:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610cead00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227004416:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.802503:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985610cead00 x1874669227004416/t214748366838(214748366838) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.802515:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.802518:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.802523:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.802530:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366838 00000100:00080000:0.0:1787826990.802541:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985610ceb480 x1874669227004800/t214748366839(214748366839) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.802552:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610cead00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227004416:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.802579:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610ceb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227004800:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.804330:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985610ceb480 x1874669227004800/t214748366839(214748366839) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.804341:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.804344:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.804349:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.804357:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366839 00000100:00080000:0.0:1787826990.804376:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985610ce8f00 x1874669227005184/t214748366840(214748366840) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.804385:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610ceb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227004800:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.804399:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610ce8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227005184:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.806148:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985610ce8f00 x1874669227005184/t214748366840(214748366840) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.806158:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.806161:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.806165:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.806172:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366840 00000100:00080000:0.0:1787826990.806183:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985610cea580 x1874669227005568/t214748366841(214748366841) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.806193:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610ce8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227005184:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.806208:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610cea580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227005568:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.807843:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985610cea580 x1874669227005568/t214748366841(214748366841) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.807855:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.807858:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.807862:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.807868:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366841 00000100:00080000:0.0:1787826990.807877:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985610ceb840 x1874669227005952/t214748366842(214748366842) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.807887:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610cea580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227005568:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.807900:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610ceb840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227005952:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.809926:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985610ceb840 x1874669227005952/t214748366842(214748366842) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.809935:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.809936:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.809939:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.809944:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366842 00000100:00080000:0.0:1787826990.809951:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985610cebc00 x1874669227006336/t214748366843(214748366843) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.809957:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610ceb840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227005952:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.809984:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610cebc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227006336:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.811772:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985610cebc00 x1874669227006336/t214748366843(214748366843) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.811784:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.811786:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.811791:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.811798:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366843 00000100:00080000:0.0:1787826990.811807:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985610cea940 x1874669227006720/t214748366844(214748366844) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.811821:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610cebc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227006336:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.811835:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610cea940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227006720:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.813816:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985610cea940 x1874669227006720/t214748366844(214748366844) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.813828:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.813830:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.813835:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.813841:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366844 00000100:00080000:0.0:1787826990.813854:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9856087112c0 x1874669227007104/t214748366845(214748366845) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.813863:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610cea940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227006720:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.813879:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856087112c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227007104:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.815458:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff9856087112c0 x1874669227007104/t214748366845(214748366845) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.815469:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.815472:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.815476:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.815483:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366845 00000100:00080000:0.0:1787826990.815492:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9856087103c0 x1874669227007488/t214748366846(214748366846) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.815501:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856087112c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227007104:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.815527:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856087103c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227007488:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.817523:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff9856087103c0 x1874669227007488/t214748366846(214748366846) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.817538:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.817540:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.817546:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.817554:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366846 00000100:00080000:0.0:1787826990.817578:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985608711680 x1874669227007872/t214748366847(214748366847) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.817590:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856087103c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227007488:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.817607:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608711680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227007872:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.819295:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985608711680 x1874669227007872/t214748366847(214748366847) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.819308:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.819312:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.819318:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.819326:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366847 00000100:00080000:0.0:1787826990.819342:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985608712d00 x1874669227008256/t214748366848(214748366848) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.819353:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608711680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227007872:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.819378:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608712d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227008256:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.821156:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985608712d00 x1874669227008256/t214748366848(214748366848) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.821169:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.821172:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.821178:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.821186:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366848 00000100:00080000:0.0:1787826990.821197:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985608711e00 x1874669227008640/t214748366849(214748366849) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.821208:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608712d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227008256:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.821225:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608711e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227008640:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.822865:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985608711e00 x1874669227008640/t214748366849(214748366849) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.822878:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.822882:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.822888:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.822896:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366849 00000100:00080000:0.0:1787826990.822910:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985608710f00 x1874669227009024/t214748366850(214748366850) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.822921:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608711e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227008640:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.822936:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608710f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227009024:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.824640:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985608710f00 x1874669227009024/t214748366850(214748366850) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.824650:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.824652:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.824657:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.824662:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366850 00000100:00080000:0.0:1787826990.824673:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985608712580 x1874669227009408/t214748366851(214748366851) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.824680:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608710f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227009024:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.824721:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608712580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227009408:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.826407:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985608712580 x1874669227009408/t214748366851(214748366851) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.826419:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.826422:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.826427:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.826438:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366851 00000100:00080000:0.0:1787826990.826450:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9856087121c0 x1874669227009792/t214748366852(214748366852) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.826459:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608712580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227009408:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.826473:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856087121c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227009792:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.828971:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff9856087121c0 x1874669227009792/t214748366852(214748366852) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.828984:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.828987:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.828993:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.829001:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366852 00000100:00080000:0.0:1787826990.829014:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985608712940 x1874669227010176/t214748366853(214748366853) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.829026:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856087121c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227009792:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.829042:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608712940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227010176:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.831730:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985608712940 x1874669227010176/t214748366853(214748366853) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.831753:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.831757:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.831764:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.831771:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366853 00000100:00080000:0.0:1787826990.831785:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985608710b40 x1874669227010560/t214748366854(214748366854) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.831797:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608712940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227010176:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.831820:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608710b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227010560:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.833747:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985608710b40 x1874669227010560/t214748366854(214748366854) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.833760:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.833763:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.833768:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.833776:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366854 00000100:00080000:0.0:1787826990.833789:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985608713c00 x1874669227010944/t214748366855(214748366855) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.833797:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608710b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227010560:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.833811:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608713c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227010944:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.835942:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985608713c00 x1874669227010944/t214748366855(214748366855) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.835952:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.835954:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.835959:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.835967:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366855 00000100:00080000:0.0:1787826990.835978:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985608713840 x1874669227011328/t214748366856(214748366856) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.835987:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608713c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227010944:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.836001:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608713840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227011328:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.837771:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985608713840 x1874669227011328/t214748366856(214748366856) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.837788:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.837791:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.837798:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.837807:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366856 00000100:00080000:0.0:1787826990.837824:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985608710000 x1874669227011712/t214748366857(214748366857) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.837836:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608713840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227011328:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.837855:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608710000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227011712:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.840015:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985608710000 x1874669227011712/t214748366857(214748366857) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.840030:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.840034:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.840040:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.840048:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366857 00000100:00080000:0.0:1787826990.840079:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985608710780 x1874669227012096/t214748366858(214748366858) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.840090:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608710000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227011712:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.840110:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608710780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227012096:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.842043:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985608710780 x1874669227012096/t214748366858(214748366858) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.842074:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.842076:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.842084:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.842092:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366858 00000100:00080000:0.0:1787826990.842106:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9856087130c0 x1874669227012480/t214748366859(214748366859) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.842117:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608710780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227012096:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.842133:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856087130c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227012480:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.844002:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff9856087130c0 x1874669227012480/t214748366859(214748366859) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.844016:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.844020:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.844026:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.844034:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366859 00000100:00080000:0.0:1787826990.844048:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985608711a40 x1874669227012864/t214748366860(214748366860) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.844071:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856087130c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227012480:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.844089:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608711a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227012864:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.845850:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985608711a40 x1874669227012864/t214748366860(214748366860) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.845862:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.845865:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.845871:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.845879:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366860 00000100:00080000:0.0:1787826990.845891:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985608713480 x1874669227013248/t214748366861(214748366861) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.845902:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608711a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227012864:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.845918:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608713480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227013248:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.847938:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985608713480 x1874669227013248/t214748366861(214748366861) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.847952:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.847955:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.847961:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.847968:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366861 00000100:00080000:0.0:1787826990.847982:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98561041de00 x1874669227013632/t214748366862(214748366862) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.847991:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608713480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227013248:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.848007:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98561041de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227013632:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.849940:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98561041de00 x1874669227013632/t214748366862(214748366862) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.849953:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.849956:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.849962:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.849970:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366862 00000100:00080000:0.0:1787826990.849986:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98561041cf00 x1874669227014016/t214748366863(214748366863) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.849995:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98561041de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227013632:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.850009:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98561041cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227014016:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.851860:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98561041cf00 x1874669227014016/t214748366863(214748366863) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.851874:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.851877:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.851884:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.851893:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366863 00000100:00080000:0.0:1787826990.851908:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98561041e580 x1874669227014400/t214748366864(214748366864) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.851916:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98561041cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227014016:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.851931:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98561041e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227014400:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.854004:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98561041e580 x1874669227014400/t214748366864(214748366864) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.854018:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.854021:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.854027:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.854043:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366864 00000100:00080000:0.0:1787826990.854069:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98561041e1c0 x1874669227014784/t214748366865(214748366865) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.854081:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98561041e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227014400:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.854099:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98561041e1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227014784:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.857804:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98561041e1c0 x1874669227014784/t214748366865(214748366865) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.857823:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.857827:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.857835:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.857843:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366865 00000100:00080000:0.0:1787826990.857865:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98561041e940 x1874669227015168/t214748366866(214748366866) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.857878:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98561041e1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227014784:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.857903:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98561041e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227015168:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.861805:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98561041e940 x1874669227015168/t214748366866(214748366866) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.861824:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.861827:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.861846:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.861856:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366866 00000100:00080000:0.0:1787826990.861879:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98561041cb40 x1874669227015552/t214748366867(214748366867) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.861894:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98561041e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227015168:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.861920:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98561041cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227015552:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.864364:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98561041cb40 x1874669227015552/t214748366867(214748366867) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.864380:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.864385:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.864392:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.864400:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366867 00000100:00080000:0.0:1787826990.864422:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98561041fc00 x1874669227015936/t214748366868(214748366868) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.864434:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98561041cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227015552:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.864454:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98561041fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227015936:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.867956:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98561041fc00 x1874669227015936/t214748366868(214748366868) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.867968:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.867971:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.867975:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.867980:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366868 00000100:00080000:0.0:1787826990.867994:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98561041f840 x1874669227016320/t214748366869(214748366869) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.868002:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98561041fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227015936:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.868020:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98561041f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227016320:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.870297:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98561041f840 x1874669227016320/t214748366869(214748366869) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.870306:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.870308:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.870311:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.870316:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366869 00000100:00080000:0.0:1787826990.870327:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98561041c000 x1874669227016704/t214748366870(214748366870) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.870333:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98561041f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227016320:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.870344:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98561041c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227016704:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.875989:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98561041c000 x1874669227016704/t214748366870(214748366870) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.876004:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.876007:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.876014:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.876023:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366870 00000100:00080000:0.0:1787826990.876040:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98561041c780 x1874669227017088/t214748366871(214748366871) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.876064:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98561041c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227016704:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.876082:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98561041c780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227017088:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.878560:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98561041c780 x1874669227017088/t214748366871(214748366871) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.878574:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.878577:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.878582:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.878589:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366871 00000100:00080000:0.0:1787826990.878602:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98561041f0c0 x1874669227017472/t214748366872(214748366872) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.878611:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98561041c780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227017088:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.878624:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98561041f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227017472:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.880371:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98561041f0c0 x1874669227017472/t214748366872(214748366872) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.880382:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.880384:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.880390:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.880397:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366872 00000100:00080000:0.0:1787826990.880409:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98561041da40 x1874669227017856/t214748366873(214748366873) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.880419:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98561041f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227017472:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.880431:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98561041da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227017856:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.886680:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98561041da40 x1874669227017856/t214748366873(214748366873) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.886721:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.886724:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.886731:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.886739:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366873 00000100:00080000:0.0:1787826990.886757:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98561041f480 x1874669227018240/t214748366874(214748366874) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.886784:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98561041da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227017856:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.886804:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98561041f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227018240:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.889912:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98561041f480 x1874669227018240/t214748366874(214748366874) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.889926:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.889928:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.889934:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.889946:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366874 00000100:00080000:0.0:1787826990.889963:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98561041c3c0 x1874669227018624/t214748366875(214748366875) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.889974:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98561041f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227018240:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.889992:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98561041c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227018624:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.891878:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98561041c3c0 x1874669227018624/t214748366875(214748366875) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.891890:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.891893:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.891898:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.891905:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366875 00000100:00080000:0.0:1787826990.891918:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98561041d680 x1874669227019008/t214748366876(214748366876) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.891961:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98561041c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227018624:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.891976:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98561041d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227019008:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.893994:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98561041d680 x1874669227019008/t214748366876(214748366876) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.894007:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.894009:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.894015:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.894021:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366876 00000100:00080000:0.0:1787826990.894034:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98561041ed00 x1874669227019392/t214748366877(214748366877) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.894045:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98561041d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227019008:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.894073:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98561041ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227019392:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.896234:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98561041ed00 x1874669227019392/t214748366877(214748366877) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.896247:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.896250:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.896255:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.896262:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366877 00000100:00080000:0.0:1787826990.896277:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98561041d2c0 x1874669227019776/t214748366878(214748366878) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.896287:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98561041ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227019392:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.896301:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98561041d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227019776:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.898112:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98561041d2c0 x1874669227019776/t214748366878(214748366878) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.898125:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.898127:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.898133:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.898144:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366878 00000100:00080000:0.0:1787826990.898159:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98560868f0c0 x1874669227020160/t214748366879(214748366879) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.898170:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98561041d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227019776:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.898180:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98560868f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227020160:192.168.203.114@tcp:36:unlinkmany.0 00010000:00010000:1.0:1787826990.901097:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787826990.901165:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98560868f0c0 x1874669227020160/t214748366879(214748366879) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.901175:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.901178:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.901182:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.901187:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366879 00000100:00080000:0.0:1787826990.901202:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98560868da40 x1874669227020544/t214748366880(214748366880) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.901212:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98560868f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227020160:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.901229:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98560868da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227020544:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.903732:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98560868da40 x1874669227020544/t214748366880(214748366880) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.903751:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.903758:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.903764:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.903771:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366880 00000100:00080000:0.0:1787826990.903784:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98560868f480 x1874669227020928/t214748366881(214748366881) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.903794:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98560868da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227020544:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.903810:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98560868f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227020928:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.905711:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98560868f480 x1874669227020928/t214748366881(214748366881) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.905722:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.905725:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.905729:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.905734:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366881 00000100:00080000:0.0:1787826990.905745:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98560868c3c0 x1874669227021312/t214748366882(214748366882) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.905756:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98560868f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227020928:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.905769:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98560868c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227021312:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.908351:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98560868c3c0 x1874669227021312/t214748366882(214748366882) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.908365:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.908368:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.908373:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.908380:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366882 00000100:00080000:0.0:1787826990.908396:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98560868d680 x1874669227021696/t214748366883(214748366883) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.908407:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98560868c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227021312:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.908423:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98560868d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227021696:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.910736:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98560868d680 x1874669227021696/t214748366883(214748366883) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.910753:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.910758:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.910766:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.910774:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366883 00000100:00080000:0.0:1787826990.910795:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98560868ed00 x1874669227022080/t214748366884(214748366884) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.910809:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98560868d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227021696:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.910830:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98560868ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227022080:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.912748:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98560868ed00 x1874669227022080/t214748366884(214748366884) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.912759:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.912761:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.912766:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.912773:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366884 00000100:00080000:0.0:1787826990.912791:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98560868d2c0 x1874669227022464/t214748366885(214748366885) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.912803:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98560868ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227022080:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.912818:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98560868d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227022464:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.914313:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98560868d2c0 x1874669227022464/t214748366885(214748366885) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.914328:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.914331:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.914337:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.914346:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366885 00000100:00080000:0.0:1787826990.914363:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98560868de00 x1874669227022848/t214748366886(214748366886) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.914375:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98560868d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227022464:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.914392:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98560868de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227022848:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.916075:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98560868de00 x1874669227022848/t214748366886(214748366886) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.916086:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.916088:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.916093:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.916099:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366886 00000100:00080000:0.0:1787826990.916114:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98560868cf00 x1874669227023232/t214748366887(214748366887) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.916122:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98560868de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227022848:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.916136:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98560868cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227023232:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.917758:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98560868cf00 x1874669227023232/t214748366887(214748366887) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.917771:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.917774:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.917780:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.917797:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366887 00000100:00080000:0.0:1787826990.917824:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98560868e580 x1874669227023616/t214748366888(214748366888) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.917833:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98560868cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227023232:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.917846:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98560868e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227023616:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.919898:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98560868e580 x1874669227023616/t214748366888(214748366888) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.919912:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.919914:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.919921:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.919928:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366888 00000100:00080000:0.0:1787826990.919945:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98560868e1c0 x1874669227024000/t214748366889(214748366889) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.919956:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98560868e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227023616:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.919974:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98560868e1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227024000:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.921669:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98560868e1c0 x1874669227024000/t214748366889(214748366889) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.921679:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.921682:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.921686:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.921722:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366889 00000100:00080000:0.0:1787826990.921740:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98560868e940 x1874669227024384/t214748366890(214748366890) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.921748:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98560868e1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227024000:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.921761:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98560868e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227024384:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.923226:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98560868e940 x1874669227024384/t214748366890(214748366890) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.923234:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.923235:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.923239:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.923244:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366890 00000100:00080000:0.0:1787826990.923255:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98560868cb40 x1874669227024768/t214748366891(214748366891) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.923261:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98560868e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227024384:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.923272:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98560868cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227024768:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.924831:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98560868cb40 x1874669227024768/t214748366891(214748366891) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.924840:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.924843:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.924847:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.924852:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366891 00000100:00080000:0.0:1787826990.924863:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98560868fc00 x1874669227025152/t214748366892(214748366892) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.924869:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98560868cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227024768:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.924880:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98560868fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227025152:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.926481:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98560868fc00 x1874669227025152/t214748366892(214748366892) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.926491:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.926493:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.926497:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.926502:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366892 00000100:00080000:0.0:1787826990.926514:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98560868f840 x1874669227025536/t214748366893(214748366893) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.926520:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98560868fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227025152:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.926531:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98560868f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227025536:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.928156:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98560868f840 x1874669227025536/t214748366893(214748366893) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.928165:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.928167:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.928172:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.928177:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366893 00000100:00080000:0.0:1787826990.928191:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98560868c000 x1874669227025920/t214748366894(214748366894) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.928201:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98560868f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227025536:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.928218:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98560868c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227025920:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.929812:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98560868c000 x1874669227025920/t214748366894(214748366894) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.929823:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.929826:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.929831:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.929846:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366894 00000100:00080000:0.0:1787826990.929861:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98560868c780 x1874669227026304/t214748366895(214748366895) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.929872:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98560868c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227025920:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.929887:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98560868c780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227026304:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.932147:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98560868c780 x1874669227026304/t214748366895(214748366895) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.932161:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.932164:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.932170:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.932178:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366895 00000100:00080000:0.0:1787826990.932196:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9856086830c0 x1874669227026688/t214748366896(214748366896) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.932208:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98560868c780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227026304:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.932225:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856086830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227026688:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.933896:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff9856086830c0 x1874669227026688/t214748366896(214748366896) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.933908:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.933911:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.933917:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.933926:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366896 00000100:00080000:0.0:1787826990.933941:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985608681a40 x1874669227027072/t214748366897(214748366897) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.933952:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856086830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227026688:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.934064:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608681a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227027072:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.935840:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985608681a40 x1874669227027072/t214748366897(214748366897) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.935852:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.935855:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.935861:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.935868:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366897 00000100:00080000:0.0:1787826990.935893:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985608683480 x1874669227027456/t214748366898(214748366898) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.935902:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608681a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227027072:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.935915:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608683480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227027456:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.937549:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985608683480 x1874669227027456/t214748366898(214748366898) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.937567:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.937570:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.937576:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.937584:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366898 00000100:00080000:0.0:1787826990.937598:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9856086803c0 x1874669227027840/t214748366899(214748366899) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.937609:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608683480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227027456:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.937622:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856086803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227027840:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.939152:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff9856086803c0 x1874669227027840/t214748366899(214748366899) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.939162:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.939175:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.939181:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.939189:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366899 00000100:00080000:0.0:1787826990.939204:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985608681680 x1874669227028224/t214748366900(214748366900) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.939214:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856086803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227027840:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.939228:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608681680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227028224:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.941188:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985608681680 x1874669227028224/t214748366900(214748366900) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.941198:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.941201:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.941206:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.941214:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366900 00000100:00080000:0.0:1787826990.941235:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985608682d00 x1874669227028608/t214748366901(214748366901) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.941245:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608681680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227028224:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.941263:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608682d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227028608:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.942855:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985608682d00 x1874669227028608/t214748366901(214748366901) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.942865:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.942868:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.942873:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.942880:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366901 00000100:00080000:0.0:1787826990.942892:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9856086812c0 x1874669227028992/t214748366902(214748366902) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.942903:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608682d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227028608:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.942915:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856086812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227028992:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.944331:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff9856086812c0 x1874669227028992/t214748366902(214748366902) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.944338:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.944340:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.944343:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.944349:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366902 00000100:00080000:0.0:1787826990.944361:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985608681e00 x1874669227029376/t214748366903(214748366903) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.944370:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856086812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227028992:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.944382:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608681e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227029376:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.945897:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985608681e00 x1874669227029376/t214748366903(214748366903) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.945909:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.945912:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.945916:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.945922:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366903 00000100:00080000:0.0:1787826990.945940:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985608680f00 x1874669227029760/t214748366904(214748366904) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.945951:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608681e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227029376:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.945981:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608680f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227029760:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.947917:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985608680f00 x1874669227029760/t214748366904(214748366904) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.947932:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.947934:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.947941:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.947949:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366904 00000100:00080000:0.0:1787826990.947981:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985608682580 x1874669227030144/t214748366905(214748366905) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.947993:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608680f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227029760:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.948011:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608682580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227030144:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.949857:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985608682580 x1874669227030144/t214748366905(214748366905) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.949870:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.949873:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.949879:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.949888:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366905 00000100:00080000:0.0:1787826990.949907:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9856084852c0 x1874669227030528/t214748366906(214748366906) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.949918:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608682580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227030144:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.949935:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856084852c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227030528:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.951827:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff9856084852c0 x1874669227030528/t214748366906(214748366906) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.951838:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.951840:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.951845:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.951850:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366906 00000100:00080000:0.0:1787826990.951867:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985608484f00 x1874669227030912/t214748366907(214748366907) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.951875:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856084852c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227030528:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.951889:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608484f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227030912:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.953853:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985608484f00 x1874669227030912/t214748366907(214748366907) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.953865:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.953869:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.953874:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.953882:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366907 00000100:00080000:0.0:1787826990.953895:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985608486580 x1874669227031296/t214748366908(214748366908) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.953906:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608484f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227030912:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.953920:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608486580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227031296:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.955439:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985608486580 x1874669227031296/t214748366908(214748366908) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.955450:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.955453:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.955462:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.955470:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366908 00000100:00080000:0.0:1787826990.955486:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9856084861c0 x1874669227031680/t214748366909(214748366909) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.955496:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608486580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227031296:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.955516:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856084861c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227031680:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.957271:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff9856084861c0 x1874669227031680/t214748366909(214748366909) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.957282:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.957285:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.957290:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.957297:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366909 00000100:00080000:0.0:1787826990.957312:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985608486940 x1874669227032064/t214748366910(214748366910) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.957321:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856084861c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227031680:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.957336:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608486940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227032064:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.958926:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985608486940 x1874669227032064/t214748366910(214748366910) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.958935:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.958938:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.958942:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.958950:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366910 00000100:00080000:0.0:1787826990.958961:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985608484b40 x1874669227032448/t214748366911(214748366911) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.958972:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608486940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227032064:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.958984:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608484b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227032448:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.960683:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985608484b40 x1874669227032448/t214748366911(214748366911) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.960717:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.960719:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.960725:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.960732:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366911 00000100:00080000:0.0:1787826990.960746:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985608487c00 x1874669227032832/t214748366912(214748366912) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.960755:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608484b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227032448:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.960769:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608487c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227032832:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.962945:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985608487c00 x1874669227032832/t214748366912(214748366912) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.962954:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.962956:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.962959:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.962965:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366912 00000100:00080000:0.0:1787826990.962980:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985608487840 x1874669227033216/t214748366913(214748366913) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.962991:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608487c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227032832:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.963007:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608487840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227033216:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.965278:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985608487840 x1874669227033216/t214748366913(214748366913) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.965292:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.965294:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.965299:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.965305:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366913 00000100:00080000:0.0:1787826990.965320:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985608484000 x1874669227033600/t214748366914(214748366914) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.965328:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608487840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227033216:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.965340:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608484000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227033600:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.968203:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985608484000 x1874669227033600/t214748366914(214748366914) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.968218:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.968220:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.968226:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.968234:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366914 00000100:00080000:0.0:1787826990.968252:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985608485e00 x1874669227033984/t214748366915(214748366915) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.968264:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608484000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227033600:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.968282:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608485e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227033984:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.970639:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985608485e00 x1874669227033984/t214748366915(214748366915) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.970650:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.970653:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.970658:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.970665:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366915 00000100:00080000:0.0:1787826990.970681:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985610e64f00 x1874669227034368/t214748366916(214748366916) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.970730:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608485e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227033984:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.970780:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e64f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227034368:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.972492:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985610e64f00 x1874669227034368/t214748366916(214748366916) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.972503:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.972505:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.972510:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.972518:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366916 00000100:00080000:0.0:1787826990.972531:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985610e67480 x1874669227034752/t214748366917(214748366917) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.972541:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e64f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227034368:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.972554:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227034752:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.974489:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985610e67480 x1874669227034752/t214748366917(214748366917) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.974507:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.974511:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.974518:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.974526:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366917 00000100:00080000:0.0:1787826990.974551:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985610e643c0 x1874669227035136/t214748366918(214748366918) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.974562:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e67480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227034752:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.974582:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e643c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227035136:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.977000:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985610e643c0 x1874669227035136/t214748366918(214748366918) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.977018:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.977021:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.977028:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.977038:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366918 00000100:00080000:0.0:1787826990.977083:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985610e65680 x1874669227035520/t214748366919(214748366919) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.977097:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e643c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227035136:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.977118:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e65680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227035520:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.979127:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985610e65680 x1874669227035520/t214748366919(214748366919) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.979140:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.979143:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.979147:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.979153:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366919 00000100:00080000:0.0:1787826990.979178:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985610e66d00 x1874669227035904/t214748366920(214748366920) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.979186:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e65680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227035520:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.979201:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e66d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227035904:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.980658:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985610e66d00 x1874669227035904/t214748366920(214748366920) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.980677:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.980679:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.980685:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.980721:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366920 00000100:00080000:0.0:1787826990.980737:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9856086821c0 x1874669227036288/t214748366921(214748366921) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.980747:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e66d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227035904:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.980762:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856086821c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227036288:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.984775:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff9856086821c0 x1874669227036288/t214748366921(214748366921) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.984789:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.984793:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.984800:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.984807:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366921 00000100:00080000:0.0:1787826990.984830:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985608682940 x1874669227036672/t214748366922(214748366922) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.984843:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856086821c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227036288:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.984863:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608682940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227036672:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.986944:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985608682940 x1874669227036672/t214748366922(214748366922) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.986958:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.986961:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.986966:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.986974:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366922 00000100:00080000:0.0:1787826990.986990:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985608680b40 x1874669227037056/t214748366923(214748366923) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.986998:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608682940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227036672:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.987012:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608680b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227037056:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.988586:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985608680b40 x1874669227037056/t214748366923(214748366923) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.988597:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.988600:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.988606:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.988613:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366923 00000100:00080000:0.0:1787826990.988628:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985608683c00 x1874669227037440/t214748366924(214748366924) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.988636:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608680b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227037056:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.988648:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608683c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227037440:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.990369:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985608683c00 x1874669227037440/t214748366924(214748366924) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.990377:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.990378:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.990382:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.990389:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366924 00000100:00080000:0.0:1787826990.990418:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985608683840 x1874669227037824/t214748366925(214748366925) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.990428:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608683c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227037440:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.990442:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608683840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227037824:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.992226:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985608683840 x1874669227037824/t214748366925(214748366925) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.992238:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.992241:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.992247:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.992254:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366925 00000100:00080000:0.0:1787826990.992270:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985608680000 x1874669227038208/t214748366926(214748366926) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.992280:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608683840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227037824:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.992293:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608680000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227038208:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.993931:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985608680000 x1874669227038208/t214748366926(214748366926) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.993942:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.993944:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.993950:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.993957:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366926 00000100:00080000:0.0:1787826990.993974:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985608680780 x1874669227038592/t214748366927(214748366927) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.993985:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608680000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227038208:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.994000:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608680780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227038592:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.995914:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985608680780 x1874669227038592/t214748366927(214748366927) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.995924:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.995926:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.995930:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.995935:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366927 00000100:00080000:0.0:1787826990.995951:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98560865da40 x1874669227038976/t214748366928(214748366928) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.995958:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608680780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227038592:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.995969:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98560865da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227038976:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826990.998565:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98560865da40 x1874669227038976/t214748366928(214748366928) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.998578:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826990.998580:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826990.998586:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.998594:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366928 00000100:00080000:0.0:1787826990.998613:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98560865cb40 x1874669227039360/t214748366929(214748366929) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.998624:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98560865da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227038976:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826990.998649:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98560865cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227039360:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.000765:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98560865cb40 x1874669227039360/t214748366929(214748366929) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827006 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.000778:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.000781:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.000786:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.000793:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366929 00000100:00080000:0.0:1787826991.000815:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98560865e940 x1874669227039744/t214748366930(214748366930) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.000825:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98560865cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227039360:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.000839:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98560865e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227039744:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.002819:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98560865e940 x1874669227039744/t214748366930(214748366930) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.002831:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.002833:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.002838:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.002846:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366930 00000100:00080000:0.0:1787826991.002863:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98560865fc00 x1874669227040128/t214748366931(214748366931) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.002874:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98560865e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227039744:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.002888:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98560865fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227040128:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.004681:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98560865fc00 x1874669227040128/t214748366931(214748366931) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.004714:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.004717:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.004722:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.004730:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366931 00000100:00080000:0.0:1787826991.004748:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98560865f840 x1874669227040512/t214748366932(214748366932) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.004759:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98560865fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227040128:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.004773:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98560865f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227040512:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.006535:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98560865f840 x1874669227040512/t214748366932(214748366932) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.006547:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.006550:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.006555:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.006562:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366932 00000100:00080000:0.0:1787826991.006579:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98560865c000 x1874669227040896/t214748366933(214748366933) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.006589:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98560865f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227040512:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.006602:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98560865c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227040896:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.008578:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98560865c000 x1874669227040896/t214748366933(214748366933) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.008590:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.008594:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.008599:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.008607:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366933 00000100:00080000:0.0:1787826991.008627:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98560865c780 x1874669227041280/t214748366934(214748366934) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.008638:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98560865c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227040896:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.008654:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98560865c780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227041280:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.010644:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98560865c780 x1874669227041280/t214748366934(214748366934) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.010655:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.010657:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.010662:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.010669:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366934 00000100:00080000:0.0:1787826991.010688:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98560865f0c0 x1874669227041664/t214748366935(214748366935) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.010725:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98560865c780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227041280:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.010740:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98560865f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227041664:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.012874:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98560865f0c0 x1874669227041664/t214748366935(214748366935) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.012888:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.012891:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.012897:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.012905:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366935 00000100:00080000:0.0:1787826991.012926:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98560865f480 x1874669227042048/t214748366936(214748366936) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.012936:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98560865f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227041664:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.012952:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98560865f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227042048:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.014671:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98560865f480 x1874669227042048/t214748366936(214748366936) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.014681:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.014684:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.014723:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.014730:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366936 00000100:00080000:0.0:1787826991.014745:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98560865c3c0 x1874669227042432/t214748366937(214748366937) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.014755:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98560865f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227042048:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.014782:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98560865c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227042432:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.016550:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98560865c3c0 x1874669227042432/t214748366937(214748366937) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.016566:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.016569:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.016574:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.016582:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366937 00000100:00080000:0.0:1787826991.016601:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98560865d680 x1874669227042816/t214748366938(214748366938) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.016610:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98560865c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227042432:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.016625:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98560865d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227042816:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.018929:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98560865d680 x1874669227042816/t214748366938(214748366938) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.018941:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.018943:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.018948:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.018955:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366938 00000100:00080000:0.0:1787826991.018974:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98560865ed00 x1874669227043200/t214748366939(214748366939) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.018984:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98560865d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227042816:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.019015:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98560865ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227043200:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.021203:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98560865ed00 x1874669227043200/t214748366939(214748366939) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.021216:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.021219:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.021223:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.021230:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366939 00000100:00080000:0.0:1787826991.021251:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98560865d2c0 x1874669227043584/t214748366940(214748366940) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.021261:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98560865ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227043200:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.021275:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98560865d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227043584:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.023874:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98560865d2c0 x1874669227043584/t214748366940(214748366940) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.023886:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.023888:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.023893:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.023901:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366940 00000100:00080000:0.0:1787826991.023919:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98560865de00 x1874669227043968/t214748366941(214748366941) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.023933:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98560865d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227043584:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.023947:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98560865de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227043968:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.025652:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98560865de00 x1874669227043968/t214748366941(214748366941) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.025663:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.025666:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.025671:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.025678:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366941 00000100:00080000:0.0:1787826991.025725:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98560865cf00 x1874669227044352/t214748366942(214748366942) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.025739:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98560865de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227043968:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.025755:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98560865cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227044352:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.027360:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98560865cf00 x1874669227044352/t214748366942(214748366942) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.027372:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.027374:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.027378:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.027383:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366942 00000100:00080000:0.0:1787826991.027413:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98560865e580 x1874669227044736/t214748366943(214748366943) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.027425:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98560865cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227044352:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.027446:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98560865e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227044736:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:1.0:1787826991.028790:0:86156:0:(client.c:2715:ptlrpc_set_wait()) set ffff985612993b40 going to sleep for 0 seconds 00000100:00080000:0.0:1787826991.029073:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98560865e580 x1874669227044736/t214748366943(214748366943) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.029087:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.029091:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.029096:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.029104:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366943 00000100:00080000:0.0:1787826991.029130:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98560865e1c0 x1874669227045120/t214748366944(214748366944) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.029141:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98560865e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227044736:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.029157:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98560865e1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227045120:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.030839:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98560865e1c0 x1874669227045120/t214748366944(214748366944) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.030851:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.030854:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.030860:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.030868:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366944 00000100:00080000:0.0:1787826991.030890:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985614176580 x1874669227045504/t214748366945(214748366945) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.030901:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98560865e1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227045120:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.030920:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985614176580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227045504:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.032530:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985614176580 x1874669227045504/t214748366945(214748366945) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.032539:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.032542:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.032546:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.032550:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366945 00000100:00080000:0.0:1787826991.032567:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985614175a40 x1874669227045888/t214748366946(214748366946) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.032575:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985614176580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227045504:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.032589:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985614175a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227045888:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.034066:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985614175a40 x1874669227045888/t214748366946(214748366946) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.034081:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.034084:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.034091:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.034100:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366946 00000100:00080000:0.0:1787826991.034123:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985614174b40 x1874669227046272/t214748366947(214748366947) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.034132:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985614175a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227045888:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.034145:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985614174b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227046272:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.035552:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985614174b40 x1874669227046272/t214748366947(214748366947) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.035562:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.035565:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.035568:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.035574:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366947 00000100:00080000:0.0:1787826991.035593:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985614177480 x1874669227046656/t214748366948(214748366948) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.035603:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985614174b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227046272:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.035617:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985614177480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227046656:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.036863:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985614177480 x1874669227046656/t214748366948(214748366948) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.036870:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.036871:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.036875:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.036879:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366948 00000100:00080000:0.0:1787826991.036891:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985614176d00 x1874669227047040/t214748366949(214748366949) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.036898:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985614177480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227046656:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.036906:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985614176d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227047040:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.038171:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985614176d00 x1874669227047040/t214748366949(214748366949) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.038181:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.038185:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.038189:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.038195:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366949 00000100:00080000:0.0:1787826991.038210:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985614174000 x1874669227047424/t214748366950(214748366950) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.038219:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985614176d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227047040:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.038231:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985614174000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227047424:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.039839:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985614174000 x1874669227047424/t214748366950(214748366950) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.039853:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.039856:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.039862:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.039870:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366950 00000100:00080000:0.0:1787826991.039892:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9856141770c0 x1874669227047808/t214748366951(214748366951) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.039903:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985614174000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227047424:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.039919:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856141770c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227047808:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.041325:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff9856141770c0 x1874669227047808/t214748366951(214748366951) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.041335:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.041338:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.041343:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.041350:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366951 00000100:00080000:0.0:1787826991.041367:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985614176940 x1874669227048192/t214748366952(214748366952) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.041381:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856141770c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227047808:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.041400:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985614176940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227048192:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.042744:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985614176940 x1874669227048192/t214748366952(214748366952) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.042755:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.042757:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.042762:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.042769:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366952 00000100:00080000:0.0:1787826991.042788:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985614174780 x1874669227048576/t214748366953(214748366953) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.042799:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985614176940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227048192:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.042812:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985614174780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227048576:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.044282:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985614174780 x1874669227048576/t214748366953(214748366953) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.044292:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.044295:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.044301:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.044307:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366953 00000100:00080000:0.0:1787826991.044331:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985614177c00 x1874669227048960/t214748366954(214748366954) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.044341:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985614174780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227048576:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.044353:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985614177c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227048960:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.045650:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985614177c00 x1874669227048960/t214748366954(214748366954) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.045659:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.045660:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.045664:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.045669:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366954 00000100:00080000:0.0:1787826991.045685:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9856141743c0 x1874669227049344/t214748366955(214748366955) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.045725:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985614177c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227048960:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.045737:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856141743c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227049344:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.047120:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff9856141743c0 x1874669227049344/t214748366955(214748366955) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.047130:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.047133:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.047139:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.047145:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366955 00000100:00080000:0.0:1787826991.047162:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9856141761c0 x1874669227049728/t214748366956(214748366956) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.047172:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856141743c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227049344:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.047183:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856141761c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227049728:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.048620:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff9856141761c0 x1874669227049728/t214748366956(214748366956) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.048633:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.048636:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.048641:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.048649:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366956 00000100:00080000:0.0:1787826991.048668:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985613163480 x1874669227050112/t214748366957(214748366957) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.048680:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856141761c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227049728:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.048720:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985613163480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227050112:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.050255:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985613163480 x1874669227050112/t214748366957(214748366957) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.050271:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.050273:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.050277:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.050283:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366957 00000100:00080000:0.0:1787826991.050306:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985613161a40 x1874669227050496/t214748366958(214748366958) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.050314:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985613163480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227050112:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.050328:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985613161a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227050496:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.053248:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985613161a40 x1874669227050496/t214748366958(214748366958) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.053257:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.053259:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.053262:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.053267:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366958 00000100:00080000:0.0:1787826991.053286:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98560993b840 x1874669227050880/t214748366959(214748366959) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.053293:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985613161a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227050496:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.053303:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98560993b840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227050880:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.055045:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98560993b840 x1874669227050880/t214748366959(214748366959) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.055067:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.055070:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.055075:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.055082:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366959 00000100:00080000:0.0:1787826991.055101:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98560993a580 x1874669227051264/t214748366960(214748366960) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.055111:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98560993b840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227050880:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.055126:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98560993a580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227051264:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.056651:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98560993a580 x1874669227051264/t214748366960(214748366960) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.056659:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.056660:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.056663:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.056668:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366960 00000100:00080000:0.0:1787826991.056681:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9856099383c0 x1874669227051648/t214748366961(214748366961) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.056687:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98560993a580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227051264:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.056718:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856099383c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227051648:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.058286:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff9856099383c0 x1874669227051648/t214748366961(214748366961) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.058297:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.058299:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.058304:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.058312:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366961 00000100:00080000:0.0:1787826991.058329:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985609938b40 x1874669227052032/t214748366962(214748366962) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.058339:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856099383c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227051648:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.058350:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985609938b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227052032:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.059712:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985609938b40 x1874669227052032/t214748366962(214748366962) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.059721:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.059724:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.059729:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.059736:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366962 00000100:00080000:0.0:1787826991.059752:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985609939e00 x1874669227052416/t214748366963(214748366963) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.059761:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985609938b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227052032:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.059774:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985609939e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227052416:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.061101:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985609939e00 x1874669227052416/t214748366963(214748366963) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.061109:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.061112:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.061116:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.061122:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366963 00000100:00080000:0.0:1787826991.061135:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98560993b0c0 x1874669227052800/t214748366964(214748366964) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.061144:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985609939e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227052416:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.061155:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98560993b0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227052800:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.062509:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98560993b0c0 x1874669227052800/t214748366964(214748366964) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.062519:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.062521:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.062525:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.062532:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366964 00000100:00080000:0.0:1787826991.062547:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98560993a940 x1874669227053184/t214748366965(214748366965) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.062556:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98560993b0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227052800:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.062566:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98560993a940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227053184:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.064145:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98560993a940 x1874669227053184/t214748366965(214748366965) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.064156:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.064159:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.064165:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.064173:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366965 00000100:00080000:0.0:1787826991.064190:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9856099392c0 x1874669227053568/t214748366966(214748366966) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.064201:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98560993a940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227053184:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.064214:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856099392c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227053568:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.066111:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff9856099392c0 x1874669227053568/t214748366966(214748366966) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.066126:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.066129:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.066136:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.066144:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366966 00000100:00080000:0.0:1787826991.066176:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985609938f00 x1874669227053952/t214748366967(214748366967) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.066192:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856099392c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227053568:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.066217:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985609938f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227053952:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.068166:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985609938f00 x1874669227053952/t214748366967(214748366967) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.068179:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.068183:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.068199:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.068207:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366967 00000100:00080000:0.0:1787826991.068233:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985609938780 x1874669227054336/t214748366968(214748366968) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.068244:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985609938f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227053952:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.068264:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985609938780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227054336:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.070137:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985609938780 x1874669227054336/t214748366968(214748366968) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.070148:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.070150:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.070155:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.070160:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366968 00000100:00080000:0.0:1787826991.070188:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98560993b480 x1874669227054720/t214748366969(214748366969) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.070199:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985609938780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227054336:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.070215:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98560993b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227054720:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.071757:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98560993b480 x1874669227054720/t214748366969(214748366969) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.071769:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.071773:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.071779:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.071786:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366969 00000100:00080000:0.0:1787826991.071804:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985609939a40 x1874669227055104/t214748366970(214748366970) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.071811:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98560993b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227054720:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.071823:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985609939a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227055104:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.073740:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985609939a40 x1874669227055104/t214748366970(214748366970) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.073753:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.073756:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.073760:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.073767:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366970 00000100:00080000:0.0:1787826991.073800:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98560993ad00 x1874669227055488/t214748366971(214748366971) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.073813:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985609939a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227055104:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.073831:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98560993ad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227055488:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.076029:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98560993ad00 x1874669227055488/t214748366971(214748366971) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.076061:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.076065:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.076071:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.076079:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366971 00000100:00080000:0.0:1787826991.076118:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985609939680 x1874669227055872/t214748366972(214748366972) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.076130:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98560993ad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227055488:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.076150:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985609939680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227055872:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.078977:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985609939680 x1874669227055872/t214748366972(214748366972) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.078995:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.078998:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.079005:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.079014:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366972 00000100:00080000:0.0:1787826991.079064:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985609938000 x1874669227056256/t214748366973(214748366973) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.079079:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985609939680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227055872:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.079101:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985609938000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227056256:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.081655:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985609938000 x1874669227056256/t214748366973(214748366973) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.081669:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.081671:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.081676:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.081681:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366973 00000100:00080000:0.0:1787826991.081733:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98560993bc00 x1874669227056640/t214748366974(214748366974) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.081743:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985609938000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227056256:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.081761:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98560993bc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227056640:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.086213:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98560993bc00 x1874669227056640/t214748366974(214748366974) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.086230:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.086234:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.086242:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.086251:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366974 00000100:00080000:0.0:1787826991.086291:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985610e4c3c0 x1874669227057024/t214748366975(214748366975) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.086308:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98560993bc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227056640:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.086336:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e4c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227057024:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.091428:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985610e4c3c0 x1874669227057024/t214748366975(214748366975) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.091443:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.091447:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.091453:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.091461:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366975 00000100:00080000:0.0:1787826991.091494:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985610e4da40 x1874669227057408/t214748366976(214748366976) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.091506:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e4c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227057024:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.091527:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e4da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227057408:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.094155:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985610e4da40 x1874669227057408/t214748366976(214748366976) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.094169:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.094171:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.094177:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.094184:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366976 00000100:00080000:0.0:1787826991.094220:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985610e4d680 x1874669227057792/t214748366977(214748366977) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.094232:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e4da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227057408:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.094249:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e4d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227057792:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.096200:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985610e4d680 x1874669227057792/t214748366977(214748366977) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.096212:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.096215:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.096219:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.096224:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366977 00000100:00080000:0.0:1787826991.096244:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985610e4d2c0 x1874669227058176/t214748366978(214748366978) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.096251:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e4d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227057792:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.096272:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e4d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227058176:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.098891:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985610e4d2c0 x1874669227058176/t214748366978(214748366978) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.098904:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.098907:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.098912:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.098920:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366978 00000100:00080000:0.0:1787826991.098946:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985610e4de00 x1874669227058560/t214748366979(214748366979) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.098957:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e4d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227058176:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.098984:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e4de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227058560:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.101170:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985610e4de00 x1874669227058560/t214748366979(214748366979) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.101183:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.101185:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.101191:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.101198:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366979 00000100:00080000:0.0:1787826991.101223:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985610e4fc00 x1874669227058944/t214748366980(214748366980) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.101233:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e4de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227058560:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.101249:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e4fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227058944:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.103427:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985610e4fc00 x1874669227058944/t214748366980(214748366980) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.103440:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.103443:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.103450:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.103467:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366980 00000100:00080000:0.0:1787826991.103493:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985610e4e940 x1874669227059328/t214748366981(214748366981) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.103504:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e4fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227058944:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.103520:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e4e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227059328:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.106007:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985610e4e940 x1874669227059328/t214748366981(214748366981) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.106021:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.106024:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.106030:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.106038:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366981 00000100:00080000:0.0:1787826991.106081:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985610e4e1c0 x1874669227059712/t214748366982(214748366982) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.106092:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e4e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227059328:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.106108:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e4e1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227059712:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.109324:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985610e4e1c0 x1874669227059712/t214748366982(214748366982) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.109338:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.109341:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.109347:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.109355:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366982 00000100:00080000:0.0:1787826991.109398:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985610e4c780 x1874669227060096/t214748366983(214748366983) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.109410:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e4e1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227059712:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.109427:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e4c780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227060096:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.112265:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985610e4c780 x1874669227060096/t214748366983(214748366983) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.112278:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.112282:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.112287:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.112295:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366983 00000100:00080000:0.0:1787826991.112323:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985610e4c000 x1874669227060480/t214748366984(214748366984) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.112333:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e4c780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227060096:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.112349:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e4c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227060480:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.115323:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985610e4c000 x1874669227060480/t214748366984(214748366984) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.115336:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.115338:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.115343:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.115348:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366984 00000100:00080000:0.0:1787826991.115378:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985610e4cf00 x1874669227060864/t214748366985(214748366985) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.115389:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e4c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227060480:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.115406:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e4cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227060864:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.121140:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985610e4cf00 x1874669227060864/t214748366985(214748366985) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.121152:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.121154:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.121158:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.121164:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366985 00000100:00080000:0.0:1787826991.121193:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985610e4e580 x1874669227061248/t214748366986(214748366986) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.121201:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e4cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227060864:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.121215:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e4e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227061248:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.127730:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985610e4e580 x1874669227061248/t214748366986(214748366986) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.127746:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.127749:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.127756:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.127764:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366986 00000100:00080000:0.0:1787826991.127798:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985610e4f0c0 x1874669227061632/t214748366987(214748366987) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.127811:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e4e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227061248:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.127829:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e4f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227061632:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.130912:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985610e4f0c0 x1874669227061632/t214748366987(214748366987) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.130926:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.130930:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.130937:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.130944:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366987 00000100:00080000:0.0:1787826991.130974:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98561408e580 x1874669227062016/t214748366988(214748366988) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.130986:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e4f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227061632:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.131004:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98561408e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227062016:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.135850:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98561408e580 x1874669227062016/t214748366988(214748366988) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.135864:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.135866:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.135872:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.135880:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366988 00000100:00080000:0.0:1787826991.135925:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98561408da40 x1874669227062400/t214748366989(214748366989) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.135938:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98561408e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227062016:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.135956:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98561408da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227062400:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.138181:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98561408da40 x1874669227062400/t214748366989(214748366989) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.138195:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.138198:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.138204:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.138212:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366989 00000100:00080000:0.0:1787826991.138241:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985610e65e00 x1874669227062784/t214748366990(214748366990) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.138253:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98561408da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227062400:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.138269:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e65e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227062784:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.141218:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985610e65e00 x1874669227062784/t214748366990(214748366990) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.141227:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.141229:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.141233:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.141238:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366990 00000100:00080000:0.0:1787826991.141265:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985610e66580 x1874669227063168/t214748366991(214748366991) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.141272:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e65e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227062784:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.141284:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e66580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227063168:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.143888:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985610e66580 x1874669227063168/t214748366991(214748366991) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.143902:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.143905:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.143911:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.143918:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366991 00000100:00080000:0.0:1787826991.143949:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985610e661c0 x1874669227063552/t214748366992(214748366992) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.143959:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e66580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227063168:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.143975:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e661c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227063552:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.146039:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985610e661c0 x1874669227063552/t214748366992(214748366992) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.146062:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.146065:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.146070:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.146078:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366992 00000100:00080000:0.0:1787826991.146104:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985610e66940 x1874669227063936/t214748366993(214748366993) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.146114:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e661c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227063552:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.146140:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e66940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227063936:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.147879:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985610e66940 x1874669227063936/t214748366993(214748366993) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.147887:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.147889:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.147893:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.147898:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366993 00000100:00080000:0.0:1787826991.147918:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985610e64b40 x1874669227064320/t214748366994(214748366994) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.147925:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e66940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227063936:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.147936:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e64b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227064320:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.149526:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985610e64b40 x1874669227064320/t214748366994(214748366994) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.149539:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.149542:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.149547:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.149556:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366994 00000100:00080000:0.0:1787826991.149580:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985610e67c00 x1874669227064704/t214748366995(214748366995) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.149592:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e64b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227064320:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.149610:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e67c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227064704:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.151564:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985610e67c00 x1874669227064704/t214748366995(214748366995) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.151575:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.151579:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.151585:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.151593:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366995 00000100:00080000:0.0:1787826991.151617:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985610e67840 x1874669227065088/t214748366996(214748366996) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.151628:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e67c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227064704:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.151643:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e67840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227065088:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.153634:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985610e67840 x1874669227065088/t214748366996(214748366996) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.153647:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.153649:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.153655:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.153662:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366996 00000100:00080000:0.0:1787826991.153685:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985610e64000 x1874669227065472/t214748366997(214748366997) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.153724:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e67840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227065088:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.153739:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227065472:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.157269:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985610e64000 x1874669227065472/t214748366997(214748366997) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.157282:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.157285:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.157290:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.157297:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366997 00000100:00080000:0.0:1787826991.157329:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985610e64780 x1874669227065856/t214748366998(214748366998) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.157342:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227065472:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.157368:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e64780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227065856:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.159914:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985610e64780 x1874669227065856/t214748366998(214748366998) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.159924:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.159926:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.159930:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.159935:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366998 00000100:00080000:0.0:1787826991.159959:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985610e670c0 x1874669227066240/t214748366999(214748366999) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.159970:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e64780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227065856:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.159986:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e670c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227066240:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.163658:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985610e670c0 x1874669227066240/t214748366999(214748366999) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.163671:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.163675:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.163680:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.163688:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748366999 00000100:00080000:0.0:1787826991.163744:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985610e65a40 x1874669227066624/t214748367000(214748367000) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.163754:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e670c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227066240:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.163771:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e65a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227066624:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.166421:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985610e65a40 x1874669227066624/t214748367000(214748367000) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.166435:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.166437:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.166443:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.166450:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367000 00000100:00080000:0.0:1787826991.166483:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985610e652c0 x1874669227067008/t214748367001(214748367001) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.166500:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e65a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227066624:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.166516:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e652c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227067008:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.169045:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985610e652c0 x1874669227067008/t214748367001(214748367001) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.169072:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.169076:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.169082:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.169090:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367001 00000100:00080000:0.0:1787826991.169117:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985602ee2d00 x1874669227067392/t214748367002(214748367002) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.169133:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e652c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227067008:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.169149:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee2d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227067392:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.171376:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985602ee2d00 x1874669227067392/t214748367002(214748367002) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.171395:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.171398:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.171405:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.171414:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367002 00000100:00080000:0.0:1787826991.171462:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985602ee03c0 x1874669227067776/t214748367003(214748367003) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.171474:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee2d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227067392:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.171493:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227067776:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.173657:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985602ee03c0 x1874669227067776/t214748367003(214748367003) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.173668:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.173671:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.173676:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.173681:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367003 00000100:00080000:0.0:1787826991.173748:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985602ee3840 x1874669227068160/t214748367004(214748367004) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.173760:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227067776:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.173779:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227068160:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.175958:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985602ee3840 x1874669227068160/t214748367004(214748367004) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.175972:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.175975:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.175985:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.175993:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367004 00000100:00080000:0.0:1787826991.176026:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985602ee2580 x1874669227068544/t214748367005(214748367005) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.176037:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227068160:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.176068:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227068544:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.177955:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985602ee2580 x1874669227068544/t214748367005(214748367005) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.177981:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.177985:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.177991:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.177999:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367005 00000100:00080000:0.0:1787826991.178033:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985602ee0000 x1874669227068928/t214748367006(214748367006) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.178058:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227068544:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.178076:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227068928:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.179858:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985602ee0000 x1874669227068928/t214748367006(214748367006) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.179872:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.179874:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.179881:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.179893:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367006 00000100:00080000:0.0:1787826991.179926:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985602ee2940 x1874669227069312/t214748367007(214748367007) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.179938:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227068928:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.179959:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227069312:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.181855:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985602ee2940 x1874669227069312/t214748367007(214748367007) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.181866:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.181868:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.181873:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.181879:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367007 00000100:00080000:0.0:1787826991.181910:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985602ee1a40 x1874669227069696/t214748367008(214748367008) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.181921:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227069312:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.181940:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227069696:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.183674:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985602ee1a40 x1874669227069696/t214748367008(214748367008) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.183687:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.183720:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.183728:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.183735:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367008 00000100:00080000:0.0:1787826991.183762:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985602ee1680 x1874669227070080/t214748367009(214748367009) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.183771:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227069696:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.183784:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227070080:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.185987:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985602ee1680 x1874669227070080/t214748367009(214748367009) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.186003:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.186007:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.186013:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.186021:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367009 00000100:00080000:0.0:1787826991.186076:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985602ee0780 x1874669227070464/t214748367010(214748367010) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.186088:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227070080:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.186104:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227070464:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.188294:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985602ee0780 x1874669227070464/t214748367010(214748367010) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.188309:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.188312:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.188318:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.188326:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367010 00000100:00080000:0.0:1787826991.188362:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985602ee30c0 x1874669227070848/t214748367011(214748367011) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.188374:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227070464:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.188393:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227070848:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.190148:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985602ee30c0 x1874669227070848/t214748367011(214748367011) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.190159:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.190163:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.190169:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.190176:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367011 00000100:00080000:0.0:1787826991.190199:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985602ee12c0 x1874669227071232/t214748367012(214748367012) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.190209:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227070848:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.190222:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227071232:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.192097:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985602ee12c0 x1874669227071232/t214748367012(214748367012) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.192109:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.192111:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.192116:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.192122:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367012 00000100:00080000:0.0:1787826991.192146:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985602ee0f00 x1874669227071616/t214748367013(214748367013) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.192157:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227071232:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.192171:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee0f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227071616:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.193937:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985602ee0f00 x1874669227071616/t214748367013(214748367013) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.193945:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.193947:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.193951:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.193956:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367013 00000100:00080000:0.0:1787826991.193970:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985602ee21c0 x1874669227072000/t214748367014(214748367014) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.193976:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee0f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227071616:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.193984:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227072000:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.195434:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985602ee21c0 x1874669227072000/t214748367014(214748367014) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.195445:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.195447:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.195452:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.195459:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367014 00000100:00080000:0.0:1787826991.195479:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98561408c3c0 x1874669227072384/t214748367015(214748367015) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.195489:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227072000:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.195501:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98561408c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227072384:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.197327:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98561408c3c0 x1874669227072384/t214748367015(214748367015) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.197338:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.197341:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.197346:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.197354:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367015 00000100:00080000:0.0:1787826991.197375:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98561408f0c0 x1874669227072768/t214748367016(214748367016) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.197382:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98561408c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227072384:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.197393:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98561408f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227072768:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.199900:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98561408f0c0 x1874669227072768/t214748367016(214748367016) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.199920:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.199923:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.199928:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.199935:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367016 00000100:00080000:0.0:1787826991.199968:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98561408cf00 x1874669227073152/t214748367017(214748367017) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.199979:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98561408f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227072768:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.199991:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98561408cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227073152:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.201864:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98561408cf00 x1874669227073152/t214748367017(214748367017) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.201875:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.201878:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.201883:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.201891:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367017 00000100:00080000:0.0:1787826991.201912:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98561408de00 x1874669227073536/t214748367018(214748367018) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.201922:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98561408cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227073152:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.201934:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98561408de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227073536:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.203444:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98561408de00 x1874669227073536/t214748367018(214748367018) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.203458:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.203461:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.203468:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.203476:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367018 00000100:00080000:0.0:1787826991.203512:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98561408e1c0 x1874669227073920/t214748367019(214748367019) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.203522:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98561408de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227073536:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.203541:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98561408e1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227073920:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.205093:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98561408e1c0 x1874669227073920/t214748367019(214748367019) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.205107:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.205110:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.205114:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.205119:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367019 00000100:00080000:0.0:1787826991.205142:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98561408f480 x1874669227074304/t214748367020(214748367020) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.205152:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98561408e1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227073920:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.205162:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98561408f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227074304:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.206676:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98561408f480 x1874669227074304/t214748367020(214748367020) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.206687:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.206689:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.206726:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.206734:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367020 00000100:00080000:0.0:1787826991.206754:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98561408c000 x1874669227074688/t214748367021(214748367021) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.206845:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98561408f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227074304:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.206862:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98561408c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227074688:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.208412:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98561408c000 x1874669227074688/t214748367021(214748367021) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.208422:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.208425:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.208431:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.208438:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367021 00000100:00080000:0.0:1787826991.208455:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98561408f840 x1874669227075072/t214748367022(214748367022) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826978 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.208464:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98561408c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227074688:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.208477:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98561408f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227075072:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.209829:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98561408f840 x1874669227075072/t214748367022(214748367022) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.209840:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.209842:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.209846:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.209854:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367022 00000100:00080000:0.0:1787826991.209870:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98561408c780 x1874669227075456/t214748367023(214748367023) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.209879:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98561408f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227075072:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.209890:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98561408c780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227075456:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.211479:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98561408c780 x1874669227075456/t214748367023(214748367023) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.211489:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.211493:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.211498:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.211506:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367023 00000100:00080000:0.0:1787826991.211526:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9856084c9680 x1874669227075840/t214748367024(214748367024) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.211535:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98561408c780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227075456:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.211545:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856084c9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227075840:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.212934:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff9856084c9680 x1874669227075840/t214748367024(214748367024) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.212945:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.212947:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.212953:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.212960:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367024 00000100:00080000:0.0:1787826991.212979:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9856084c83c0 x1874669227076224/t214748367025(214748367025) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.212990:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856084c9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227075840:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.213002:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856084c83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227076224:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.214451:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff9856084c83c0 x1874669227076224/t214748367025(214748367025) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.214461:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.214464:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.214469:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.214477:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367025 00000100:00080000:0.0:1787826991.214495:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9856084cb480 x1874669227076608/t214748367026(214748367026) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.214501:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856084c83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227076224:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.214512:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856084cb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227076608:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.215740:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff9856084cb480 x1874669227076608/t214748367026(214748367026) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.215747:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.215748:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.215751:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.215756:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367026 00000100:00080000:0.0:1787826991.215771:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9856084c9a40 x1874669227076992/t214748367027(214748367027) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.215777:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856084cb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227076608:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.215786:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856084c9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227076992:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.217291:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff9856084c9a40 x1874669227076992/t214748367027(214748367027) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.217302:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.217305:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.217311:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.217316:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367027 00000100:00080000:0.0:1787826991.217332:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9856084cb0c0 x1874669227077376/t214748367028(214748367028) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.217339:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856084c9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227076992:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.217350:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856084cb0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227077376:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.218748:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff9856084cb0c0 x1874669227077376/t214748367028(214748367028) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.218758:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.218760:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.218765:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.218772:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367028 00000100:00080000:0.0:1787826991.218790:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9856084c8780 x1874669227077760/t214748367029(214748367029) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.218798:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856084cb0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227077376:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.218808:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856084c8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227077760:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.220360:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff9856084c8780 x1874669227077760/t214748367029(214748367029) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.220369:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.220371:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.220375:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.220380:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367029 00000100:00080000:0.0:1787826991.220406:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9856084c8000 x1874669227078144/t214748367030(214748367030) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.220412:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856084c8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227077760:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.220423:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856084c8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227078144:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.223224:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff9856084c8000 x1874669227078144/t214748367030(214748367030) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.223241:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.223244:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.223251:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.223258:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367030 00000100:00080000:0.0:1787826991.223302:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9856084cb840 x1874669227078528/t214748367031(214748367031) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.223313:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856084c8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227078144:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.223335:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856084cb840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227078528:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.225337:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff9856084cb840 x1874669227078528/t214748367031(214748367031) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.225351:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.225355:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.225361:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.225369:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367031 00000100:00080000:0.0:1787826991.225413:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9856084cbc00 x1874669227078912/t214748367032(214748367032) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.225426:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856084cb840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227078528:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.225443:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856084cbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227078912:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.227217:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff9856084cbc00 x1874669227078912/t214748367032(214748367032) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.227229:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.227231:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.227237:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.227245:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367032 00000100:00080000:0.0:1787826991.227274:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9856084c8b40 x1874669227079296/t214748367033(214748367033) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.227284:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856084cbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227078912:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.227299:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856084c8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227079296:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.229556:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff9856084c8b40 x1874669227079296/t214748367033(214748367033) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.229581:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.229585:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.229592:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.229598:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367033 00000100:00080000:0.0:1787826991.229641:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9856084ca940 x1874669227079680/t214748367034(214748367034) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.229657:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856084c8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227079296:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.229681:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856084ca940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227079680:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.232106:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff9856084ca940 x1874669227079680/t214748367034(214748367034) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.232125:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.232128:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.232135:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.232148:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367034 00000100:00080000:0.0:1787826991.232208:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9856084ca1c0 x1874669227080064/t214748367035(214748367035) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.232223:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856084ca940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227079680:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.232247:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856084ca1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227080064:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.236755:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff9856084ca1c0 x1874669227080064/t214748367035(214748367035) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.236773:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.236779:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.236796:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.236805:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367035 00000100:00080000:0.0:1787826991.236871:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9856084ca580 x1874669227080448/t214748367036(214748367036) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.236885:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856084ca1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227080064:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.236914:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856084ca580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227080448:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.240217:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff9856084ca580 x1874669227080448/t214748367036(214748367036) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.240236:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.240239:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.240245:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.240254:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367036 00000100:00080000:0.0:1787826991.240310:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9856084c8f00 x1874669227080832/t214748367037(214748367037) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.240325:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856084ca580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227080448:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.240360:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856084c8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227080832:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.242666:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff9856084c8f00 x1874669227080832/t214748367037(214748367037) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.242679:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.242683:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.242688:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.242729:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367037 00000100:00080000:0.0:1787826991.242793:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9856084c9e00 x1874669227081216/t214748367038(214748367038) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.242809:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856084c8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227080832:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.242833:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856084c9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227081216:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.245392:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff9856084c9e00 x1874669227081216/t214748367038(214748367038) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.245409:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.245412:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.245419:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.245428:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367038 00000100:00080000:0.0:1787826991.245484:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9856084c92c0 x1874669227081600/t214748367039(214748367039) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.245504:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856084c9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227081216:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.245528:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856084c92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227081600:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.248282:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff9856084c92c0 x1874669227081600/t214748367039(214748367039) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.248299:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.248304:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.248324:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.248333:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367039 00000100:00080000:0.0:1787826991.248386:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9856084cad00 x1874669227081984/t214748367040(214748367040) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.248400:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856084c92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227081600:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.248425:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856084cad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227081984:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.250676:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff9856084cad00 x1874669227081984/t214748367040(214748367040) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.250723:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.250726:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.250733:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.250741:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367040 00000100:00080000:0.0:1787826991.250792:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9856056f9e00 x1874669227082368/t214748367041(214748367041) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.250813:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856084cad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227081984:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.250837:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856056f9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227082368:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.252622:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff9856056f9e00 x1874669227082368/t214748367041(214748367041) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.252636:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.252639:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.252645:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.252653:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367041 00000100:00080000:0.0:1787826991.252729:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9856056f92c0 x1874669227082752/t214748367042(214748367042) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.252742:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856056f9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227082368:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.252760:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856056f92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227082752:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.254640:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff9856056f92c0 x1874669227082752/t214748367042(214748367042) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.254651:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.254653:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.254657:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.254663:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367042 00000100:00080000:0.0:1787826991.254730:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9856056fad00 x1874669227083136/t214748367043(214748367043) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.254738:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856056f92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227082752:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.254752:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856056fad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227083136:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.257670:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff9856056fad00 x1874669227083136/t214748367043(214748367043) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.257685:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.257716:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.257724:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.257733:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367043 00000100:00080000:0.0:1787826991.257797:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9856056f9680 x1874669227083520/t214748367044(214748367044) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.257810:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856056fad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227083136:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.257831:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856056f9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227083520:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.261632:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff9856056f9680 x1874669227083520/t214748367044(214748367044) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.261648:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.261652:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.261658:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.261667:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367044 00000100:00080000:0.0:1787826991.261766:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9856056f83c0 x1874669227083904/t214748367045(214748367045) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.261780:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856056f9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227083520:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.261803:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856056f83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227083904:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.263383:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff9856056f83c0 x1874669227083904/t214748367045(214748367045) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.263396:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.263399:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.263404:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.263411:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367045 00000100:00080000:0.0:1787826991.263451:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9856056fb480 x1874669227084288/t214748367046(214748367046) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.263461:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856056f83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227083904:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.263479:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856056fb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227084288:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.264965:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff9856056fb480 x1874669227084288/t214748367046(214748367046) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.264978:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.264981:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.264988:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.264996:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367046 00000100:00080000:0.0:1787826991.265023:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9856056f9a40 x1874669227084672/t214748367047(214748367047) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.265035:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856056fb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227084288:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.265066:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856056f9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227084672:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.266569:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff9856056f9a40 x1874669227084672/t214748367047(214748367047) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.266580:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.266583:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.266588:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.266596:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367047 00000100:00080000:0.0:1787826991.266619:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9856056fb0c0 x1874669227085056/t214748367048(214748367048) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.266628:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856056f9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227084672:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.266639:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856056fb0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227085056:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.267923:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff9856056fb0c0 x1874669227085056/t214748367048(214748367048) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.267934:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.267937:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.267942:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.267949:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367048 00000100:00080000:0.0:1787826991.267977:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9856056f8780 x1874669227085440/t214748367049(214748367049) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.267988:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856056fb0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227085056:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.267999:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856056f8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227085440:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.269536:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff9856056f8780 x1874669227085440/t214748367049(214748367049) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.269547:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.269550:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.269555:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.269563:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367049 00000100:00080000:0.0:1787826991.269587:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985602ee3c00 x1874669227085824/t214748367050(214748367050) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.269598:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856056f8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227085440:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.269613:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227085824:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.271147:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985602ee3c00 x1874669227085824/t214748367050(214748367050) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.271156:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.271159:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.271163:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.271169:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367050 00000100:00080000:0.0:1787826991.271192:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985602ee1e00 x1874669227086208/t214748367051(214748367051) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.271201:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227085824:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.271213:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985602ee1e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227086208:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.272584:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985602ee1e00 x1874669227086208/t214748367051(214748367051) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.272591:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.272593:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.272596:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.272601:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367051 00000100:00080000:0.0:1787826991.272617:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9856085370c0 x1874669227086592/t214748367052(214748367052) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.272623:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985602ee1e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227086208:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.272630:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085370c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227086592:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.274146:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff9856085370c0 x1874669227086592/t214748367052(214748367052) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.274157:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.274159:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.274165:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.274174:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367052 00000100:00080000:0.0:1787826991.274197:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985608534780 x1874669227086976/t214748367053(214748367053) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.274207:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085370c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227086592:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.274219:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608534780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227086976:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.275659:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985608534780 x1874669227086976/t214748367053(214748367053) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.275670:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.275673:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.275678:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.275686:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367053 00000100:00080000:0.0:1787826991.275740:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985608534000 x1874669227087360/t214748367054(214748367054) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.275751:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608534780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227086976:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.275775:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608534000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227087360:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.277106:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985608534000 x1874669227087360/t214748367054(214748367054) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.277113:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.277115:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.277118:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.277123:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367054 00000100:00080000:0.0:1787826991.277143:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9856056f8000 x1874669227087744/t214748367055(214748367055) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.277153:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608534000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227087360:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.277164:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856056f8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227087744:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.278558:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff9856056f8000 x1874669227087744/t214748367055(214748367055) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.278567:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.278570:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.278573:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.278579:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367055 00000100:00080000:0.0:1787826991.278601:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9856056fb840 x1874669227088128/t214748367056(214748367056) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.278607:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856056f8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227087744:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.278618:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856056fb840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227088128:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.280387:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff9856056fb840 x1874669227088128/t214748367056(214748367056) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.280402:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.280404:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.280411:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.280419:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367056 00000100:00080000:0.0:1787826991.280461:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9856056fbc00 x1874669227088512/t214748367057(214748367057) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.280473:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856056fb840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227088128:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.280488:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856056fbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227088512:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.282237:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff9856056fbc00 x1874669227088512/t214748367057(214748367057) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.282245:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.282248:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.282251:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.282256:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367057 00000100:00080000:0.0:1787826991.282285:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9856056f8b40 x1874669227088896/t214748367058(214748367058) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.282293:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856056fbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227088512:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.282307:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856056f8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227088896:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.283943:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff9856056f8b40 x1874669227088896/t214748367058(214748367058) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.283956:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.283959:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.283965:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.283987:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367058 00000100:00080000:0.0:1787826991.284023:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9856056fa940 x1874669227089280/t214748367059(214748367059) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.284037:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856056f8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227088896:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.284062:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856056fa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227089280:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.286452:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff9856056fa940 x1874669227089280/t214748367059(214748367059) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.286462:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.286465:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.286469:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.286474:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367059 00000100:00080000:0.0:1787826991.286515:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9856056fa1c0 x1874669227089664/t214748367060(214748367060) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.286526:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856056fa940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227089280:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.286543:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856056fa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227089664:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.288422:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff9856056fa1c0 x1874669227089664/t214748367060(214748367060) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.288439:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.288442:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.288449:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.288458:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367060 00000100:00080000:0.0:1787826991.288508:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9856056fa580 x1874669227090048/t214748367061(214748367061) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.288521:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856056fa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227089664:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.288544:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856056fa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227090048:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.290353:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff9856056fa580 x1874669227090048/t214748367061(214748367061) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.290366:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.290371:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.290377:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.290384:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367061 00000100:00080000:0.0:1787826991.290423:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9856056f8f00 x1874669227090432/t214748367062(214748367062) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.290436:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856056fa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227090048:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.290453:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856056f8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227090432:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.291971:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff9856056f8f00 x1874669227090432/t214748367062(214748367062) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.291983:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.291985:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.291990:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.291997:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367062 00000100:00080000:0.0:1787826991.292026:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9856057861c0 x1874669227090816/t214748367063(214748367063) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.292037:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856056f8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227090432:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.292061:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856057861c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227090816:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.293497:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff9856057861c0 x1874669227090816/t214748367063(214748367063) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.293506:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.293509:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.293513:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.293521:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367063 00000100:00080000:0.0:1787826991.293546:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985605786580 x1874669227091200/t214748367064(214748367064) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.293556:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856057861c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227090816:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.293567:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985605786580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227091200:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.294892:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985605786580 x1874669227091200/t214748367064(214748367064) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.294901:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.294903:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.294906:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.294911:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367064 00000100:00080000:0.0:1787826991.294928:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985608537840 x1874669227091584/t214748367065(214748367065) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.294934:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985605786580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227091200:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.294944:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608537840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227091584:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.296318:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985608537840 x1874669227091584/t214748367065(214748367065) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.296326:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.296329:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.296332:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.296336:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367065 00000100:00080000:0.0:1787826991.296353:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985608537c00 x1874669227091968/t214748367066(214748367066) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.296359:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608537840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227091584:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.296367:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608537c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227091968:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.297649:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985608537c00 x1874669227091968/t214748367066(214748367066) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.297660:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.297662:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.297668:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.297676:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367066 00000100:00080000:0.0:1787826991.297728:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985608534b40 x1874669227092352/t214748367067(214748367067) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.297739:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608537c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227091968:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.297750:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608534b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227092352:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.299255:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985608534b40 x1874669227092352/t214748367067(214748367067) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.299273:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.299276:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.299281:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.299288:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367067 00000100:00080000:0.0:1787826991.299311:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985608536940 x1874669227092736/t214748367068(214748367068) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.299322:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608534b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227092352:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.299334:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608536940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227092736:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.300674:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985608536940 x1874669227092736/t214748367068(214748367068) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.300683:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.300685:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.300718:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.300726:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367068 00000100:00080000:0.0:1787826991.300754:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9856085361c0 x1874669227093120/t214748367069(214748367069) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.300764:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608536940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227092736:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.300774:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085361c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227093120:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.302092:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff9856085361c0 x1874669227093120/t214748367069(214748367069) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.302102:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.302105:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.302109:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.302115:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367069 00000100:00080000:0.0:1787826991.302133:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985608536580 x1874669227093504/t214748367070(214748367070) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.302142:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085361c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227093120:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.302153:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608536580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227093504:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.303546:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985608536580 x1874669227093504/t214748367070(214748367070) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.303557:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.303560:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.303565:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.303573:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367070 00000100:00080000:0.0:1787826991.303595:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985608534f00 x1874669227093888/t214748367071(214748367071) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.303605:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608536580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227093504:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.303617:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608534f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227093888:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.304978:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985608534f00 x1874669227093888/t214748367071(214748367071) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.304988:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.304991:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.304996:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.305011:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367071 00000100:00080000:0.0:1787826991.305033:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985608535e00 x1874669227094272/t214748367072(214748367072) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.305040:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608534f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227093888:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.305065:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608535e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227094272:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.306456:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985608535e00 x1874669227094272/t214748367072(214748367072) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.306466:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.306469:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.306474:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.306482:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367072 00000100:00080000:0.0:1787826991.306503:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9856085352c0 x1874669227094656/t214748367073(214748367073) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.306513:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608535e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227094272:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.306526:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085352c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227094656:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.307997:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff9856085352c0 x1874669227094656/t214748367073(214748367073) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.308008:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.308012:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.308018:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.308026:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367073 00000100:00080000:0.0:1787826991.308062:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985608536d00 x1874669227095040/t214748367074(214748367074) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.308073:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085352c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227094656:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.308088:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608536d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227095040:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.310914:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985608536d00 x1874669227095040/t214748367074(214748367074) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.310931:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.310934:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.310941:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.310949:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367074 00000100:00080000:0.0:1787826991.311003:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985608535680 x1874669227095424/t214748367075(214748367075) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.311016:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608536d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227095040:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.311037:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608535680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227095424:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.313028:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985608535680 x1874669227095424/t214748367075(214748367075) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.313039:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.313042:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.313057:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.313063:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367075 00000100:00080000:0.0:1787826991.313117:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9856085343c0 x1874669227095808/t214748367076(214748367076) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.313140:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608535680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227095424:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.313161:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085343c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227095808:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.314900:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff9856085343c0 x1874669227095808/t214748367076(214748367076) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.314910:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.314913:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.314921:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.314928:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367076 00000100:00080000:0.0:1787826991.314973:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985608537480 x1874669227096192/t214748367077(214748367077) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.314985:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085343c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227095808:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.315001:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608537480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227096192:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.316404:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985608537480 x1874669227096192/t214748367077(214748367077) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.316414:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.316417:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.316423:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.316430:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367077 00000100:00080000:0.0:1787826991.316464:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985608535a40 x1874669227096576/t214748367078(214748367078) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.316483:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608537480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227096192:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.316498:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608535a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227096576:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.318163:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985608535a40 x1874669227096576/t214748367078(214748367078) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.318175:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.318178:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.318184:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.318192:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367078 00000100:00080000:0.0:1787826991.318246:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985610e1b0c0 x1874669227096960/t214748367079(214748367079) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.318257:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608535a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227096576:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.318283:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e1b0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227096960:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.320091:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985610e1b0c0 x1874669227096960/t214748367079(214748367079) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.320100:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.320103:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.320107:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.320112:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367079 00000100:00080000:0.0:1787826991.320154:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985610e18780 x1874669227097344/t214748367080(214748367080) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.320164:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e1b0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227096960:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.320178:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e18780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227097344:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.321804:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985610e18780 x1874669227097344/t214748367080(214748367080) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.321817:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.321819:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.321825:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.321834:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367080 00000100:00080000:0.0:1787826991.321865:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985610e18000 x1874669227097728/t214748367081(214748367081) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.321878:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e18780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227097344:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.321889:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e18000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227097728:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.323490:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985610e18000 x1874669227097728/t214748367081(214748367081) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.323509:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.323512:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.323518:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.323527:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367081 00000100:00080000:0.0:1787826991.323564:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985610e1b840 x1874669227098112/t214748367082(214748367082) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.323574:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e18000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227097728:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.323594:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e1b840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227098112:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.325443:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985610e1b840 x1874669227098112/t214748367082(214748367082) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.325453:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.325455:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.325459:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.325465:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367082 00000100:00080000:0.0:1787826991.325505:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985610e1bc00 x1874669227098496/t214748367083(214748367083) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.325515:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e1b840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227098112:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.325532:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e1bc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227098496:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.327168:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985610e1bc00 x1874669227098496/t214748367083(214748367083) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.327182:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.327185:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.327192:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.327201:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367083 00000100:00080000:0.0:1787826991.327232:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985610e18b40 x1874669227098880/t214748367084(214748367084) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.327243:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e1bc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227098496:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.327264:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e18b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227098880:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.328654:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985610e18b40 x1874669227098880/t214748367084(214748367084) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.328665:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.328667:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.328672:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.328679:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367084 00000100:00080000:0.0:1787826991.328730:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985610e1a940 x1874669227099264/t214748367085(214748367085) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.328740:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e18b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227098880:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.328753:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e1a940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227099264:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.330130:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985610e1a940 x1874669227099264/t214748367085(214748367085) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.330143:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.330146:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.330153:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.330159:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367085 00000100:00080000:0.0:1787826991.330190:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985610e1a1c0 x1874669227099648/t214748367086(214748367086) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.330199:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e1a940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227099264:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.330214:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e1a1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227099648:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.331640:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985610e1a1c0 x1874669227099648/t214748367086(214748367086) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.331648:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.331650:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.331655:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.331662:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367086 00000100:00080000:0.0:1787826991.331686:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985610e1a580 x1874669227100032/t214748367087(214748367087) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.331722:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e1a1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227099648:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.331733:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e1a580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227100032:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.333218:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985610e1a580 x1874669227100032/t214748367087(214748367087) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.333230:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.333233:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.333239:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.333246:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367087 00000100:00080000:0.0:1787826991.333276:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985610e18f00 x1874669227100416/t214748367088(214748367088) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.333287:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e1a580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227100032:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.333302:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e18f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227100416:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.334732:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985610e18f00 x1874669227100416/t214748367088(214748367088) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.334743:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.334745:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.334750:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.334758:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367088 00000100:00080000:0.0:1787826991.334786:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985610e19e00 x1874669227100800/t214748367089(214748367089) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.334797:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e18f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227100416:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.334810:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e19e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227100800:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.336182:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985610e19e00 x1874669227100800/t214748367089(214748367089) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.336190:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.336192:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.336196:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.336201:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367089 00000100:00080000:0.0:1787826991.336223:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985610e192c0 x1874669227101184/t214748367090(214748367090) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.336229:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e19e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227100800:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.336239:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e192c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227101184:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.337599:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985610e192c0 x1874669227101184/t214748367090(214748367090) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.337611:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.337614:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.337619:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.337626:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367090 00000100:00080000:0.0:1787826991.337661:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985610e1ad00 x1874669227101568/t214748367091(214748367091) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.337672:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e192c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227101184:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.337685:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e1ad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227101568:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.338884:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985610e1ad00 x1874669227101568/t214748367091(214748367091) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.338890:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.338893:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.338895:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.338899:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367091 00000100:00080000:0.0:1787826991.338913:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985610e19680 x1874669227101952/t214748367092(214748367092) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.338918:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e1ad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227101568:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.338925:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e19680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227101952:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.340084:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985610e19680 x1874669227101952/t214748367092(214748367092) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.340091:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.340098:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.340101:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.340108:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367092 00000100:00080000:0.0:1787826991.340125:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985610e183c0 x1874669227102336/t214748367093(214748367093) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.340131:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e19680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227101952:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.340139:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e183c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227102336:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.342511:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985610e183c0 x1874669227102336/t214748367093(214748367093) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.342528:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.342531:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.342539:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.342547:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367093 00000100:00080000:0.0:1787826991.342593:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985610e1b480 x1874669227102720/t214748367094(214748367094) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.342606:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e183c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227102336:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.342630:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e1b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227102720:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.345313:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985610e1b480 x1874669227102720/t214748367094(214748367094) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.345334:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.345336:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.345344:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.345352:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367094 00000100:00080000:0.0:1787826991.345419:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985610e19a40 x1874669227103104/t214748367095(214748367095) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.345429:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e1b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227102720:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.345455:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e19a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227103104:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.347493:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985610e19a40 x1874669227103104/t214748367095(214748367095) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.347508:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.347512:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.347519:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.347526:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367095 00000100:00080000:0.0:1787826991.347577:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9856085470c0 x1874669227103488/t214748367096(214748367096) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.347590:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e19a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227103104:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.347608:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085470c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227103488:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.348831:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607ce5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669227149952:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787826991.348836:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607ce7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2383:1874669227149824:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787826991.348916:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607ce43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669227150080:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787826991.348921:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607ce6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669227150208:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787826991.350101:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607ce7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2383:1874669227149824:192.168.203.114@tcp:400:kworker.0 00000100:00080000:0.0:1787826991.350103:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff9856085470c0 x1874669227103488/t214748367096(214748367096) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.350113:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.350116:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.350130:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.350141:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367096 00000100:00100000:1.0:1787826991.350166:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607ce5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669227149952:192.168.203.114@tcp:400:kworker.0 00000100:00080000:0.0:1787826991.350196:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985608544780 x1874669227103872/t214748367097(214748367097) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.350209:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085470c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227103488:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:1.0:1787826991.350220:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607ce43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669227150080:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787826991.350224:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608544780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227103872:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:1.0:1787826991.350247:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607ce6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669227150208:192.168.203.114@tcp:400:kworker.0 00000100:00080000:0.0:1787826991.351755:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985608544780 x1874669227103872/t214748367097(214748367097) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.351766:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.351768:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.351774:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.351783:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367097 00000100:00080000:0.0:1787826991.351815:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985608544000 x1874669227104256/t214748367098(214748367098) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.351826:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608544780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227103872:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.351839:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608544000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227104256:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.353193:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985608544000 x1874669227104256/t214748367098(214748367098) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.353204:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.353207:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.353212:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.353220:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367098 00000100:00080000:0.0:1787826991.353245:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985608547840 x1874669227104640/t214748367099(214748367099) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.353256:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608544000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227104256:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.353268:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608547840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227104640:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.354616:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985608547840 x1874669227104640/t214748367099(214748367099) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.354625:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.354626:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.354630:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.354635:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367099 00000100:00080000:0.0:1787826991.354659:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985608547c00 x1874669227105024/t214748367100(214748367100) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.354665:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608547840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227104640:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.354675:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608547c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227105024:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.355994:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985608547c00 x1874669227105024/t214748367100(214748367100) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.356006:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.356008:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.356013:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.356021:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367100 00000100:00080000:0.0:1787826991.356043:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985608544b40 x1874669227105408/t214748367101(214748367101) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.356065:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608547c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227105024:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.356076:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608544b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227105408:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.357319:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985608544b40 x1874669227105408/t214748367101(214748367101) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.357329:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.357331:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.357345:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.357354:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367101 00000100:00080000:0.0:1787826991.357380:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985608546940 x1874669227105792/t214748367102(214748367102) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.357393:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608544b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227105408:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.357405:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608546940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227105792:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.358822:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985608546940 x1874669227105792/t214748367102(214748367102) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.358830:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.358832:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.358835:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.358841:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367102 00000100:00080000:0.0:1787826991.358859:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9856085461c0 x1874669227106176/t214748367103(214748367103) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.358865:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608546940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227105792:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.358874:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085461c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227106176:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.360306:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff9856085461c0 x1874669227106176/t214748367103(214748367103) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.360317:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.360320:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.360326:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.360334:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367103 00000100:00080000:0.0:1787826991.360359:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985608546580 x1874669227106560/t214748367104(214748367104) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.360366:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085461c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227106176:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.360375:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608546580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227106560:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.361677:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985608546580 x1874669227106560/t214748367104(214748367104) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.361684:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.361686:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.361711:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.361716:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367104 00000100:00080000:0.0:1787826991.361736:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985608544f00 x1874669227106944/t214748367105(214748367105) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.361742:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608546580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227106560:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.361751:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608544f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227106944:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.364021:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985608544f00 x1874669227106944/t214748367105(214748367105) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.364033:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.364035:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.364041:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.364057:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367105 00000100:00080000:0.0:1787826991.364088:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985608545e00 x1874669227107328/t214748367106(214748367106) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.364096:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608544f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227106944:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.364108:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608545e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227107328:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.365604:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985608545e00 x1874669227107328/t214748367106(214748367106) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.365619:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.365622:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.365629:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.365637:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367106 00000100:00080000:0.0:1787826991.365724:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9856085452c0 x1874669227107712/t214748367107(214748367107) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.365738:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608545e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227107328:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.365758:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085452c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227107712:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.367102:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff9856085452c0 x1874669227107712/t214748367107(214748367107) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.367112:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.367114:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.367120:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.367127:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367107 00000100:00080000:0.0:1787826991.367158:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985608546d00 x1874669227108096/t214748367108(214748367108) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.367167:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085452c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227107712:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.367179:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608546d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227108096:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.369014:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985608546d00 x1874669227108096/t214748367108(214748367108) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.369026:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.369029:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.369035:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.369042:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367108 00000100:00080000:0.0:1787826991.369096:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985608545680 x1874669227108480/t214748367109(214748367109) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.369118:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608546d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227108096:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.369139:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608545680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227108480:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.370762:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985608545680 x1874669227108480/t214748367109(214748367109) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.370774:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.370777:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.370783:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.370791:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367109 00000100:00080000:0.0:1787826991.370824:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9856085443c0 x1874669227108864/t214748367110(214748367110) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.370834:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608545680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227108480:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.370846:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085443c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227108864:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.372181:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff9856085443c0 x1874669227108864/t214748367110(214748367110) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.372190:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.372193:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.372198:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.372206:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367110 00000100:00080000:0.0:1787826991.372229:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985608547480 x1874669227109248/t214748367111(214748367111) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.372238:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085443c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227108864:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.372249:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608547480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227109248:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.373852:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985608547480 x1874669227109248/t214748367111(214748367111) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.373861:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.373862:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.373866:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.373871:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367111 00000100:00080000:0.0:1787826991.373908:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985608545a40 x1874669227109632/t214748367112(214748367112) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.373918:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608547480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227109248:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.373933:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608545a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227109632:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.375538:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985608545a40 x1874669227109632/t214748367112(214748367112) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.375549:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.375551:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.375560:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.375565:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367112 00000100:00080000:0.0:1787826991.375603:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985607f0a580 x1874669227110016/t214748367113(214748367113) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.375613:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608545a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227109632:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.375628:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607f0a580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227110016:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.377603:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985607f0a580 x1874669227110016/t214748367113(214748367113) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.377613:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.377615:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.377619:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.377624:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367113 00000100:00080000:0.0:1787826991.377667:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985607f08f00 x1874669227110400/t214748367114(214748367114) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.377674:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607f0a580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227110016:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.377689:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607f08f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227110400:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.379608:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985607f08f00 x1874669227110400/t214748367114(214748367114) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.379617:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.379619:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.379623:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.379628:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367114 00000100:00080000:0.0:1787826991.379670:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985607f09e00 x1874669227110784/t214748367115(214748367115) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.379677:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607f08f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227110400:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.379710:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607f09e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227110784:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.382159:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985607f09e00 x1874669227110784/t214748367115(214748367115) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.382175:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.382178:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.382192:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.382200:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367115 00000100:00080000:0.0:1787826991.382256:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985610e803c0 x1874669227111168/t214748367116(214748367116) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.382268:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607f09e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227110784:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.382286:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227111168:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.384617:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985610e803c0 x1874669227111168/t214748367116(214748367116) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.384632:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.384634:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.384640:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.384648:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367116 00000100:00080000:0.0:1787826991.384726:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985610e82580 x1874669227111552/t214748367117(214748367117) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.384740:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227111168:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.384760:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227111552:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.386602:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985610e82580 x1874669227111552/t214748367117(214748367117) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.386614:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.386616:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.386628:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.386636:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367117 00000100:00080000:0.0:1787826991.386666:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985610e80b40 x1874669227111936/t214748367118(214748367118) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.386676:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227111552:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.386688:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227111936:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.388336:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985610e80b40 x1874669227111936/t214748367118(214748367118) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.388344:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.388345:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.388349:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.388354:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367118 00000100:00080000:0.0:1787826991.388380:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985610e80f00 x1874669227112320/t214748367119(214748367119) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.388409:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227111936:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.388422:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227112320:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.390346:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985610e80f00 x1874669227112320/t214748367119(214748367119) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.390355:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.390357:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.390362:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.390388:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367119 00000100:00080000:0.0:1787826991.390426:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985610e82d00 x1874669227112704/t214748367120(214748367120) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.390437:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227112320:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.390449:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227112704:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.392603:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985610e82d00 x1874669227112704/t214748367120(214748367120) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.392616:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.392619:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.392626:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.392634:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367120 00000100:00080000:0.0:1787826991.392668:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985610e80780 x1874669227113088/t214748367121(214748367121) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.392686:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227112704:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.392727:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e80780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227113088:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.394806:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985610e80780 x1874669227113088/t214748367121(214748367121) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.394818:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.394821:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.394827:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.394834:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367121 00000100:00080000:0.0:1787826991.394874:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985610e81e00 x1874669227113472/t214748367122(214748367122) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.394885:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e80780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227113088:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.394897:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227113472:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.396603:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985610e81e00 x1874669227113472/t214748367122(214748367122) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.396610:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.396611:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.396614:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.396619:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367122 00000100:00080000:0.0:1787826991.396645:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985610e80000 x1874669227113856/t214748367123(214748367123) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.396654:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227113472:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.396666:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227113856:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.398139:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985610e80000 x1874669227113856/t214748367123(214748367123) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.398145:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.398147:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.398150:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.398154:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367123 00000100:00080000:0.0:1787826991.398171:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985610e83480 x1874669227114240/t214748367124(214748367124) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.398183:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227113856:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.398191:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e83480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227114240:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.399779:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985610e83480 x1874669227114240/t214748367124(214748367124) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.399787:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.399789:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.399793:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.399798:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367124 00000100:00080000:0.0:1787826991.399821:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985607f0ad00 x1874669227114624/t214748367125(214748367125) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.399828:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e83480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227114240:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.399838:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607f0ad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227114624:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.401514:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985607f0ad00 x1874669227114624/t214748367125(214748367125) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.401531:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.401533:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.401538:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.401545:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367125 00000100:00080000:0.0:1787826991.401570:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985607f09680 x1874669227115008/t214748367126(214748367126) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.401580:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607f0ad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227114624:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.401592:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607f09680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227115008:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.403319:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985607f09680 x1874669227115008/t214748367126(214748367126) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.403329:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.403332:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.403337:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.403345:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367126 00000100:00080000:0.0:1787826991.403371:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985607f083c0 x1874669227115392/t214748367127(214748367127) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.403382:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607f09680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227115008:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.403394:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607f083c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227115392:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.404961:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985607f083c0 x1874669227115392/t214748367127(214748367127) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.404968:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.404969:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.404973:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.404978:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367127 00000100:00080000:0.0:1787826991.405002:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985607f0b480 x1874669227115776/t214748367128(214748367128) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.405012:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607f083c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227115392:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.405031:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607f0b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227115776:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.406534:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985607f0b480 x1874669227115776/t214748367128(214748367128) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.406543:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.406545:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.406550:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.406556:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367128 00000100:00080000:0.0:1787826991.406578:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985607f09a40 x1874669227116160/t214748367129(214748367129) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.406586:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607f0b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227115776:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.406596:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607f09a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227116160:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.408180:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985607f09a40 x1874669227116160/t214748367129(214748367129) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.408187:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.408188:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.408191:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.408196:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367129 00000100:00080000:0.0:1787826991.408215:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985607f0b0c0 x1874669227116544/t214748367130(214748367130) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.408220:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607f09a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227116160:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.408228:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607f0b0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227116544:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.410207:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985607f0b0c0 x1874669227116544/t214748367130(214748367130) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.410215:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.410217:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.410220:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.410225:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367130 00000100:00080000:0.0:1787826991.410249:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985607f08780 x1874669227116928/t214748367131(214748367131) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.410256:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607f0b0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227116544:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.410265:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607f08780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227116928:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.412040:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985607f08780 x1874669227116928/t214748367131(214748367131) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.412061:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.412063:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.412069:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.412076:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367131 00000100:00080000:0.0:1787826991.412112:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985607f08000 x1874669227117312/t214748367132(214748367132) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.412120:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607f08780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227116928:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.412131:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607f08000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227117312:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.414064:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985607f08000 x1874669227117312/t214748367132(214748367132) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.414091:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.414093:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.414097:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.414103:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367132 00000100:00080000:0.0:1787826991.414127:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985607f0b840 x1874669227117696/t214748367133(214748367133) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.414136:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607f08000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227117312:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.414149:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607f0b840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227117696:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.416147:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985607f0b840 x1874669227117696/t214748367133(214748367133) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.416158:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.416161:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.416166:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.416173:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367133 00000100:00080000:0.0:1787826991.416208:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985607f0bc00 x1874669227118080/t214748367134(214748367134) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.416219:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607f0b840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227117696:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.416232:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607f0bc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227118080:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.419653:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985607f0bc00 x1874669227118080/t214748367134(214748367134) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.419667:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.419670:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.419676:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.419683:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367134 00000100:00080000:0.0:1787826991.419752:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985607f08b40 x1874669227118464/t214748367135(214748367135) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.419764:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607f0bc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227118080:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.419780:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607f08b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227118464:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.421617:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985607f08b40 x1874669227118464/t214748367135(214748367135) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.421626:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.421627:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.421631:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.421637:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367135 00000100:00080000:0.0:1787826991.421672:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985607f0a940 x1874669227118848/t214748367136(214748367136) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.421681:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607f08b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227118464:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.421719:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607f0a940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227118848:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.423150:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985607f0a940 x1874669227118848/t214748367136(214748367136) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.423157:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.423159:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.423162:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.423166:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367136 00000100:00080000:0.0:1787826991.423187:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985607f0a1c0 x1874669227119232/t214748367137(214748367137) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.423195:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607f0a940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227118848:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.423206:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607f0a1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227119232:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.424728:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985607f0a1c0 x1874669227119232/t214748367137(214748367137) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.424739:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.424741:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.424747:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.424753:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367137 00000100:00080000:0.0:1787826991.424777:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985607f092c0 x1874669227119616/t214748367138(214748367138) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.424785:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607f0a1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227119232:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.424796:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607f092c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227119616:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.427030:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985607f092c0 x1874669227119616/t214748367138(214748367138) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.427042:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.427059:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.427066:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.427073:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367138 00000100:00080000:0.0:1787826991.427110:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985608402d00 x1874669227120000/t214748367139(214748367139) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.427121:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607f092c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227119616:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.427135:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608402d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227120000:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.428898:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985608402d00 x1874669227120000/t214748367139(214748367139) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.428911:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.428913:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.428918:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.428924:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367139 00000100:00080000:0.0:1787826991.428960:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985608401680 x1874669227120384/t214748367140(214748367140) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.428969:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608402d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227120000:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.428980:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608401680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227120384:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.431170:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985608401680 x1874669227120384/t214748367140(214748367140) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.431190:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.431193:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.431199:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.431205:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367140 00000100:00080000:0.0:1787826991.431257:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9856084003c0 x1874669227120768/t214748367141(214748367141) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.431268:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608401680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227120384:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.431284:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856084003c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227120768:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.433432:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff9856084003c0 x1874669227120768/t214748367141(214748367141) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.433446:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.433448:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.433454:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.433460:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367141 00000100:00080000:0.0:1787826991.433502:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985608403480 x1874669227121152/t214748367142(214748367142) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.433513:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856084003c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227120768:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.433529:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608403480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227121152:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.435433:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985608403480 x1874669227121152/t214748367142(214748367142) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.435448:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.435451:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.435458:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.435465:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367142 00000100:00080000:0.0:1787826991.435527:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985608401a40 x1874669227121536/t214748367143(214748367143) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.435541:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608403480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227121152:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.435560:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608401a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227121536:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.437413:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985608401a40 x1874669227121536/t214748367143(214748367143) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.437425:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.437427:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.437433:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.437441:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367143 00000100:00080000:0.0:1787826991.437487:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9856084030c0 x1874669227121920/t214748367144(214748367144) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.437498:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608401a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227121536:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.437515:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856084030c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227121920:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.439030:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff9856084030c0 x1874669227121920/t214748367144(214748367144) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.439042:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.439055:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.439061:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.439069:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367144 00000100:00080000:0.0:1787826991.439112:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985608400780 x1874669227122304/t214748367145(214748367145) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.439119:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856084030c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227121920:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.439132:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608400780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227122304:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.440919:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985608400780 x1874669227122304/t214748367145(214748367145) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.440930:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.440932:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.440936:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.440944:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367145 00000100:00080000:0.0:1787826991.440997:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985608400000 x1874669227122688/t214748367146(214748367146) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.441009:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608400780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227122304:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.441029:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608400000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227122688:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.443156:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985608400000 x1874669227122688/t214748367146(214748367146) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.443170:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.443173:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.443179:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.443186:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367146 00000100:00080000:0.0:1787826991.443252:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985608403840 x1874669227123072/t214748367147(214748367147) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.443263:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608400000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227122688:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.443283:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608403840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227123072:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.448967:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985608403840 x1874669227123072/t214748367147(214748367147) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.448982:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.448985:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.448991:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.448999:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367147 00000100:00080000:0.0:1787826991.449064:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985608403c00 x1874669227123456/t214748367148(214748367148) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.449077:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608403840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227123072:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.449097:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608403c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227123456:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.451648:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985608403c00 x1874669227123456/t214748367148(214748367148) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.451665:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.451668:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.451675:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.451683:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367148 00000100:00080000:0.0:1787826991.451785:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985608400b40 x1874669227123840/t214748367149(214748367149) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.451799:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608403c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227123456:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.451822:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608400b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227123840:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.453822:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985608400b40 x1874669227123840/t214748367149(214748367149) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.453836:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.453839:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.453845:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.453854:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367149 00000100:00080000:0.0:1787826991.453922:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985608402940 x1874669227124224/t214748367150(214748367150) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.453932:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608400b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227123840:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.453949:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608402940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227124224:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.455901:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985608402940 x1874669227124224/t214748367150(214748367150) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.455914:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.455917:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.455923:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.455932:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367150 00000100:00080000:0.0:1787826991.455999:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9856084021c0 x1874669227124608/t214748367151(214748367151) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.456009:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608402940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227124224:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.456025:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856084021c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227124608:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.457643:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff9856084021c0 x1874669227124608/t214748367151(214748367151) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.457651:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.457653:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.457657:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.457671:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367151 00000100:00080000:0.0:1787826991.457750:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985608402580 x1874669227124992/t214748367152(214748367152) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.457761:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856084021c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227124608:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.457779:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608402580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227124992:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.459379:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985608402580 x1874669227124992/t214748367152(214748367152) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.459393:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.459396:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.459420:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.459435:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367152 00000100:00080000:0.0:1787826991.459504:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985608400f00 x1874669227125376/t214748367153(214748367153) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.459515:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608402580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227124992:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.459530:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608400f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227125376:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.461082:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985608400f00 x1874669227125376/t214748367153(214748367153) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.461095:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.461097:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.461103:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.461111:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367153 00000100:00080000:0.0:1787826991.461165:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985605785e00 x1874669227125760/t214748367154(214748367154) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.461175:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608400f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227125376:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.461191:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985605785e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227125760:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.462958:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985605785e00 x1874669227125760/t214748367154(214748367154) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.462973:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.462975:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.462982:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.462989:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367154 00000100:00080000:0.0:1787826991.463035:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9856057852c0 x1874669227126144/t214748367155(214748367155) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.463058:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985605785e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227125760:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.463078:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856057852c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227126144:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.465001:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff9856057852c0 x1874669227126144/t214748367155(214748367155) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.465012:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.465014:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.465018:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.465024:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367155 00000100:00080000:0.0:1787826991.465099:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985605786d00 x1874669227126528/t214748367156(214748367156) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.465110:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856057852c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227126144:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.465129:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985605786d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227126528:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.466940:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985605786d00 x1874669227126528/t214748367156(214748367156) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.466949:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.466951:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.466956:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.466964:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367156 00000100:00080000:0.0:1787826991.467010:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985605785680 x1874669227126912/t214748367157(214748367157) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.467022:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985605786d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227126528:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.467038:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985605785680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227126912:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.468910:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985605785680 x1874669227126912/t214748367157(214748367157) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.468925:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.468928:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.468934:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.468942:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367157 00000100:00080000:0.0:1787826991.469002:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9856057843c0 x1874669227127296/t214748367158(214748367158) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.469014:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985605785680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227126912:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.469032:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856057843c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227127296:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.470908:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff9856057843c0 x1874669227127296/t214748367158(214748367158) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.470921:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.470924:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.470930:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.470940:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367158 00000100:00080000:0.0:1787826991.470993:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985605787480 x1874669227127680/t214748367159(214748367159) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.471003:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856057843c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227127296:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.471017:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985605787480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227127680:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.472725:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985605787480 x1874669227127680/t214748367159(214748367159) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.472734:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.472735:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.472740:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.472745:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367159 00000100:00080000:0.0:1787826991.472796:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985605785a40 x1874669227128064/t214748367160(214748367160) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.472807:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985605787480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227127680:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.472824:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985605785a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227128064:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.474356:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985605785a40 x1874669227128064/t214748367160(214748367160) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.474364:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.474365:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.474369:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.474375:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367160 00000100:00080000:0.0:1787826991.474423:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9856057870c0 x1874669227128448/t214748367161(214748367161) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.474432:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985605785a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227128064:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.474444:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856057870c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227128448:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.476206:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff9856057870c0 x1874669227128448/t214748367161(214748367161) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.476219:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.476221:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.476227:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.476234:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367161 00000100:00080000:0.0:1787826991.476296:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985605784780 x1874669227128832/t214748367162(214748367162) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.476308:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856057870c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227128448:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.476324:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985605784780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227128832:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.478490:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985605784780 x1874669227128832/t214748367162(214748367162) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.478505:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.478507:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.478514:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.478522:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367162 00000100:00080000:0.0:1787826991.478580:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985605784000 x1874669227129216/t214748367163(214748367163) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.478589:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985605784780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227128832:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.478605:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985605784000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227129216:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.480589:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985605784000 x1874669227129216/t214748367163(214748367163) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.480605:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.480608:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.480614:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.480620:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367163 00000100:00080000:0.0:1787826991.480683:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985605787840 x1874669227129600/t214748367164(214748367164) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.480729:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985605784000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227129216:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.480749:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985605787840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227129600:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.482773:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985605787840 x1874669227129600/t214748367164(214748367164) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.482783:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.482785:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.482790:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.482795:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367164 00000100:00080000:0.0:1787826991.482851:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985605787c00 x1874669227129984/t214748367165(214748367165) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.482859:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985605787840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227129600:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.482884:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985605787c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227129984:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.484839:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985605787c00 x1874669227129984/t214748367165(214748367165) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.484849:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.484851:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.484856:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.484861:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367165 00000100:00080000:0.0:1787826991.484919:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985605784b40 x1874669227130368/t214748367166(214748367166) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.484927:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985605787c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227129984:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.484942:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985605784b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227130368:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.486900:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985605784b40 x1874669227130368/t214748367166(214748367166) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.486915:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.486918:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.486925:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.486933:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367166 00000100:00080000:0.0:1787826991.486999:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985605786940 x1874669227130752/t214748367167(214748367167) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.487010:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985605784b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227130368:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.487028:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985605786940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227130752:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.488933:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985605786940 x1874669227130752/t214748367167(214748367167) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.488947:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.488950:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.488957:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.488965:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367167 00000100:00080000:0.0:1787826991.489027:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985605784f00 x1874669227131136/t214748367168(214748367168) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.489037:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985605786940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227130752:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.489066:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985605784f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227131136:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.491135:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985605784f00 x1874669227131136/t214748367168(214748367168) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.491148:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.491151:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.491158:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.491174:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367168 00000100:00080000:0.0:1787826991.491242:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98560575fc00 x1874669227131520/t214748367169(214748367169) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.491263:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985605784f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227131136:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.491280:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98560575fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227131520:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.493227:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98560575fc00 x1874669227131520/t214748367169(214748367169) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.493241:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.493244:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.493250:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.493258:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367169 00000100:00080000:0.0:1787826991.493321:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98560575cb40 x1874669227131904/t214748367170(214748367170) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.493334:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98560575fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227131520:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.493350:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98560575cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227131904:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.495232:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98560575cb40 x1874669227131904/t214748367170(214748367170) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.495247:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.495249:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.495255:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.495263:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367170 00000100:00080000:0.0:1787826991.495316:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98560575e940 x1874669227132288/t214748367171(214748367171) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.495327:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98560575cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227131904:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.495343:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98560575e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227132288:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.498116:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98560575e940 x1874669227132288/t214748367171(214748367171) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.498130:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.498132:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.498139:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.498154:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367171 00000100:00080000:0.0:1787826991.498206:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98560575e1c0 x1874669227132672/t214748367172(214748367172) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.498218:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98560575e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227132288:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.498232:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98560575e1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227132672:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.500411:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98560575e1c0 x1874669227132672/t214748367172(214748367172) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.500422:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.500425:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.500431:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.500439:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367172 00000100:00080000:0.0:1787826991.500484:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98560575e580 x1874669227133056/t214748367173(214748367173) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.500494:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98560575e1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227132672:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.500509:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98560575e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227133056:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.502315:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98560575e580 x1874669227133056/t214748367173(214748367173) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.502327:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.502330:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.502335:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.502343:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367173 00000100:00080000:0.0:1787826991.502381:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98560575cf00 x1874669227133440/t214748367174(214748367174) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.502458:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98560575e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227133056:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.502475:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98560575cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227133440:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.504565:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98560575cf00 x1874669227133440/t214748367174(214748367174) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.504577:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.504580:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.504586:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.504593:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367174 00000100:00080000:0.0:1787826991.504639:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98560575de00 x1874669227133824/t214748367175(214748367175) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.504651:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98560575cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227133440:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.504666:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98560575de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227133824:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.506862:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98560575de00 x1874669227133824/t214748367175(214748367175) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.506875:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.506878:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.506883:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.506891:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367175 00000100:00080000:0.0:1787826991.506943:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98560575d2c0 x1874669227134208/t214748367176(214748367176) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.506954:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98560575de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227133824:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.506970:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98560575d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227134208:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.509287:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98560575d2c0 x1874669227134208/t214748367176(214748367176) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.509301:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.509303:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.509310:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.509317:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367176 00000100:00080000:0.0:1787826991.509377:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98560575ed00 x1874669227134592/t214748367177(214748367177) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.509389:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98560575d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227134208:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.509405:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98560575ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227134592:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.511788:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98560575ed00 x1874669227134592/t214748367177(214748367177) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.511800:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.511803:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.511807:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.511813:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367177 00000100:00080000:0.0:1787826991.511881:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98560575d680 x1874669227134976/t214748367178(214748367178) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.511891:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98560575ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227134592:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.511915:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98560575d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227134976:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.513766:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98560575d680 x1874669227134976/t214748367178(214748367178) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.513776:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.513778:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.513782:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.513790:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367178 00000100:00080000:0.0:1787826991.513845:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98560575c3c0 x1874669227135360/t214748367179(214748367179) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.513857:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98560575d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227134976:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.513875:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98560575c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227135360:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.515921:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98560575c3c0 x1874669227135360/t214748367179(214748367179) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.515937:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.515940:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.515947:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.515957:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367179 00000100:00080000:0.0:1787826991.516059:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98560575f480 x1874669227135744/t214748367180(214748367180) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.516076:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98560575c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227135360:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.516099:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98560575f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227135744:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.518173:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98560575f480 x1874669227135744/t214748367180(214748367180) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.518188:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.518191:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.518198:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.518207:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367180 00000100:00080000:0.0:1787826991.518272:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98560575da40 x1874669227136128/t214748367181(214748367181) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.518284:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98560575f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227135744:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.518302:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98560575da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227136128:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.520406:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98560575da40 x1874669227136128/t214748367181(214748367181) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.520417:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.520419:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.520424:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.520429:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367181 00000100:00080000:0.0:1787826991.520488:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98560575f0c0 x1874669227136512/t214748367182(214748367182) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.520499:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98560575da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227136128:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.520519:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98560575f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227136512:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.522412:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98560575f0c0 x1874669227136512/t214748367182(214748367182) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.522429:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.522432:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.522439:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.522447:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367182 00000100:00080000:0.0:1787826991.522513:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98560575c780 x1874669227136896/t214748367183(214748367183) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.522526:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98560575f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227136512:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.522546:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98560575c780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227136896:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.525226:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98560575c780 x1874669227136896/t214748367183(214748367183) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.525238:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.525241:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.525246:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.525252:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367183 00000100:00080000:0.0:1787826991.525339:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98560575c000 x1874669227137280/t214748367184(214748367184) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.525352:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98560575c780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227136896:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.525389:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98560575c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227137280:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.528945:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98560575c000 x1874669227137280/t214748367184(214748367184) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.528963:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.528966:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.528973:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.528982:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367184 00000100:00080000:0.0:1787826991.529083:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff98560575f840 x1874669227137664/t214748367185(214748367185) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.529097:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98560575c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227137280:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.529123:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff98560575f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227137664:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.532789:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff98560575f840 x1874669227137664/t214748367185(214748367185) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.532808:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.532810:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.532818:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.532827:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367185 00000100:00080000:0.0:1787826991.532915:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985607faed00 x1874669227138048/t214748367186(214748367186) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.532930:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff98560575f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227137664:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.532954:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607faed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227138048:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.535396:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985607faed00 x1874669227138048/t214748367186(214748367186) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.535413:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.535416:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.535424:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.535431:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367186 00000100:00080000:0.0:1787826991.535523:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985607fad680 x1874669227138432/t214748367187(214748367187) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.535537:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607faed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227138048:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.535559:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607fad680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227138432:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.537650:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985607fad680 x1874669227138432/t214748367187(214748367187) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.537661:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.537663:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.537667:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.537674:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367187 00000100:00080000:0.0:1787826991.537754:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985607fac3c0 x1874669227138816/t214748367188(214748367188) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.537769:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607fad680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227138432:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.537790:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607fac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227138816:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.539833:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985607fac3c0 x1874669227138816/t214748367188(214748367188) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.539853:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.539855:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.539860:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.539867:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367188 00000100:00080000:0.0:1787826991.539911:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985607faf480 x1874669227139200/t214748367189(214748367189) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.539921:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607fac3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227138816:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.539935:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607faf480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227139200:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.542304:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985607faf480 x1874669227139200/t214748367189(214748367189) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.542316:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.542319:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.542325:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.542333:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367189 00000100:00080000:0.0:1787826991.542391:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985607fada40 x1874669227139584/t214748367190(214748367190) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.542402:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607faf480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227139200:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.542421:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607fada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227139584:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.544347:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985607fada40 x1874669227139584/t214748367190(214748367190) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.544360:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.544363:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.544368:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.544375:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367190 00000100:00080000:0.0:1787826991.544420:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985607faf0c0 x1874669227139968/t214748367191(214748367191) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.544432:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607fada40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227139584:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.544461:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607faf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227139968:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.546966:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985607faf0c0 x1874669227139968/t214748367191(214748367191) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.546979:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.546982:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.546989:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.546997:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367191 00000100:00080000:0.0:1787826991.547069:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985607fac780 x1874669227140352/t214748367192(214748367192) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.547081:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607faf0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227139968:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.547099:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607fac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227140352:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.549196:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985607fac780 x1874669227140352/t214748367192(214748367192) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.549210:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.549212:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.549218:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.549227:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367192 00000100:00080000:0.0:1787826991.549273:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985607fac000 x1874669227140736/t214748367193(214748367193) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.549284:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607fac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227140352:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.549299:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607fac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227140736:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.551269:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985607fac000 x1874669227140736/t214748367193(214748367193) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.551280:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.551282:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.551286:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.551291:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367193 00000100:00080000:0.0:1787826991.551341:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985607faf840 x1874669227141120/t214748367194(214748367194) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.551351:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607fac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227140736:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.551367:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607faf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227141120:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.553607:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985607faf840 x1874669227141120/t214748367194(214748367194) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.553621:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.553623:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.553629:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.553636:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367194 00000100:00080000:0.0:1787826991.553712:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985607fafc00 x1874669227141504/t214748367195(214748367195) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.553721:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607faf840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227141120:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.553734:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607fafc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227141504:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.555622:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985607fafc00 x1874669227141504/t214748367195(214748367195) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.555636:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.555639:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.555643:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.555648:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367195 00000100:00080000:0.0:1787826991.555728:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985607facb40 x1874669227141888/t214748367196(214748367196) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.555737:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607fafc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227141504:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.555750:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607facb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227141888:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.557986:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985607facb40 x1874669227141888/t214748367196(214748367196) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.557996:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.557998:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.558002:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.558007:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367196 00000100:00080000:0.0:1787826991.558066:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985607fae940 x1874669227142272/t214748367197(214748367197) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.558076:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607facb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227141888:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.558091:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607fae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227142272:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.560017:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985607fae940 x1874669227142272/t214748367197(214748367197) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.560030:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.560032:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.560039:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.560061:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367197 00000100:00080000:0.0:1787826991.560107:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985607fae1c0 x1874669227142656/t214748367198(214748367198) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.560118:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607fae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227142272:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.560132:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607fae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227142656:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.562215:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985607fae1c0 x1874669227142656/t214748367198(214748367198) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.562223:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.562225:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.562229:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.562234:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367198 00000100:00080000:0.0:1787826991.562274:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985607fae580 x1874669227143040/t214748367199(214748367199) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.562283:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607fae1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227142656:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.562298:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607fae580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227143040:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.564872:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985607fae580 x1874669227143040/t214748367199(214748367199) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.564885:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.564888:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.564894:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.564913:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367199 00000100:00080000:0.0:1787826991.564961:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985607facf00 x1874669227143424/t214748367200(214748367200) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.564972:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607fae580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227143040:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.564989:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607facf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227143424:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.566850:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985607facf00 x1874669227143424/t214748367200(214748367200) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.566862:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.566864:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.566870:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.566877:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367200 00000100:00080000:0.0:1787826991.566921:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985607fade00 x1874669227143808/t214748367201(214748367201) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.566928:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607facf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227143424:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.566941:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607fade00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227143808:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.569301:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985607fade00 x1874669227143808/t214748367201(214748367201) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.569315:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.569317:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.569322:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.569327:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367201 00000100:00080000:0.0:1787826991.569383:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985607fad2c0 x1874669227144192/t214748367202(214748367202) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787826979 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826991.569396:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607fade00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227143808:192.168.203.114@tcp:36:unlinkmany.0 00000100:00100000:0.0:1787826991.569412:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607fad2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227144192:192.168.203.114@tcp:36:unlinkmany.0 00000100:00080000:0.0:1787826991.575157:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985607fad2c0 x1874669227144192/t214748367202(214748367202) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827007 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.575172:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826991.575174:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748366801 gen 9 00000100:00100000:0.0:1787826991.575179:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610ce8b40 x1874669226989952/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827006 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.575185:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 214748366801 last 214748367202 00000100:00080000:0.0:1787826991.575239:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from REPLAY to REPLAY_LOCKS 00000100:00100000:0.0:1787826991.576109:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607fad2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227144192:192.168.203.114@tcp:36:unlinkmany.0 00010000:00080000:2.0:1787826991.576210:0:86171:0:(ldlm_request.c:2972:ldlm_lock_replay_thread()) lock replay thread lustre-MDT0000-mdc-ffff985602bc9800 to lustre-MDT0000_UUID@192.168.203.114@tcp 00010000:00010000:2.0:1787826991.576215:0:86171:0:(ldlm_request.c:2888:ldlm_cancel_unused_locks_for_replay()) Dropping as many unused locks as possible before replay for namespace lustre-MDT0000-mdc-ffff985602bc9800 (400) 00000080:00200000:2.0:1787826991.576673:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x641:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.576750:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9a00/0xb3f83dad1b7e8240 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x641:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c257e expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.576774:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x640:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.576787:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9800/0xb3f83dad1b7e8232 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x640:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c254d expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.576802:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x63f:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.576814:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9c00/0xb3f83dad1b7e8224 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x63f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c251c expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.576829:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x63e:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.576840:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8a00/0xb3f83dad1b7e8216 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x63e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c24eb expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.576860:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x63d:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.576870:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8800/0xb3f83dad1b7e8208 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x63d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c24ba expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.576883:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x63c:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.576895:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9200/0xb3f83dad1b7e81fa lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x63c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c2489 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.576908:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x63b:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.576919:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8400/0xb3f83dad1b7e81ec lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x63b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c2458 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.576931:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x63a:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.576942:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5e00/0xb3f83dad1b7e81de lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x63a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c2427 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.576971:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x639:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.576983:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57200/0xb3f83dad1b7e81d0 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x639:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c23f6 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.576999:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x638:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.577023:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f31800/0xb3f83dad1b7e81c2 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x638:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c23c5 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.577039:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x637:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.577065:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f30400/0xb3f83dad1b7e81b4 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x637:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c2394 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.577085:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x636:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.577096:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608567600/0xb3f83dad1b7e81a6 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x636:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c2363 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.577109:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x635:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.577119:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f30c00/0xb3f83dad1b7e8198 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x635:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c2332 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.577133:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x634:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.577143:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f30000/0xb3f83dad1b7e818a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x634:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c2301 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.577158:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x633:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.577169:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f30800/0xb3f83dad1b7e817c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x633:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c22d0 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.577183:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x632:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.577194:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4800/0xb3f83dad1b7e816e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x632:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c229f expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.577208:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x631:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.577219:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4200/0xb3f83dad1b7e8160 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x631:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c226e expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.577234:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x630:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.577245:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5a00/0xb3f83dad1b7e8152 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x630:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c223d expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.577264:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x62f:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.577276:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5600/0xb3f83dad1b7e8144 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x62f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c220c expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.577289:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x62e:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.577299:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f31200/0xb3f83dad1b7e8136 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x62e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c21db expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.577311:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x62d:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.577320:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f31a00/0xb3f83dad1b7e8128 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x62d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c21aa expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.577332:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x62c:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.577340:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f31000/0xb3f83dad1b7e811a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x62c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c2179 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.577351:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x62b:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.577360:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f30a00/0xb3f83dad1b7e810c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x62b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c2148 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.577371:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x62a:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.577379:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f31c00/0xb3f83dad1b7e80fe lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x62a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c2117 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.577391:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x629:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.577400:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f31400/0xb3f83dad1b7e80f0 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x629:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c20e6 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.577429:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x628:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.577440:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f31e00/0xb3f83dad1b7e80e2 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x628:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c20b5 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.577452:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x627:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.577461:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f30600/0xb3f83dad1b7e80d4 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x627:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c2084 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.577479:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x626:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.577492:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f31600/0xb3f83dad1b7e80c6 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x626:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c2053 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.577505:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x625:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.577517:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f30e00/0xb3f83dad1b7e80b8 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x625:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c2022 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.577529:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x624:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.577541:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f30200/0xb3f83dad1b7e80aa lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x624:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c1ff1 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.577553:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x623:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.577565:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608567800/0xb3f83dad1b7e809c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x623:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c1fc0 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.577578:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x622:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.577588:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608566000/0xb3f83dad1b7e808e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x622:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c1f8f expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.577606:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x621:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.577617:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084c6800/0xb3f83dad1b7e8080 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x621:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c1f5e expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.577630:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x620:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.577641:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084c6e00/0xb3f83dad1b7e8072 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x620:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c1f2d expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.577653:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x61f:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.577664:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084c7400/0xb3f83dad1b7e8064 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x61f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c1efc expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.577678:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x61e:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.577689:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084c6600/0xb3f83dad1b7e8056 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x61e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c1ecb expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.577742:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x61d:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.577754:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084c6000/0xb3f83dad1b7e8048 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x61d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c1e9a expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.577768:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x61c:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.577783:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084c7200/0xb3f83dad1b7e803a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x61c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c1e69 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.577799:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x61b:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.577811:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084c7e00/0xb3f83dad1b7e802c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x61b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c1e38 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.577830:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x61a:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.577841:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084c6400/0xb3f83dad1b7e801e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x61a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c1e07 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.577855:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x619:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.577865:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084c7c00/0xb3f83dad1b7e8010 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x619:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c1dd6 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.577878:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x618:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.577888:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084c7a00/0xb3f83dad1b7e8002 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x618:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c1da5 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.577902:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x617:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.577912:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084c7800/0xb3f83dad1b7e7ff4 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x617:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c1d74 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.577925:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x616:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.577936:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084c6a00/0xb3f83dad1b7e7fe6 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x616:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c1d43 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.577950:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x615:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.577968:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084c6c00/0xb3f83dad1b7e7fd8 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x615:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c1d12 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.577982:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x614:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.577994:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084c7000/0xb3f83dad1b7e7fca lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x614:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c1ce1 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.578013:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x613:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.578024:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084c7600/0xb3f83dad1b7e7fbc lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x613:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c1cb0 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.578039:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x612:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.578062:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856057f8800/0xb3f83dad1b7e7fae lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x612:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c1c7f expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.578077:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x611:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.578087:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608566a00/0xb3f83dad1b7e7fa0 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x611:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c1c4e expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.578100:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x610:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.578110:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608566c00/0xb3f83dad1b7e7f92 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x610:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c1c1d expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.578123:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x60f:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.578133:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fa8c00/0xb3f83dad1b7e7f84 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x60f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c1bec expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.578147:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x60e:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.578157:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608567c00/0xb3f83dad1b7e7f76 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x60e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c1bbb expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.578170:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x60d:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.578181:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608567000/0xb3f83dad1b7e7f68 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x60d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c1b8a expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.578197:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x60c:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.578208:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608566600/0xb3f83dad1b7e7f5a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x60c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c1b59 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.578220:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x60b:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.578228:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608567e00/0xb3f83dad1b7e7f4c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x60b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c1b28 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.578239:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x60a:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.578248:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608567200/0xb3f83dad1b7e7f3e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x60a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c1af7 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.578260:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x609:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.578269:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608566e00/0xb3f83dad1b7e7f30 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x609:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c1ac6 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.578281:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x608:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.578292:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608567400/0xb3f83dad1b7e7f22 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x608:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c1a95 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.578305:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x607:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.578314:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608566200/0xb3f83dad1b7e7f14 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x607:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c1a64 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.578327:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x606:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.578338:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608566400/0xb3f83dad1b7e7f06 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x606:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c1a33 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.578355:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x605:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.578365:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608566800/0xb3f83dad1b7e7ef8 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x605:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c1a02 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.578377:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x604:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.578387:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608567a00/0xb3f83dad1b7e7eea lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x604:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c19d1 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.578400:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x603:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.578410:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fa8800/0xb3f83dad1b7e7edc lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x603:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c19a0 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.578421:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x602:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.578431:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856057f8a00/0xb3f83dad1b7e7ece lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x602:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c196f expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.578443:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x601:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.578451:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856057f8400/0xb3f83dad1b7e7ec0 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x601:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c193e expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.578463:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x600:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.578474:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fa8a00/0xb3f83dad1b7e7eb2 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x600:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c190d expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.578487:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5ff:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.578498:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fa9e00/0xb3f83dad1b7e7ea4 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5ff:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c18dc expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.578514:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5fe:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.578525:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fa9c00/0xb3f83dad1b7e7e96 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5fe:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c18ab expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.578537:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5fd:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.578548:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fa8600/0xb3f83dad1b7e7e88 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5fd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c187a expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.578561:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5fc:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.578572:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fa9000/0xb3f83dad1b7e7e7a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5fc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c1849 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.578584:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5fb:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.578595:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fa9a00/0xb3f83dad1b7e7e6c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5fb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c1818 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.578608:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5fa:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.578618:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fa8200/0xb3f83dad1b7e7e5e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5fa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c17e7 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.578631:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5f9:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.578642:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fa9600/0xb3f83dad1b7e7e50 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5f9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c17b6 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.578655:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5f8:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.578666:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fa8e00/0xb3f83dad1b7e7e42 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5f8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c1785 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.578684:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5f7:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.578719:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fa9400/0xb3f83dad1b7e7e34 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5f7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c1754 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.578733:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5f6:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.578744:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fa8000/0xb3f83dad1b7e7e26 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5f6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c1723 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.578757:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5f5:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.578767:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fa8400/0xb3f83dad1b7e7e18 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5f5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c16f2 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.578781:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5f4:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.578793:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fa9200/0xb3f83dad1b7e7e0a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5f4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c16c1 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.578806:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5f3:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.578817:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fa9800/0xb3f83dad1b7e7dfc lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5f3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c1690 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.578830:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5f2:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.578841:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607db8e00/0xb3f83dad1b7e7dee lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5f2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c165f expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.578853:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5f1:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.578864:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084bcc00/0xb3f83dad1b7e7de0 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5f1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c162e expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.578881:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5f0:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.578892:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856057f8e00/0xb3f83dad1b7e7dd2 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5f0:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c15fd expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.578906:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5ef:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.578918:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856057f9400/0xb3f83dad1b7e7dc4 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5ef:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c15cc expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.578930:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5ee:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.578941:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856057f8200/0xb3f83dad1b7e7db6 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5ee:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c159b expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.578955:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5ed:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.578964:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856057f8000/0xb3f83dad1b7e7da8 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5ed:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c156a expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.578977:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5ec:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.578988:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856057f9800/0xb3f83dad1b7e7d9a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5ec:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c1539 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.579001:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5eb:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.579012:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856057f9200/0xb3f83dad1b7e7d8c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5eb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c1508 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.579025:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5ea:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.579037:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856057f9a00/0xb3f83dad1b7e7d7e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5ea:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c14d7 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.579496:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5e9:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.579514:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856057f9000/0xb3f83dad1b7e7d70 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5e9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c14a6 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.579527:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5e8:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.579539:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856057f9600/0xb3f83dad1b7e7d62 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5e8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c1475 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.579552:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5e7:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.579564:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856057f9c00/0xb3f83dad1b7e7d54 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5e7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c1444 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.579577:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5e6:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.579589:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856057f8c00/0xb3f83dad1b7e7d46 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5e6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c1413 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.579601:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5e5:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.579612:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856057f8600/0xb3f83dad1b7e7d38 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5e5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c13e2 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.579625:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5e4:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.579636:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5800/0xb3f83dad1b7e7d2a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5e4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c13b1 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.579650:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5e3:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.579661:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06600/0xb3f83dad1b7e7d1c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5e3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c1380 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.579679:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5e2:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.579709:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06000/0xb3f83dad1b7e7d0e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5e2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c134f expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.579725:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5e1:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.579738:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856057f9e00/0xb3f83dad1b7e7d00 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5e1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c131e expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.579752:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5e0:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.579764:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607db8200/0xb3f83dad1b7e7cf2 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5e0:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c12ed expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.579776:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5df:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.579788:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607db8600/0xb3f83dad1b7e7ce4 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5df:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c12bc expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.579801:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5de:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.579811:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4400/0xb3f83dad1b7e7cd6 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5de:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c128b expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.579825:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5dd:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.579841:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4e00/0xb3f83dad1b7e7cc8 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5dd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c125a expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.579855:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5dc:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.579864:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4a00/0xb3f83dad1b7e7cba lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5dc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c1229 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.579881:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5db:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.579892:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5200/0xb3f83dad1b7e7cac lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5db:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c11f8 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.579904:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5da:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.579914:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5000/0xb3f83dad1b7e7c9e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5da:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c11c7 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.579927:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5d9:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.579938:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5400/0xb3f83dad1b7e7c90 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5d9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c1196 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.579949:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5d8:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.579959:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d4c00/0xb3f83dad1b7e7c82 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5d8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c1165 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.579972:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5d7:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.579982:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856056d5c00/0xb3f83dad1b7e7c74 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5d7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c1134 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.579996:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5d6:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.580239:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607c63800/0xb3f83dad1b7e7c66 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5d6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c1103 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.580258:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5d5:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.580271:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607db8800/0xb3f83dad1b7e7c58 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5d5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c10d2 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.580289:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5d4:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.580301:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607db9600/0xb3f83dad1b7e7c4a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5d4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c10a1 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.580314:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5d3:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.580325:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607db8400/0xb3f83dad1b7e7c3c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5d3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c1070 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.580339:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5d2:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.580349:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607db9200/0xb3f83dad1b7e7c2e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5d2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c103f expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.580363:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5d1:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.580373:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607db9e00/0xb3f83dad1b7e7c20 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5d1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c100e expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.580387:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5d0:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.580396:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607db9400/0xb3f83dad1b7e7c12 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5d0:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c0fdd expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.580410:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5cf:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.580420:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607db8000/0xb3f83dad1b7e7c04 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5cf:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c0fac expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.580433:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5ce:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.580442:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607db9a00/0xb3f83dad1b7e7bf6 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5ce:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c0f7b expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.580458:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5cd:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.580468:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607db8a00/0xb3f83dad1b7e7be8 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5cd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c0f4a expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.580483:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5cc:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.580494:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607db8c00/0xb3f83dad1b7e7bda lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5cc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c0f19 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.580507:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5cb:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.580518:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607db9800/0xb3f83dad1b7e7bcc lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5cb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c0ee8 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.580531:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5ca:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.580542:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607db9c00/0xb3f83dad1b7e7bbe lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5ca:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c0eb7 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.580555:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5c9:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.580566:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607db9000/0xb3f83dad1b7e7bb0 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5c9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c0e86 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.580579:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5c8:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.580590:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610416800/0xb3f83dad1b7e7ba2 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5c8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c0e55 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.580604:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5c7:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.580615:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607c63a00/0xb3f83dad1b7e7b94 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5c7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c0e24 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.580631:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5c6:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.580640:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084bce00/0xb3f83dad1b7e7b86 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5c6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c0df3 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.580653:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5c5:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.580663:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985612920c00/0xb3f83dad1b7e7b78 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5c5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c0dc2 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.580676:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5c4:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.580685:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084bd600/0xb3f83dad1b7e7b6a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5c4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c0d91 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.580720:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5c3:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.580730:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084bd800/0xb3f83dad1b7e7b5c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5c3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c0d60 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.580743:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5c2:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.580754:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084bca00/0xb3f83dad1b7e7b4e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5c2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c0d2f expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.580768:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5c1:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.580778:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084bde00/0xb3f83dad1b7e7b40 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5c1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c0cfe expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.580791:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5c0:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.580801:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084bdc00/0xb3f83dad1b7e7b32 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5c0:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c0ccd expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.580819:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5bf:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.580828:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084bd000/0xb3f83dad1b7e7b24 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5bf:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c0c9c expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.580840:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5be:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.580849:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084bc600/0xb3f83dad1b7e7b16 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5be:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c0c6b expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.580861:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5bd:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.580869:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084bc400/0xb3f83dad1b7e7b08 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5bd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c0c3a expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.580881:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5bc:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.580890:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084bc200/0xb3f83dad1b7e7afa lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5bc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c0c09 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.580901:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5bb:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.580910:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084bd200/0xb3f83dad1b7e7aec lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5bb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c0bd8 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.580921:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5ba:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.580930:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084bd400/0xb3f83dad1b7e7ade lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5ba:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c0ba7 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.580942:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5b9:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.580951:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084bc000/0xb3f83dad1b7e7ad0 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5b9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c0b76 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.580966:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5b8:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.580976:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084bc800/0xb3f83dad1b7e7ac2 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5b8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c0b45 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.580987:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5b7:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.581003:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084bda00/0xb3f83dad1b7e7ab4 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5b7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c0b14 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.581016:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5b6:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.581025:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607c62400/0xb3f83dad1b7e7aa6 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5b6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c0ae3 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.581037:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5b5:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.581057:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607c63200/0xb3f83dad1b7e7a98 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5b5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c0ab2 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.581069:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5b4:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.581077:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607c63c00/0xb3f83dad1b7e7a8a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5b4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c0a81 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.581087:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5b3:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.581097:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607c63600/0xb3f83dad1b7e7a7c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5b3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c0a50 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.581109:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5b2:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.581116:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607c63000/0xb3f83dad1b7e7a6e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5b2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c0a1f expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.581130:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5b1:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.581137:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607c62a00/0xb3f83dad1b7e7a60 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5b1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c09ee expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.581148:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5b0:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.581156:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607c62e00/0xb3f83dad1b7e7a52 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5b0:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c09bd expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.581167:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5af:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.581174:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607c62c00/0xb3f83dad1b7e7a44 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5af:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c098c expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.581185:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5ae:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.581193:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607c62200/0xb3f83dad1b7e7a36 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5ae:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c095b expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.581203:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5ad:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.581211:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607c62600/0xb3f83dad1b7e7a28 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5ad:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c092a expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.581222:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5ac:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.581229:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607c63e00/0xb3f83dad1b7e7a1a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5ac:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c08f9 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.581240:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5ab:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.581250:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607c63400/0xb3f83dad1b7e7a0c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5ab:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c08c8 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.581266:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5aa:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.581277:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607c62000/0xb3f83dad1b7e79fe lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5aa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c0897 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.581290:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5a9:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.581300:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607c62800/0xb3f83dad1b7e79f0 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5a9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c0866 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.581312:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5a8:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.581321:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856057f0600/0xb3f83dad1b7e79e2 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5a8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c0835 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.581332:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5a7:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.581341:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610417800/0xb3f83dad1b7e79d4 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5a7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c0804 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.581353:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5a6:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.581363:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610416a00/0xb3f83dad1b7e79c6 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5a6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c07d3 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.581376:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5a5:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.581385:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985612921400/0xb3f83dad1b7e79b8 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5a5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c07a2 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.581397:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5a4:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.581409:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985612921e00/0xb3f83dad1b7e79aa lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5a4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c0771 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.581427:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5a3:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.581438:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985612921000/0xb3f83dad1b7e799c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5a3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c0740 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.581451:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5a2:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.581461:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985612920600/0xb3f83dad1b7e798e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5a2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c070f expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.581475:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5a1:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.581486:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985612920a00/0xb3f83dad1b7e7980 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5a1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c06de expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.581500:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5a0:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.581511:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985612921600/0xb3f83dad1b7e7972 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5a0:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c06ad expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.581523:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x59f:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.581534:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985612920e00/0xb3f83dad1b7e7964 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x59f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c067c expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.581547:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x59e:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.581557:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985612921a00/0xb3f83dad1b7e7956 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x59e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c064b expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.581570:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x59d:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.581578:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985612920200/0xb3f83dad1b7e7948 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x59d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c061a expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.581589:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x59c:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.581595:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985612920000/0xb3f83dad1b7e793a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x59c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c05e9 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.581603:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x59b:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.581609:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985612920400/0xb3f83dad1b7e792c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x59b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c05b8 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.581616:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x59a:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.581623:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985612920800/0xb3f83dad1b7e791e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x59a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c0587 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.581631:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x599:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.581637:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985612921200/0xb3f83dad1b7e7910 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x599:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c0556 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.581645:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x598:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.581651:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985612921800/0xb3f83dad1b7e7902 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x598:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c0525 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.581659:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x597:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.581665:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985612921c00/0xb3f83dad1b7e78f4 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x597:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c04f4 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.581672:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x596:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.581678:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608524000/0xb3f83dad1b7e78e6 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x596:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c04c3 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.581689:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x595:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.581751:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610417a00/0xb3f83dad1b7e78d8 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x595:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c0492 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.581762:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x594:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.581770:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610416c00/0xb3f83dad1b7e78ca lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x594:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c0461 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.581780:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x593:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.581787:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610416000/0xb3f83dad1b7e78bc lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x593:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c0430 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.581796:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x592:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.581807:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610417c00/0xb3f83dad1b7e78ae lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x592:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c03ff expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.581817:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x591:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.581824:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610417000/0xb3f83dad1b7e78a0 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x591:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c03ce expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.581833:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x590:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.581840:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610417e00/0xb3f83dad1b7e7892 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x590:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c039d expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.581850:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x58f:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.581860:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610416600/0xb3f83dad1b7e7884 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x58f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c036c expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.581871:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x58e:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.581877:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610417600/0xb3f83dad1b7e7876 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x58e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c033b expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.581885:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x58d:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.581891:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610416e00/0xb3f83dad1b7e7868 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x58d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c030a expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.581899:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x58c:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.581906:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610417400/0xb3f83dad1b7e785a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x58c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c02d9 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.581913:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x58b:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.581919:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610416200/0xb3f83dad1b7e784c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x58b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c02a8 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.581926:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x58a:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.581933:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610416400/0xb3f83dad1b7e783e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x58a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c0277 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.581940:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x589:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.581945:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610417200/0xb3f83dad1b7e7830 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x589:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c0246 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.581953:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x588:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.581959:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607cae000/0xb3f83dad1b7e7822 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x588:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c0215 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.581970:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x587:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.581976:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608525000/0xb3f83dad1b7e7814 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x587:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c01e4 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.581984:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x586:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.581990:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856085b0400/0xb3f83dad1b7e7806 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x586:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c01b3 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.581998:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x585:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.582003:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608524e00/0xb3f83dad1b7e77f8 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x585:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c0182 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.582011:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x584:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.582018:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608525400/0xb3f83dad1b7e77ea lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x584:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c0151 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.582025:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x583:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.582031:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608524800/0xb3f83dad1b7e77dc lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x583:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c0120 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.582038:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x582:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.582056:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608524200/0xb3f83dad1b7e77ce lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x582:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c00ef expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.582069:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x581:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.582077:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608524400/0xb3f83dad1b7e77c0 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x581:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c00be expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.582088:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x580:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.582095:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608524a00/0xb3f83dad1b7e77b2 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x580:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c008d expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.582103:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x57f:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.582108:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608525200/0xb3f83dad1b7e77a4 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x57f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c005c expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.582116:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x57e:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.582122:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608524c00/0xb3f83dad1b7e7796 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x57e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91c002b expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.582133:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x57d:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.582143:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608525a00/0xb3f83dad1b7e7788 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x57d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bfffa expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.582156:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x57c:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.582166:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608525800/0xb3f83dad1b7e777a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x57c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bffc9 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.582178:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x57b:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.582188:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608525c00/0xb3f83dad1b7e776c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x57b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bff98 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.582199:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x57a:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.582208:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608525600/0xb3f83dad1b7e775e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x57a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bff67 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.582224:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x579:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.582234:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608524600/0xb3f83dad1b7e7750 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x579:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bff36 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.582246:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x578:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.582255:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608525e00/0xb3f83dad1b7e7742 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x578:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bff05 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.582272:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x577:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.582284:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856057f1c00/0xb3f83dad1b7e7734 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x577:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bfed4 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.582298:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x576:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.582310:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856085b0e00/0xb3f83dad1b7e7726 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x576:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bfea3 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.582324:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x575:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.582334:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856085b1600/0xb3f83dad1b7e7718 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x575:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bfe72 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.582348:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x574:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.582359:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856085b1000/0xb3f83dad1b7e770a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x574:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bfe41 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.582373:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x573:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.582383:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856085b0600/0xb3f83dad1b7e76fc lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x573:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bfe10 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.582396:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x572:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.582406:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856085b1c00/0xb3f83dad1b7e76ee lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x572:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bfddf expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.582420:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x571:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.582433:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856085b1e00/0xb3f83dad1b7e76e0 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x571:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bfdae expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.582447:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x570:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.582459:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856085b1400/0xb3f83dad1b7e76d2 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x570:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bfd7d expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.582472:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x56f:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.582484:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856085b1800/0xb3f83dad1b7e76c4 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x56f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bfd4c expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.582497:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x56e:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.582509:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856085b0200/0xb3f83dad1b7e76b6 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x56e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bfd1b expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.582523:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x56d:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.582533:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856085b0000/0xb3f83dad1b7e76a8 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x56d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bfcea expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.582546:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x56c:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.582557:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856085b0800/0xb3f83dad1b7e769a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x56c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bfcb9 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.582575:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x56b:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.582587:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856085b1200/0xb3f83dad1b7e768c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x56b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bfc88 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.582600:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x56a:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.582612:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856085b1a00/0xb3f83dad1b7e767e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x56a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bfc57 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.582625:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x569:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.582637:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856085b0a00/0xb3f83dad1b7e7670 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x569:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bfc26 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.582652:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x568:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.582662:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856085b0c00/0xb3f83dad1b7e7662 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x568:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bfbf5 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.582671:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x567:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.582678:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856057f0000/0xb3f83dad1b7e7654 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x567:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bfbc4 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.582686:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x566:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.582711:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856057f1e00/0xb3f83dad1b7e7646 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x566:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bfb93 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.582720:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x565:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.582727:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607caea00/0xb3f83dad1b7e7638 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x565:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bfb62 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.582737:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x564:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.582743:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607caf200/0xb3f83dad1b7e762a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x564:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bfb31 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.582751:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x563:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.582757:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856030b0000/0xb3f83dad1b7e761c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x563:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bfb00 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.582765:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x562:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.582771:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856057f0a00/0xb3f83dad1b7e760e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x562:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bfacf expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.582779:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x561:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.582785:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856057f0c00/0xb3f83dad1b7e7600 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x561:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bfa9e expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.582792:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x560:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.582798:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856057f1000/0xb3f83dad1b7e75f2 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x560:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bfa6d expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.582806:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x55f:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.582812:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856057f1400/0xb3f83dad1b7e75e4 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x55f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bfa3c expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.582819:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x55e:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.582827:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856057f0800/0xb3f83dad1b7e75d6 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x55e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bfa0b expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.582866:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x55d:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.582881:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856057f1600/0xb3f83dad1b7e75c8 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x55d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bf9da expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.582891:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x55c:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.582898:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856057f0e00/0xb3f83dad1b7e75ba lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x55c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bf9a9 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.582906:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x55b:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.582912:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856057f1a00/0xb3f83dad1b7e75ac lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x55b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bf978 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.582920:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x55a:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.582927:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856057f0200/0xb3f83dad1b7e759e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x55a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bf947 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.582934:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x559:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.582940:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856057f0400/0xb3f83dad1b7e7590 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x559:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bf916 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.582947:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x558:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.582954:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856057f1200/0xb3f83dad1b7e7582 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x558:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bf8e5 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.582962:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x557:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.582968:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856057f1800/0xb3f83dad1b7e7574 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x557:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bf8b4 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.582978:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x556:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.582984:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856030b1000/0xb3f83dad1b7e7566 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x556:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bf883 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.582992:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x555:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.582998:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856030b0e00/0xb3f83dad1b7e7558 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x555:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bf852 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.583006:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x554:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.583013:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607cae400/0xb3f83dad1b7e754a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x554:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bf821 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.583020:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x553:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.583026:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607cae200/0xb3f83dad1b7e753c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x553:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bf7f0 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.583034:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x552:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.583040:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607cae800/0xb3f83dad1b7e752e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x552:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bf7bf expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.583063:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x551:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.583072:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607cafc00/0xb3f83dad1b7e7520 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x551:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bf78e expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.583080:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x550:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.583086:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607cafa00/0xb3f83dad1b7e7512 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x550:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bf75d expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.583097:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x54f:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.583103:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607caf800/0xb3f83dad1b7e7504 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x54f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bf72c expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.583110:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x54e:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.583117:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607caf000/0xb3f83dad1b7e74f6 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x54e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bf6fb expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.583124:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x54d:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.583130:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607caec00/0xb3f83dad1b7e74e8 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x54d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bf6ca expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.583137:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x54c:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.583143:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607caf600/0xb3f83dad1b7e74da lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x54c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bf699 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.583151:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x54b:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.583156:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607caee00/0xb3f83dad1b7e74cc lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x54b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bf668 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.583164:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x54a:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.583171:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607cae600/0xb3f83dad1b7e74be lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x54a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bf637 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.583178:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x549:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.583184:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607cafe00/0xb3f83dad1b7e74b0 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x549:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bf606 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.583194:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x548:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.583200:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607caf400/0xb3f83dad1b7e74a2 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x548:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bf5d5 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.583208:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x547:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.583218:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d57000/0xb3f83dad1b7e7494 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x547:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bf5a4 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.583233:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x546:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.583245:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607d74e00/0xb3f83dad1b7e7486 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x546:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bf573 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.583258:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x545:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.583268:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856030b1600/0xb3f83dad1b7e7478 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x545:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bf542 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.583281:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x544:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.583305:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856030b0200/0xb3f83dad1b7e746a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x544:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bf511 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.583322:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x543:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.583334:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856030b0600/0xb3f83dad1b7e745c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x543:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bf4e0 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.583346:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x542:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.583356:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856030b1e00/0xb3f83dad1b7e744e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x542:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bf4af expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.583375:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x541:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.583385:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856030b0800/0xb3f83dad1b7e7440 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x541:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bf47e expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.583399:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x540:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.583410:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856030b1400/0xb3f83dad1b7e7432 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x540:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bf44d expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.583424:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x53f:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.583434:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856030b1200/0xb3f83dad1b7e7424 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x53f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bf41c expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.583447:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x53e:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.583457:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856030b0400/0xb3f83dad1b7e7416 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x53e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bf3eb expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.583470:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x53d:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.583480:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856030b1c00/0xb3f83dad1b7e7408 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x53d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bf3ba expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.583493:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x53c:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.583503:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856030b1800/0xb3f83dad1b7e73fa lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x53c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bf389 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.583515:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x53b:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.583524:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856030b1a00/0xb3f83dad1b7e73ec lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x53b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bf358 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.583541:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x53a:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.583551:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856030b0a00/0xb3f83dad1b7e73de lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x53a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bf327 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.583565:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x539:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.583574:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856030b0c00/0xb3f83dad1b7e73d0 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x539:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bf2f6 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.583587:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x538:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.583598:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d57e00/0xb3f83dad1b7e73c2 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x538:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bf2c5 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.583609:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x537:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.583620:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607d75600/0xb3f83dad1b7e73b4 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x537:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bf294 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.583634:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x536:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.583645:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084bb600/0xb3f83dad1b7e73a6 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x536:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bf263 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.583658:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x535:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.583669:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d56a00/0xb3f83dad1b7e7398 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x535:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bf232 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.583682:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x534:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.583713:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d57a00/0xb3f83dad1b7e738a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x534:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bf201 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.583731:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x533:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.583741:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d57800/0xb3f83dad1b7e737c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x533:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bf1d0 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.583752:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x532:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.583760:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d57c00/0xb3f83dad1b7e736e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x532:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bf19f expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.583771:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x531:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.583779:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d57400/0xb3f83dad1b7e7360 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x531:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bf16e expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.583790:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x530:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.583798:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d56800/0xb3f83dad1b7e7352 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x530:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bf13d expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.583810:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x52f:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.583819:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d56c00/0xb3f83dad1b7e7344 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x52f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bf10c expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.583831:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x52e:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.583839:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d56000/0xb3f83dad1b7e7336 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x52e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bf0db expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.583852:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x52d:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.583861:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d56600/0xb3f83dad1b7e7328 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x52d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bf0aa expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.583877:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x52c:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.583887:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d57600/0xb3f83dad1b7e731a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x52c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bf079 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.583899:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x52b:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.583908:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d56e00/0xb3f83dad1b7e730c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x52b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bf048 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.583933:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x52a:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.583944:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d56200/0xb3f83dad1b7e72fe lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x52a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bf017 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.583956:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x529:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.583967:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d56400/0xb3f83dad1b7e72f0 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x529:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91befe6 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.583980:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x528:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.583991:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d57200/0xb3f83dad1b7e72e2 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x528:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91befb5 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.584005:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x527:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.584027:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084ba800/0xb3f83dad1b7e72d4 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x527:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bef84 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.584057:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x526:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.584070:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607d74200/0xb3f83dad1b7e72c6 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x526:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bef53 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.584089:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x525:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.584101:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607d74000/0xb3f83dad1b7e72b8 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x525:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bef22 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.584113:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x524:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.584124:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607d74400/0xb3f83dad1b7e72aa lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x524:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91beef1 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.584138:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x523:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.584150:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607d74800/0xb3f83dad1b7e729c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x523:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91beec0 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.584164:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x522:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.584175:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607d75200/0xb3f83dad1b7e728e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x522:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bee8f expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.584189:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x521:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.584200:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607d74a00/0xb3f83dad1b7e7280 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x521:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bee5e expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.584214:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x520:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.584226:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607d75a00/0xb3f83dad1b7e7272 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x520:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bee2d expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.584239:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x51f:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.584249:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607d75800/0xb3f83dad1b7e7264 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x51f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bedfc expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.584267:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x51e:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.584278:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607d75c00/0xb3f83dad1b7e7256 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x51e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bedcb expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.584294:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x51d:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.584305:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607d74c00/0xb3f83dad1b7e7248 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x51d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bed9a expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.584317:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x51c:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.584329:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607d75000/0xb3f83dad1b7e723a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x51c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bed69 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.584342:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x51b:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.584353:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607d74600/0xb3f83dad1b7e722c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x51b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bed38 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.584366:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x51a:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.584377:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607d75e00/0xb3f83dad1b7e721e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x51a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bed07 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.584390:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x519:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.584401:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607d75400/0xb3f83dad1b7e7210 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x519:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91becd6 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.584415:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x518:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.584427:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560861d600/0xb3f83dad1b7e7202 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x518:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91beca5 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.584445:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x517:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.584456:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084bb800/0xb3f83dad1b7e71f4 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x517:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bec74 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.584469:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x516:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.584481:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560861cc00/0xb3f83dad1b7e71e6 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x516:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bec43 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.584494:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x515:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.584505:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560861ce00/0xb3f83dad1b7e71d8 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x515:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bec12 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.584517:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x514:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.584528:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605439800/0xb3f83dad1b7e71ca lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x514:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bebe1 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.584541:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x513:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.584552:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084ba400/0xb3f83dad1b7e71bc lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x513:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bebb0 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.584566:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x512:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.584577:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084bb200/0xb3f83dad1b7e71ae lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x512:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91beb7f expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.584591:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x511:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.584615:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084ba600/0xb3f83dad1b7e71a0 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x511:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91beb4e expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.584636:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x510:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.584647:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084bba00/0xb3f83dad1b7e7192 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x510:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91beb1d expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.584661:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x50f:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.584673:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084bae00/0xb3f83dad1b7e7184 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x50f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91beaec expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.584685:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x50e:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.584717:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084baa00/0xb3f83dad1b7e7176 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x50e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91beabb expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.584733:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x50d:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.584744:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084bbc00/0xb3f83dad1b7e7168 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x50d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bea8a expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.584759:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x50c:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.584770:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084bac00/0xb3f83dad1b7e715a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x50c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bea59 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.584785:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x50b:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.584796:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084bb000/0xb3f83dad1b7e714c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x50b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bea28 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.584810:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x50a:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.584820:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084ba200/0xb3f83dad1b7e713e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x50a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91be9f7 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.584838:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x509:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.584848:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084bbe00/0xb3f83dad1b7e7130 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x509:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91be9c6 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.584861:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x508:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.584872:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084bb400/0xb3f83dad1b7e7122 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x508:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91be995 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.584885:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x507:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.584896:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856084ba000/0xb3f83dad1b7e7114 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x507:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91be964 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.584910:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x506:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.584920:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608613200/0xb3f83dad1b7e7106 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x506:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91be933 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.584934:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x505:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.584945:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560861c800/0xb3f83dad1b7e70f8 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x505:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91be902 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.584958:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x504:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.584969:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560861c200/0xb3f83dad1b7e70ea lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x504:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91be8d1 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.584981:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x503:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.584992:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560861c000/0xb3f83dad1b7e70dc lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x503:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91be8a0 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.585009:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x502:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.585020:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560861c400/0xb3f83dad1b7e70ce lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x502:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91be86f expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.585034:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x501:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.585056:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560861d200/0xb3f83dad1b7e70c0 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x501:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91be83e expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.585072:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x500:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.585083:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560861da00/0xb3f83dad1b7e70b2 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x500:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91be80d expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.585096:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4ff:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.585106:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560861d800/0xb3f83dad1b7e70a4 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4ff:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91be7dc expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.585118:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4fe:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.585128:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560861ca00/0xb3f83dad1b7e7096 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4fe:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91be7ab expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.585141:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4fd:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.585151:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560861dc00/0xb3f83dad1b7e7088 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4fd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91be77a expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.585164:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4fc:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.585174:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560861d000/0xb3f83dad1b7e707a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4fc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91be749 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.585191:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4fb:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.585202:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560861c600/0xb3f83dad1b7e706c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4fb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91be718 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.585216:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4fa:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.585226:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560861de00/0xb3f83dad1b7e705e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4fa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91be6e7 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.585239:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4f9:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.585249:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98560861d400/0xb3f83dad1b7e7050 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4f9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91be6b6 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.585262:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4f8:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.585273:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608496800/0xb3f83dad1b7e7042 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4f8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91be685 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.585299:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4f7:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.585314:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608613c00/0xb3f83dad1b7e7034 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4f7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91be654 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.585327:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4f6:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.585338:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856057c1e00/0xb3f83dad1b7e7026 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4f6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91be623 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.585351:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4f5:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.585362:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608613800/0xb3f83dad1b7e7018 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4f5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91be5f2 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.585380:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4f4:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.585391:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608612e00/0xb3f83dad1b7e700a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4f4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91be5c1 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.585405:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4f3:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.585416:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608613600/0xb3f83dad1b7e6ffc lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4f3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91be590 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.585430:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4f2:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.585440:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608613a00/0xb3f83dad1b7e6fee lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4f2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91be55f expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.585454:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4f1:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.585465:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608612a00/0xb3f83dad1b7e6fe0 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4f1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91be52e expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.585478:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4f0:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.585491:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608612c00/0xb3f83dad1b7e6fd2 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4f0:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91be4fd expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.585505:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4ef:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.585517:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608612400/0xb3f83dad1b7e6fc4 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4ef:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91be4cc expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.585530:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4ee:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.585540:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608612200/0xb3f83dad1b7e6fb6 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4ee:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91be49b expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.585557:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4ed:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.585568:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608613000/0xb3f83dad1b7e6fa8 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4ed:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91be46a expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.585580:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4ec:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.585590:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608612600/0xb3f83dad1b7e6f9a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4ec:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91be439 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.585604:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4eb:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.585615:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608613e00/0xb3f83dad1b7e6f8c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4eb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91be408 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.585629:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4ea:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.585639:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608613400/0xb3f83dad1b7e6f7e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4ea:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91be3d7 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.585652:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4e9:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.585662:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608612000/0xb3f83dad1b7e6f70 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4e9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91be3a6 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.585675:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4e8:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.585686:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608612800/0xb3f83dad1b7e6f62 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4e8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91be375 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.585719:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4e7:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.585733:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605439600/0xb3f83dad1b7e6f54 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4e7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91be344 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.585751:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4e6:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.585763:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608496e00/0xb3f83dad1b7e6f46 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4e6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91be313 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.585783:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4e5:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.585795:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608496c00/0xb3f83dad1b7e6f38 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4e5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91be2e2 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.585807:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4e4:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.585817:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605439e00/0xb3f83dad1b7e6f2a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4e4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91be2b1 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.585829:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4e3:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.585837:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605439c00/0xb3f83dad1b7e6f1c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4e3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91be280 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.585849:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4e2:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.585858:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605438c00/0xb3f83dad1b7e6f0e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4e2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91be24f expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.585870:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4e1:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.585879:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605439000/0xb3f83dad1b7e6f00 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4e1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91be21e expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.585891:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4e0:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.585901:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605438600/0xb3f83dad1b7e6ef2 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4e0:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91be1ed expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.585917:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4df:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.585928:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605438e00/0xb3f83dad1b7e6ee4 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4df:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91be1bc expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.585940:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4de:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.585949:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605439400/0xb3f83dad1b7e6ed6 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4de:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91be18b expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.585973:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4dd:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.585985:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605438400/0xb3f83dad1b7e6ec8 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4dd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91be15a expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.586001:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4dc:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.586012:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605439200/0xb3f83dad1b7e6eba lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4dc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91be129 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.586025:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4db:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.586035:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605438200/0xb3f83dad1b7e6eac lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4db:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91be0f8 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.586062:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4da:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.586074:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605438000/0xb3f83dad1b7e6e9e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4da:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91be0c7 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.586087:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4d9:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.586098:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605438800/0xb3f83dad1b7e6e90 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4d9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91be096 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.586116:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4d8:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.586126:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605439a00/0xb3f83dad1b7e6e82 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4d8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91be065 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.586140:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4d7:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.586151:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985605438a00/0xb3f83dad1b7e6e74 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4d7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91be034 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.586164:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4d6:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.586174:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856057c1c00/0xb3f83dad1b7e6e66 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4d6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91be003 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.586186:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4d5:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.586197:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607df8400/0xb3f83dad1b7e6e58 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4d5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bdfd2 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.586209:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4d4:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.586220:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608497e00/0xb3f83dad1b7e6e4a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4d4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bdfa1 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.586235:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4d3:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.586246:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608497400/0xb3f83dad1b7e6e3c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4d3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bdf70 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.586258:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4d2:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.586268:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608496a00/0xb3f83dad1b7e6e2e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4d2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bdf3f expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.586284:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4d1:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.586294:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608496200/0xb3f83dad1b7e6e20 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4d1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bdf0e expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.586308:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4d0:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.586319:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608496000/0xb3f83dad1b7e6e12 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4d0:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bdedd expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.586333:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4cf:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.586344:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608496400/0xb3f83dad1b7e6e04 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4cf:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bdeac expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.586357:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4ce:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.586368:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608497200/0xb3f83dad1b7e6df6 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4ce:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bde7b expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.586381:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4cd:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.586393:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608497a00/0xb3f83dad1b7e6de8 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4cd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bde4a expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.586406:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4cc:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.586417:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608497800/0xb3f83dad1b7e6dda lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4cc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bde19 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.586433:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4cb:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.586443:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608496600/0xb3f83dad1b7e6dcc lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4cb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bdde8 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.586459:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4ca:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.586470:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608497c00/0xb3f83dad1b7e6dbe lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4ca:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bddb7 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.586482:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4c9:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.586492:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608497000/0xb3f83dad1b7e6db0 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4c9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bdd86 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.586504:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4c8:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.586515:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985608497600/0xb3f83dad1b7e6da2 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4c8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bdd55 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.586528:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4c7:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.586539:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98561288ca00/0xb3f83dad1b7e6d94 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4c7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bdd24 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.586552:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4c6:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.586563:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856057c0000/0xb3f83dad1b7e6d86 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4c6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bdcf3 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.586575:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4c5:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.586586:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856057c1800/0xb3f83dad1b7e6d78 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4c5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bdcc2 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.586600:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4c4:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.586626:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856057c0a00/0xb3f83dad1b7e6d6a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4c4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bdc91 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.586647:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4c3:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.586659:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856057c0c00/0xb3f83dad1b7e6d5c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4c3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bdc60 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.586671:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4c2:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.586682:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856057c1000/0xb3f83dad1b7e6d4e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4c2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bdc2f expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.586715:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4c1:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.586727:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856057c0600/0xb3f83dad1b7e6d40 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4c1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bdbfe expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.586740:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4c0:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.586750:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856057c1600/0xb3f83dad1b7e6d32 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4c0:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bdbcd expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.586764:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4bf:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.586775:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856057c1a00/0xb3f83dad1b7e6d24 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4bf:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bdb9c expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.586788:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4be:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.586799:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856057c0e00/0xb3f83dad1b7e6d16 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4be:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bdb6b expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.586811:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4bd:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.586821:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856057c1400/0xb3f83dad1b7e6d08 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4bd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bdb3a expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.586839:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4bc:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.586850:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856057c0800/0xb3f83dad1b7e6cfa lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4bc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bdb09 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.586863:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4bb:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.586874:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856057c0200/0xb3f83dad1b7e6cec lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4bb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bdad8 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.586888:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4ba:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.586898:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856057c0400/0xb3f83dad1b7e6cde lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4ba:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bdaa7 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.586911:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4b9:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.586921:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856057c1200/0xb3f83dad1b7e6cd0 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4b9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bda76 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.586935:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4b8:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.586944:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98561288d000/0xb3f83dad1b7e6cc2 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4b8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bda45 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.586958:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4b7:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.586968:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98561288c200/0xb3f83dad1b7e6cb4 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4b7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bda14 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.586981:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4b6:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.586991:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607df8a00/0xb3f83dad1b7e6ca6 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4b6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bd9e3 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.587007:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4b5:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.587018:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff9856108f4a00/0xb3f83dad1b7e6c98 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4b5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bd9b2 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.587032:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4b4:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.587042:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98561288cc00/0xb3f83dad1b7e6c8a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4b4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bd981 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.587069:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4b3:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.587082:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98561288c600/0xb3f83dad1b7e6c7c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4b3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bd950 expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826991.587095:0:86171:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4b2:0x0]: losing layout lock 00010000:00010000:2.0:1787826991.587105:0:86171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff98561288d600/0xb3f83dad1b7e6c6e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4b2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91bd91f expref: -99 pid: 85489 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826991.587116:0:86171:0:(ldlm_request.c:2901:ldlm_cancel_unused_locks_for_replay()) Canceled 0 unused locks from namespace lustre-MDT0000-mdc-ffff985602bc9800 00000100:00080000:2.0:1787826991.587530:0:86171:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from REPLAY_LOCKS to REPLAY_WAIT 00000100:00100000:0.0:1787826991.587622:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227150336:192.168.203.114@tcp:400:ldlm_lock_repla.0 00000100:00100000:0.0:1787826991.592756:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 214748367202 gen 9 00000100:00080000:0.0:1787826991.594936:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from REPLAY_WAIT to RECOVER 00000100:00080000:0.0:1787826991.594952:0:2382:0:(client.c:3247:ptlrpc_resend_req()) @@@ going to resend req@ffff9856140d8780 x1874669227150336/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827007 ref 1 fl Interpret:RQU/2c0/0 rc 0/0 job:'ldlm_lock_repla.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.594965:0:2382:0:(client.c:3256:ptlrpc_resend_req()) @@@ it has reply, so skip it req@ffff9856140d8780 x1874669227150336/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827007 ref 1 fl Interpret:RQU/2c0/0 rc 0/0 job:'ldlm_lock_repla.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826991.594972:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from RECOVER to FULL 00000100:02000000:1.0:1787826991.595215:0:2382:0:(import.c:1743:ptlrpc_import_recovery_state_machine()) lustre-MDT0000-mdc-ffff985602bc9800: Connection restored to 192.168.203.114@tcp (at 192.168.203.114@tcp) 00000100:00080000:1.0:1787826991.595225:0:2382:0:(recover.c:191:ptlrpc_wake_delayed()) @@@ waking (set 00000000dc9d15c9): req@ffff985608550780 x1874669227149184/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 576/1152 e 0 to 0 dl 0 ref 2 fl Rpc:WQU/600/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:0 00000100:00100000:1.0:1787826991.595255:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669227150336:192.168.203.114@tcp:400:ldlm_lock_repla.0 00000100:00100000:3.0:1787826991.595450:0:86156:0:(client.c:2715:ptlrpc_set_wait()) set ffff985612993b40 going to sleep for 11 seconds 00000100:00100000:3.0:1787826991.597842:0:86156:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608550780 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:86156:1874669227149184:192.168.203.114@tcp:101:lfs.0 00010000:00010000:3.0:1787826991.597864:0:86156:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe6a00/0xb3f83dad1b7e8d68 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91c9117 expref: -99 pid: 86156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826991.597876:0:86156:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe6a00/0xb3f83dad1b7e8d68 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91c9117 expref: -99 pid: 86156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826991.597887:0:86156:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe6a00/0xb3f83dad1b7e8d68 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91c9117 expref: -99 pid: 86156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826991.597898:0:86156:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe6a00/0xb3f83dad1b7e8d68 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91c9117 expref: -99 pid: 86156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826991.597911:0:86156:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe6a00/0xb3f83dad1b7e8d68 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91c9117 expref: -99 pid: 86156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826991.597920:0:86156:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe6a00/0xb3f83dad1b7e8d68 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91c9117 expref: -99 pid: 86156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826991.597928:0:86156:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe6a00/0xb3f83dad1b7e8d68 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91c9117 expref: -99 pid: 86156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826991.597941:0:86156:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff985608550780 x1874669227149184/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 576/688 e 0 to 0 dl 1787827007 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826991.597955:0:86156:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe6a00/0xb3f83dad1b7e8d68 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91c9117 expref: -99 pid: 86156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826991.597967:0:86156:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:3.0:1787826991.597974:0:86156:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:3.0:1787826991.598000:0:86156:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 69632 00000080:00010000:3.0:1787826991.598008:0:86156:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e8d68 00000080:00010000:3.0:1787826991.598013:0:86156:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:3.0:1787826991.598016:0:86156:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e8d68 from it ffffab3ac0d17cb8 00010000:00010000:3.0:1787826991.598020:0:86156:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe6a00/0xb3f83dad1b7e8d68 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91c9117 expref: -99 pid: 86156 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826991.598034:0:86156:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe6a00/0xb3f83dad1b7e8d68 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91c9117 expref: -99 pid: 86156 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:3.0:1787826991.598749:0:86156:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 0, flags 0, foreign 0 00000080:00200000:3.0:1787826991.598755:0:86156:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:3.0:1787826991.598771:0:86156:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826991.598781:0:86156:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe6a00/0xb3f83dad1b7e8d68 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91c9117 expref: -99 pid: 86156 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826991.598791:0:86156:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe6a00/0xb3f83dad1b7e8d68 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91c9117 expref: -99 pid: 86156 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826991.598799:0:86156:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826991.598804:0:86156:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:3.0:1787826991.598807:0:86156:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e8d68 00000080:00010000:3.0:1787826991.598811:0:86156:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e8d68 from it ffffab3ac0d17cb8 00010000:00010000:3.0:1787826991.598814:0:86156:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe6a00/0xb3f83dad1b7e8d68 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91c9117 expref: -99 pid: 86156 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826991.598821:0:86156:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe6a00/0xb3f83dad1b7e8d68 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91c9117 expref: -99 pid: 86156 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:3.0:1787826991.598911:0:86156:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:3.0:1787826991.598919:0:86156:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826991.598926:0:86156:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe6a00/0xb3f83dad1b7e8d68 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91c9117 expref: -99 pid: 86156 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826991.598940:0:86156:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe6a00/0xb3f83dad1b7e8d68 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91c9117 expref: -99 pid: 86156 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:3.0:1787826991.598946:0:86156:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826991.598950:0:86156:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:3.0:1787826991.598952:0:86156:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e8d68 00000080:00010000:3.0:1787826991.598956:0:86156:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e8d68 from it ffffab3ac0d17bb0 00010000:00010000:3.0:1787826991.598959:0:86156:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe6a00/0xb3f83dad1b7e8d68 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91c9117 expref: -99 pid: 86156 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826991.598967:0:86156:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe6a00/0xb3f83dad1b7e8d68 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91c9117 expref: -99 pid: 86156 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 1 00000080:00200000:3.0:1787826991.598976:0:86156:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 44 00000080:00200008:3.0:1787826991.598983:0:86156:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 100000 00000080:00200080:3.0:1787826991.599075:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00800000:00000080:3.0:1787826991.599089:0:86156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00800000:00000004:3.0:1787826991.599096:0:86156:0:(obd_class.h:1091:obd_statfs()) lustre-MDT0000-mdc-ffff985602bc9800: age 3109, max_age 3141 00000100:00100000:3.0:1787826991.599129:0:86156:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e83480 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:86156:1874669227150464:192.168.203.114@tcp:41:lfs.0 00000100:00100000:3.0:1787826991.599199:0:86156:0:(client.c:2715:ptlrpc_set_wait()) set ffff98560682ca80 going to sleep for 11 seconds 00000100:00100000:3.0:1787826991.600558:0:86156:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e83480 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:86156:1874669227150464:192.168.203.114@tcp:41:lfs.0 00800000:00000004:3.0:1787826991.600576:0:86156:0:(obd_class.h:1117:obd_statfs()) lustre-MDT0000-mdc-ffff985602bc9800: update ffff985606a337f0 cache blocks 317325/353529 objects 1022888/1024000 00000080:00200080:3.0:1787826991.600627:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00800000:00000080:3.0:1787826991.600638:0:86156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00800000:00000004:3.0:1787826991.600644:0:86156:0:(obd_class.h:1091:obd_statfs()) lustre-MDT0001-mdc-ffff985602bc9800: age 3109, max_age 3141 00000100:00100000:3.0:1787826991.600677:0:86156:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e83480 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:86156:1874669227150592:192.168.203.114@tcp:41:lfs.0 00000100:00100000:3.0:1787826991.600754:0:86156:0:(client.c:2715:ptlrpc_set_wait()) set ffff98560682ca80 going to sleep for 11 seconds 00000100:00100000:3.0:1787826991.601861:0:86156:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e83480 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:86156:1874669227150592:192.168.203.114@tcp:41:lfs.0 00800000:00000004:3.0:1787826991.601875:0:86156:0:(obd_class.h:1117:obd_statfs()) lustre-MDT0001-mdc-ffff985602bc9800: update ffff98560952b7f0 cache blocks 317401/353529 objects 1023737/1024000 00000080:00200080:3.0:1787826991.601914:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00800000:00000080:3.0:1787826991.601924:0:86156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.601933:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00800000:00000080:3.0:1787826991.601937:0:86156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.601943:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00800000:00000080:3.0:1787826991.601947:0:86156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.601952:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00800000:00000080:3.0:1787826991.601957:0:86156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.601961:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00800000:00000080:3.0:1787826991.601978:0:86156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.601985:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00800000:00000080:3.0:1787826991.601989:0:86156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.601993:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00800000:00000080:3.0:1787826991.601997:0:86156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.602001:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00800000:00000080:3.0:1787826991.602005:0:86156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.602010:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00800000:00000080:3.0:1787826991.602014:0:86156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.602019:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00800000:00000080:3.0:1787826991.602024:0:86156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.602036:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00800000:00000080:3.0:1787826991.602042:0:86156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.602057:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00800000:00000080:3.0:1787826991.602062:0:86156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.602067:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00800000:00000080:3.0:1787826991.602071:0:86156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.602075:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00800000:00000080:3.0:1787826991.602081:0:86156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.602085:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00800000:00000080:3.0:1787826991.602089:0:86156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.602094:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00800000:00000080:3.0:1787826991.602099:0:86156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.602104:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00800000:00000080:3.0:1787826991.602108:0:86156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.602113:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00800000:00000080:3.0:1787826991.602117:0:86156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.602122:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00800000:00000080:3.0:1787826991.602127:0:86156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.602132:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00800000:00000080:3.0:1787826991.602136:0:86156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.602140:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00800000:00000080:3.0:1787826991.602145:0:86156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.602150:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00800000:00000080:3.0:1787826991.602154:0:86156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.602159:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00800000:00000080:3.0:1787826991.602163:0:86156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.602168:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00800000:00000080:3.0:1787826991.602172:0:86156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.602176:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00800000:00000080:3.0:1787826991.602179:0:86156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.602193:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00800000:00000080:3.0:1787826991.602198:0:86156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.602203:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00800000:00000080:3.0:1787826991.602207:0:86156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.602211:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00800000:00000080:3.0:1787826991.602214:0:86156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.602218:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00800000:00000080:3.0:1787826991.602222:0:86156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.602226:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00800000:00000080:3.0:1787826991.602230:0:86156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.602233:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00800000:00000080:3.0:1787826991.602237:0:86156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.602241:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00800000:00000080:3.0:1787826991.602244:0:86156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.602248:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00800000:00000080:3.0:1787826991.602252:0:86156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.602256:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00800000:00000080:3.0:1787826991.602260:0:86156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.602263:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00800000:00000080:3.0:1787826991.602267:0:86156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.602272:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00800000:00000080:3.0:1787826991.602275:0:86156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.602280:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00800000:00000080:3.0:1787826991.602283:0:86156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.602287:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00800000:00000080:3.0:1787826991.602290:0:86156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.602294:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00800000:00000080:3.0:1787826991.602297:0:86156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.602301:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00800000:00000080:3.0:1787826991.602305:0:86156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.602309:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00800000:00000080:3.0:1787826991.602313:0:86156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.602317:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00800000:00000080:3.0:1787826991.602320:0:86156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.602324:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00800000:00000080:3.0:1787826991.602328:0:86156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.602332:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00800000:00000080:3.0:1787826991.602336:0:86156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.602340:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00800000:00000080:3.0:1787826991.602343:0:86156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.602347:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00800000:00000080:3.0:1787826991.602351:0:86156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.602355:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00800000:00000080:3.0:1787826991.602359:0:86156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.602363:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00800000:00000080:3.0:1787826991.602367:0:86156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.602371:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00800000:00000080:3.0:1787826991.602374:0:86156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.602378:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00800000:00000080:3.0:1787826991.602382:0:86156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.602386:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00800000:00000080:3.0:1787826991.602390:0:86156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.602394:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00800000:00000080:3.0:1787826991.602398:0:86156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.602401:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00800000:00000080:3.0:1787826991.602405:0:86156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.602409:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00800000:00000080:3.0:1787826991.602412:0:86156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.602416:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00800000:00000080:3.0:1787826991.602420:0:86156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.602424:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00800000:00000080:3.0:1787826991.602427:0:86156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.602431:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00800000:00000080:3.0:1787826991.602435:0:86156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.602439:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00800000:00000080:3.0:1787826991.602442:0:86156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.602451:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00800000:00000080:3.0:1787826991.602455:0:86156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.602459:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00800000:00000080:3.0:1787826991.602463:0:86156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.602467:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00800000:00000080:3.0:1787826991.602471:0:86156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.602475:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00800000:00000080:3.0:1787826991.602479:0:86156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.602483:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00800000:00000080:3.0:1787826991.602487:0:86156:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.602491:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00020000:00000080:3.0:1787826991.602498:0:86156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00020000:00000004:3.0:1787826991.602506:0:86156:0:(obd_class.h:1091:obd_statfs()) lustre-OST0000-osc-ffff985602bc9800: age 3109, max_age 3141 00000100:00100000:3.0:1787826991.602529:0:86156:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e83480 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:86156:1874669227150720:192.168.203.114@tcp:13:lfs.0 00000100:00100000:3.0:1787826991.602584:0:86156:0:(client.c:2715:ptlrpc_set_wait()) set ffff98560682ca80 going to sleep for 11 seconds 00000100:00100000:3.0:1787826991.603672:0:86156:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e83480 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:86156:1874669227150720:192.168.203.114@tcp:13:lfs.0 00020000:00000004:3.0:1787826991.603688:0:86156:0:(obd_class.h:1117:obd_statfs()) lustre-OST0000-osc-ffff985602bc9800: update ffff985609528450 cache blocks 901357/958279 objects 261460/262144 00000080:00200080:3.0:1787826991.603756:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00020000:00000080:3.0:1787826991.603766:0:86156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00020000:00000004:3.0:1787826991.603772:0:86156:0:(obd_class.h:1091:obd_statfs()) lustre-OST0001-osc-ffff985602bc9800: age 3109, max_age 3141 00000100:00100000:3.0:1787826991.603791:0:86156:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e83480 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:86156:1874669227150848:192.168.203.114@tcp:13:lfs.0 00000100:00100000:3.0:1787826991.603845:0:86156:0:(client.c:2715:ptlrpc_set_wait()) set ffff98560682ca80 going to sleep for 11 seconds 00000100:00100000:3.0:1787826991.605080:0:86156:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e83480 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:86156:1874669227150848:192.168.203.114@tcp:13:lfs.0 00020000:00000004:3.0:1787826991.605096:0:86156:0:(obd_class.h:1117:obd_statfs()) lustre-OST0001-osc-ffff985602bc9800: update ffff98560952d1c0 cache blocks 901363/958279 objects 261466/262144 00000080:00200080:3.0:1787826991.605127:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00020000:00000080:3.0:1787826991.605135:0:86156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.605141:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00020000:00000080:3.0:1787826991.605145:0:86156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.605150:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00020000:00000080:3.0:1787826991.605154:0:86156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.605158:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00020000:00000080:3.0:1787826991.605163:0:86156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.605167:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00020000:00000080:3.0:1787826991.605171:0:86156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.605175:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00020000:00000080:3.0:1787826991.605178:0:86156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.605183:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00020000:00000080:3.0:1787826991.605186:0:86156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.605191:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00020000:00000080:3.0:1787826991.605195:0:86156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.605207:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00020000:00000080:3.0:1787826991.605212:0:86156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.605216:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00020000:00000080:3.0:1787826991.605219:0:86156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.605223:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00020000:00000080:3.0:1787826991.605229:0:86156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.605238:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00020000:00000080:3.0:1787826991.605242:0:86156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.605248:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00020000:00000080:3.0:1787826991.605253:0:86156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.605257:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00020000:00000080:3.0:1787826991.605261:0:86156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.605264:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00020000:00000080:3.0:1787826991.605268:0:86156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.605271:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00020000:00000080:3.0:1787826991.605275:0:86156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.605278:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00020000:00000080:3.0:1787826991.605281:0:86156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.605285:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00020000:00000080:3.0:1787826991.605288:0:86156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.605291:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00020000:00000080:3.0:1787826991.605294:0:86156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.605298:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00020000:00000080:3.0:1787826991.605301:0:86156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.605305:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00020000:00000080:3.0:1787826991.605309:0:86156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.605313:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00020000:00000080:3.0:1787826991.605317:0:86156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.605321:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00020000:00000080:3.0:1787826991.605324:0:86156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.605327:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00020000:00000080:3.0:1787826991.605332:0:86156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.605336:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00020000:00000080:3.0:1787826991.605341:0:86156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.605344:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00020000:00000080:3.0:1787826991.605347:0:86156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.605351:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00020000:00000080:3.0:1787826991.605355:0:86156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.605360:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00020000:00000080:3.0:1787826991.605364:0:86156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.605368:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00020000:00000080:3.0:1787826991.605371:0:86156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.605374:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00020000:00000080:3.0:1787826991.605377:0:86156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.605381:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00020000:00000080:3.0:1787826991.605385:0:86156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.605388:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00020000:00000080:3.0:1787826991.605392:0:86156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.605396:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00020000:00000080:3.0:1787826991.605399:0:86156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.605403:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00020000:00000080:3.0:1787826991.605407:0:86156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.605411:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00020000:00000080:3.0:1787826991.605414:0:86156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.605417:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00020000:00000080:3.0:1787826991.605421:0:86156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.605426:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00020000:00000080:3.0:1787826991.605430:0:86156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.605434:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00020000:00000080:3.0:1787826991.605437:0:86156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.605441:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00020000:00000080:3.0:1787826991.605444:0:86156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.605447:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00020000:00000080:3.0:1787826991.605451:0:86156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.605455:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00020000:00000080:3.0:1787826991.605458:0:86156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.605462:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00020000:00000080:3.0:1787826991.605465:0:86156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.605468:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00020000:00000080:3.0:1787826991.605471:0:86156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.605474:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00020000:00000080:3.0:1787826991.605477:0:86156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.605480:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00020000:00000080:3.0:1787826991.605484:0:86156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.605487:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00020000:00000080:3.0:1787826991.605490:0:86156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.605494:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00020000:00000080:3.0:1787826991.605497:0:86156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.605500:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00020000:00000080:3.0:1787826991.605503:0:86156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.605506:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00020000:00000080:3.0:1787826991.605509:0:86156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.605512:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00020000:00000080:3.0:1787826991.605516:0:86156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.605519:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00020000:00000080:3.0:1787826991.605522:0:86156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.605525:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00020000:00000080:3.0:1787826991.605528:0:86156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.605532:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00020000:00000080:3.0:1787826991.605535:0:86156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.605538:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00020000:00000080:3.0:1787826991.605541:0:86156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.605545:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00020000:00000080:3.0:1787826991.605547:0:86156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.605551:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00020000:00000080:3.0:1787826991.605554:0:86156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.605557:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00020000:00000080:3.0:1787826991.605560:0:86156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.605563:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00020000:00000080:3.0:1787826991.605566:0:86156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.605570:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00020000:00000080:3.0:1787826991.605573:0:86156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.605576:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00020000:00000080:3.0:1787826991.605579:0:86156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.605582:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00020000:00000080:3.0:1787826991.605585:0:86156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.605588:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00020000:00000080:3.0:1787826991.605591:0:86156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200080:3.0:1787826991.605595:0:86156:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffeb86521f0 00020000:00000080:3.0:1787826991.605598:0:86156:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985604fbe400 uarg=0000000000000000 00000080:00200008:3.0:1787826991.605610:0:86156:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 100000 00010000:00010000:3.0:1787826991.924966:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826992.948942:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:0.0:1787826993.675381:0:86438:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: oleg314-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 00010000:00010000:0.0:1787826993.972842:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:2.0:1787826994.252389:0:86587:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec 00010000:00010000:3.0:1787826994.996941:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200008:1.0:1787826995.594675:0:86625:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:1.0:1787826995.594687:0:86625:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826995.594729:0:86625:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826995.594747:0:86625:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe6a00/0xb3f83dad1b7e8d68 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91c9117 expref: -99 pid: 86156 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 1 00010000:00010000:1.0:1787826995.594763:0:86625:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe6a00/0xb3f83dad1b7e8d68 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91c9117 expref: -99 pid: 86156 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 1 00000080:00200000:1.0:1787826995.594771:0:86625:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826995.594779:0:86625:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826995.594783:0:86625:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e8d68 00000080:00010000:1.0:1787826995.594791:0:86625:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e8d68 from it ffffab3ac0c97cb8 00010000:00010000:1.0:1787826995.594795:0:86625:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe6a00/0xb3f83dad1b7e8d68 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91c9117 expref: -99 pid: 86156 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 1 00010000:00010000:1.0:1787826995.594805:0:86625:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe6a00/0xb3f83dad1b7e8d68 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91c9117 expref: -99 pid: 86156 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00000080:00200008:1.0:1787826995.595302:0:86625:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 0, flags 0, foreign 0 00000080:00200000:1.0:1787826995.595306:0:86625:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826995.595312:0:86625:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826995.595317:0:86625:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe6a00/0xb3f83dad1b7e8d68 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91c9117 expref: -99 pid: 86156 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00010000:00010000:1.0:1787826995.595327:0:86625:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe6a00/0xb3f83dad1b7e8d68 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91c9117 expref: -99 pid: 86156 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00000080:00200000:1.0:1787826995.595331:0:86625:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826995.595334:0:86625:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826995.595336:0:86625:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e8d68 00000080:00010000:1.0:1787826995.595338:0:86625:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e8d68 from it ffffab3ac0c97cb8 00010000:00010000:1.0:1787826995.595340:0:86625:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe6a00/0xb3f83dad1b7e8d68 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91c9117 expref: -99 pid: 86156 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00010000:00010000:1.0:1787826995.595345:0:86625:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe6a00/0xb3f83dad1b7e8d68 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91c9117 expref: -99 pid: 86156 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00000080:00200000:1.0:1787826995.595413:0:86625:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787826995.595422:0:86625:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826995.595429:0:86625:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe6a00/0xb3f83dad1b7e8d68 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91c9117 expref: -99 pid: 86156 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00010000:00010000:1.0:1787826995.595437:0:86625:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe6a00/0xb3f83dad1b7e8d68 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91c9117 expref: -99 pid: 86156 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00000080:00200000:1.0:1787826995.595443:0:86625:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826995.595447:0:86625:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787826995.595450:0:86625:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7e8d68 00000080:00010000:1.0:1787826995.595453:0:86625:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7e8d68 from it ffffab3ac0c97bb0 00010000:00010000:1.0:1787826995.595456:0:86625:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe6a00/0xb3f83dad1b7e8d68 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91c9117 expref: -99 pid: 86156 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00010000:00010000:1.0:1787826995.595464:0:86625:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe6a00/0xb3f83dad1b7e8d68 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91c9117 expref: -99 pid: 86156 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00000080:00200000:1.0:1787826995.595472:0:86625:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 44 00000080:00200008:1.0:1787826995.595478:0:86625:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 100000 00000080:00200080:1.0:1787826995.595544:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00800000:00000080:1.0:1787826995.595558:0:86625:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00800000:00000004:1.0:1787826995.595565:0:86625:0:(obd_class.h:1091:obd_statfs()) lustre-MDT0000-mdc-ffff985602bc9800: age 3142, max_age 3145 00000100:00100000:1.0:1787826995.595604:0:86625:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608553c00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:86625:1874669227150976:192.168.203.114@tcp:41:lfs.0 00000100:00100000:1.0:1787826995.595889:0:86625:0:(client.c:2715:ptlrpc_set_wait()) set ffff985612993b40 going to sleep for 11 seconds 00000100:00100000:1.0:1787826995.597101:0:86625:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608553c00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:86625:1874669227150976:192.168.203.114@tcp:41:lfs.0 00800000:00000004:1.0:1787826995.597126:0:86625:0:(obd_class.h:1117:obd_statfs()) lustre-MDT0000-mdc-ffff985602bc9800: update ffff985606a337f0 cache blocks 317325/353529 objects 1023289/1024000 00000080:00200080:1.0:1787826995.597172:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00800000:00000080:1.0:1787826995.597183:0:86625:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00800000:00000004:1.0:1787826995.597190:0:86625:0:(obd_class.h:1091:obd_statfs()) lustre-MDT0001-mdc-ffff985602bc9800: age 3142, max_age 3145 00000100:00100000:1.0:1787826995.597212:0:86625:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608553c00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:86625:1874669227151104:192.168.203.114@tcp:41:lfs.0 00000100:00100000:1.0:1787826995.597259:0:86625:0:(client.c:2715:ptlrpc_set_wait()) set ffff985612993b40 going to sleep for 11 seconds 00000100:00100000:1.0:1787826995.598999:0:86625:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608553c00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:86625:1874669227151104:192.168.203.114@tcp:41:lfs.0 00800000:00000004:1.0:1787826995.599034:0:86625:0:(obd_class.h:1117:obd_statfs()) lustre-MDT0001-mdc-ffff985602bc9800: update ffff98560952b7f0 cache blocks 317401/353529 objects 1023737/1024000 00000080:00200080:1.0:1787826995.599079:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00800000:00000080:1.0:1787826995.599091:0:86625:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.599098:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00800000:00000080:1.0:1787826995.599104:0:86625:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.599111:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00800000:00000080:1.0:1787826995.599116:0:86625:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.599121:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00800000:00000080:1.0:1787826995.599126:0:86625:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.599131:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00800000:00000080:1.0:1787826995.599135:0:86625:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.599140:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00800000:00000080:1.0:1787826995.599144:0:86625:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.599148:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00800000:00000080:1.0:1787826995.599153:0:86625:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.599159:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00800000:00000080:1.0:1787826995.599163:0:86625:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.599168:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00800000:00000080:1.0:1787826995.599173:0:86625:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.599178:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00800000:00000080:1.0:1787826995.599181:0:86625:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.599185:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00800000:00000080:1.0:1787826995.599189:0:86625:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.599195:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00800000:00000080:1.0:1787826995.599199:0:86625:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.599204:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00800000:00000080:1.0:1787826995.599209:0:86625:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.599214:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00800000:00000080:1.0:1787826995.599217:0:86625:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.599220:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00800000:00000080:1.0:1787826995.599223:0:86625:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.599225:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00800000:00000080:1.0:1787826995.599228:0:86625:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.599231:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00800000:00000080:1.0:1787826995.599233:0:86625:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.599237:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00800000:00000080:1.0:1787826995.599240:0:86625:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.599242:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00800000:00000080:1.0:1787826995.599244:0:86625:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.599246:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00800000:00000080:1.0:1787826995.599248:0:86625:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.599252:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00800000:00000080:1.0:1787826995.599254:0:86625:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.599256:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00800000:00000080:1.0:1787826995.599259:0:86625:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.599262:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00800000:00000080:1.0:1787826995.599265:0:86625:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.599268:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00800000:00000080:1.0:1787826995.599269:0:86625:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.599271:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00800000:00000080:1.0:1787826995.599273:0:86625:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.599275:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00800000:00000080:1.0:1787826995.599277:0:86625:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.599279:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00800000:00000080:1.0:1787826995.599280:0:86625:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.599283:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00800000:00000080:1.0:1787826995.599284:0:86625:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.599286:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00800000:00000080:1.0:1787826995.599288:0:86625:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.599290:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00800000:00000080:1.0:1787826995.599292:0:86625:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.599295:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00800000:00000080:1.0:1787826995.599297:0:86625:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.599302:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00800000:00000080:1.0:1787826995.599304:0:86625:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.599307:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00800000:00000080:1.0:1787826995.599309:0:86625:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.599311:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00800000:00000080:1.0:1787826995.599313:0:86625:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.599315:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00800000:00000080:1.0:1787826995.599317:0:86625:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.599319:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00800000:00000080:1.0:1787826995.599322:0:86625:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.599324:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00800000:00000080:1.0:1787826995.599327:0:86625:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.599329:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00800000:00000080:1.0:1787826995.599330:0:86625:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.599332:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00800000:00000080:1.0:1787826995.599334:0:86625:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.599336:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00800000:00000080:1.0:1787826995.599339:0:86625:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.599343:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00800000:00000080:1.0:1787826995.599345:0:86625:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.599347:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00800000:00000080:1.0:1787826995.599349:0:86625:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.599351:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00800000:00000080:1.0:1787826995.599352:0:86625:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.599354:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00800000:00000080:1.0:1787826995.599356:0:86625:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.599358:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00800000:00000080:1.0:1787826995.599359:0:86625:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.599361:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00800000:00000080:1.0:1787826995.599363:0:86625:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.599365:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00800000:00000080:1.0:1787826995.599367:0:86625:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.599378:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00800000:00000080:1.0:1787826995.599390:0:86625:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.599396:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00800000:00000080:1.0:1787826995.599404:0:86625:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.599409:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00800000:00000080:1.0:1787826995.599414:0:86625:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.599418:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00800000:00000080:1.0:1787826995.599422:0:86625:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.599425:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00800000:00000080:1.0:1787826995.599429:0:86625:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.599433:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00800000:00000080:1.0:1787826995.599437:0:86625:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.599440:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00800000:00000080:1.0:1787826995.599444:0:86625:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.599448:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00800000:00000080:1.0:1787826995.599452:0:86625:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.599455:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00800000:00000080:1.0:1787826995.599459:0:86625:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.599463:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00800000:00000080:1.0:1787826995.599466:0:86625:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.599470:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00800000:00000080:1.0:1787826995.599473:0:86625:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.599476:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00800000:00000080:1.0:1787826995.599480:0:86625:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.599483:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00800000:00000080:1.0:1787826995.599487:0:86625:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.599492:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00800000:00000080:1.0:1787826995.599496:0:86625:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.599500:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00800000:00000080:1.0:1787826995.599504:0:86625:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.599507:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00800000:00000080:1.0:1787826995.599509:0:86625:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.599513:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00020000:00000080:1.0:1787826995.599518:0:86625:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00020000:00000004:1.0:1787826995.599523:0:86625:0:(obd_class.h:1091:obd_statfs()) lustre-OST0000-osc-ffff985602bc9800: age 3142, max_age 3145 00000100:00100000:1.0:1787826995.599553:0:86625:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608553c00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:86625:1874669227151232:192.168.203.114@tcp:13:lfs.0 00000100:00100000:1.0:1787826995.599612:0:86625:0:(client.c:2715:ptlrpc_set_wait()) set ffff985612993b40 going to sleep for 11 seconds 00000100:00100000:1.0:1787826995.601452:0:86625:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608553c00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:86625:1874669227151232:192.168.203.114@tcp:13:lfs.0 00020000:00000004:1.0:1787826995.601485:0:86625:0:(obd_class.h:1117:obd_statfs()) lustre-OST0000-osc-ffff985602bc9800: update ffff985609528450 cache blocks 901357/958279 objects 261444/262144 00000080:00200080:1.0:1787826995.601535:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00020000:00000080:1.0:1787826995.601547:0:86625:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00020000:00000004:1.0:1787826995.601553:0:86625:0:(obd_class.h:1091:obd_statfs()) lustre-OST0001-osc-ffff985602bc9800: age 3142, max_age 3145 00000100:00100000:1.0:1787826995.601575:0:86625:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608553c00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:86625:1874669227151360:192.168.203.114@tcp:13:lfs.0 00000100:00100000:1.0:1787826995.601620:0:86625:0:(client.c:2715:ptlrpc_set_wait()) set ffff985612993b40 going to sleep for 11 seconds 00000100:00100000:1.0:1787826995.603077:0:86625:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608553c00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:86625:1874669227151360:192.168.203.114@tcp:13:lfs.0 00020000:00000004:1.0:1787826995.603095:0:86625:0:(obd_class.h:1117:obd_statfs()) lustre-OST0001-osc-ffff985602bc9800: update ffff98560952d1c0 cache blocks 901363/958279 objects 261450/262144 00000080:00200080:1.0:1787826995.603135:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00020000:00000080:1.0:1787826995.603146:0:86625:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.603152:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00020000:00000080:1.0:1787826995.603156:0:86625:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.603160:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00020000:00000080:1.0:1787826995.603164:0:86625:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.603167:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00020000:00000080:1.0:1787826995.603170:0:86625:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.603174:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00020000:00000080:1.0:1787826995.603177:0:86625:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.603180:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00020000:00000080:1.0:1787826995.603184:0:86625:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.603187:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00020000:00000080:1.0:1787826995.603191:0:86625:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.603206:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00020000:00000080:1.0:1787826995.603210:0:86625:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.603214:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00020000:00000080:1.0:1787826995.603218:0:86625:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.603222:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00020000:00000080:1.0:1787826995.603226:0:86625:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.603230:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00020000:00000080:1.0:1787826995.603235:0:86625:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.603240:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00020000:00000080:1.0:1787826995.603244:0:86625:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.603250:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00020000:00000080:1.0:1787826995.603256:0:86625:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.603261:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00020000:00000080:1.0:1787826995.603264:0:86625:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.603269:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00020000:00000080:1.0:1787826995.603273:0:86625:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.603278:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00020000:00000080:1.0:1787826995.603281:0:86625:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.603285:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00020000:00000080:1.0:1787826995.603289:0:86625:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.603293:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00020000:00000080:1.0:1787826995.603297:0:86625:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.603303:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00020000:00000080:1.0:1787826995.603308:0:86625:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.603313:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00020000:00000080:1.0:1787826995.603319:0:86625:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.603323:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00020000:00000080:1.0:1787826995.603327:0:86625:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.603331:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00020000:00000080:1.0:1787826995.603335:0:86625:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.603340:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00020000:00000080:1.0:1787826995.603343:0:86625:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.603348:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00020000:00000080:1.0:1787826995.603352:0:86625:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.603357:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00020000:00000080:1.0:1787826995.603361:0:86625:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.603365:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00020000:00000080:1.0:1787826995.603369:0:86625:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.603373:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00020000:00000080:1.0:1787826995.603376:0:86625:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.603389:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00020000:00000080:1.0:1787826995.603397:0:86625:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.603403:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00020000:00000080:1.0:1787826995.603409:0:86625:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.603416:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00020000:00000080:1.0:1787826995.603420:0:86625:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.603426:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00020000:00000080:1.0:1787826995.603431:0:86625:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.603436:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00020000:00000080:1.0:1787826995.603440:0:86625:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.603445:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00020000:00000080:1.0:1787826995.603450:0:86625:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.603470:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00020000:00000080:1.0:1787826995.603475:0:86625:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.603479:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00020000:00000080:1.0:1787826995.603483:0:86625:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.603488:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00020000:00000080:1.0:1787826995.603493:0:86625:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.603503:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00020000:00000080:1.0:1787826995.603509:0:86625:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.603514:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00020000:00000080:1.0:1787826995.603518:0:86625:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.603523:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00020000:00000080:1.0:1787826995.603528:0:86625:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.603533:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00020000:00000080:1.0:1787826995.603538:0:86625:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.603543:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00020000:00000080:1.0:1787826995.603546:0:86625:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.603550:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00020000:00000080:1.0:1787826995.603553:0:86625:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.603557:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00020000:00000080:1.0:1787826995.603561:0:86625:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.603565:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00020000:00000080:1.0:1787826995.603568:0:86625:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.603573:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00020000:00000080:1.0:1787826995.603587:0:86625:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.603597:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00020000:00000080:1.0:1787826995.603601:0:86625:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.603606:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00020000:00000080:1.0:1787826995.603610:0:86625:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.603615:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00020000:00000080:1.0:1787826995.603618:0:86625:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.603623:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00020000:00000080:1.0:1787826995.603627:0:86625:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.603632:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00020000:00000080:1.0:1787826995.603636:0:86625:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.603640:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00020000:00000080:1.0:1787826995.603643:0:86625:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.603647:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00020000:00000080:1.0:1787826995.603650:0:86625:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.603655:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00020000:00000080:1.0:1787826995.603666:0:86625:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.603675:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00020000:00000080:1.0:1787826995.603680:0:86625:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.603685:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00020000:00000080:1.0:1787826995.603689:0:86625:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.603715:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00020000:00000080:1.0:1787826995.603720:0:86625:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.603726:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00020000:00000080:1.0:1787826995.603731:0:86625:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.603736:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00020000:00000080:1.0:1787826995.603741:0:86625:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.603748:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00020000:00000080:1.0:1787826995.603753:0:86625:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.603758:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00020000:00000080:1.0:1787826995.603763:0:86625:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.603768:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00020000:00000080:1.0:1787826995.603771:0:86625:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.603776:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00020000:00000080:1.0:1787826995.603780:0:86625:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200080:1.0:1787826995.603784:0:86625:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff67b2a010 00020000:00000080:1.0:1787826995.603788:0:86625:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608479c00 uarg=0000000000000000 00000080:00200008:1.0:1787826995.603813:0:86625:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 100000 00010000:00010000:3.0:1787826996.021841:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826996.468828:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607ce4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2383:1874669227151488:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787826996.468845:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607ce4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669227151616:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787826996.468932:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607ce70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669227151744:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787826996.469377:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607ce6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669227152000:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787826996.469381:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607ce5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669227151872:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787826996.471146:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607ce4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2383:1874669227151488:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787826996.471231:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607ce4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669227151616:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787826996.471249:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607ce5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669227151872:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787826996.471323:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607ce70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669227151744:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787826996.471333:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607ce6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669227152000:192.168.203.114@tcp:400:kworker.0 00010000:00010000:3.0:1787827000.117178:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827001.140996:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827001.589092:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607ca5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669227356928:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827001.589195:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607ca7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669227357184:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827001.589239:0:2383:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [1->0] 00000100:00100000:1.0:1787827001.589249:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607ca6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2383:1874669227356800:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827001.589298:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607ca43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669227357056:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827001.589579:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607ca5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669227357312:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827001.591537:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607ca6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2383:1874669227356800:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827001.591563:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607ca5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669227357312:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827001.591574:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607ca7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669227357184:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827001.592744:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607ca43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669227357056:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827001.592758:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607ca5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669227356928:192.168.203.114@tcp:400:kworker.0 00000080:00200000:1.0:1787827001.954325:0:85228:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787827001.954356:0:85228:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827001.954417:0:85228:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe7600/0xb3f83dad1b7ea348 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 85228 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827001.954428:0:85228:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe7600/0xb3f83dad1b7ea348 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 85228 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827001.954438:0:85228:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe7600/0xb3f83dad1b7ea348 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 85228 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787827001.954472:0:85228:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607ca5680 pname:cluuid:pid:xid:nid:opc:job bash:7b4e8535-9db7-485d-abf6-bda966b63320:85228:1874669227357440:192.168.203.114@tcp:101:bash.0 00000100:00100000:1.0:1787827001.954566:0:85228:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607ee2000 going to sleep for 11 seconds 00000100:00100000:1.0:1787827001.957386:0:85228:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607ca5680 pname:cluuid:pid:xid:nid:opc:job bash:7b4e8535-9db7-485d-abf6-bda966b63320:85228:1874669227357440:192.168.203.114@tcp:101:bash.0 00010000:00010000:1.0:1787827001.957406:0:85228:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe7600/0xb3f83dad1b7ea348 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91d0404 expref: -99 pid: 85228 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827001.957417:0:85228:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe7600/0xb3f83dad1b7ea348 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91d0404 expref: -99 pid: 85228 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827001.957430:0:85228:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe7600/0xb3f83dad1b7ea348 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91d0404 expref: -99 pid: 85228 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827001.957440:0:85228:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe7600/0xb3f83dad1b7ea348 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91d0404 expref: -99 pid: 85228 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827001.957452:0:85228:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe7600/0xb3f83dad1b7ea348 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91d0404 expref: -99 pid: 85228 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827001.957460:0:85228:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe7600/0xb3f83dad1b7ea348 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91d0404 expref: -99 pid: 85228 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827001.957468:0:85228:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe7600/0xb3f83dad1b7ea348 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91d0404 expref: -99 pid: 85228 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787827001.957479:0:85228:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff985607ca5680 x1874669227357440/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 576/688 e 0 to 0 dl 1787827017 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'bash.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787827001.957492:0:85228:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe7600/0xb3f83dad1b7ea348 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91d0404 expref: -99 pid: 85228 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827001.957527:0:85228:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:1.0:1787827001.957535:0:85228:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:1.0:1787827001.957559:0:85228:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 69632 00000080:00010000:1.0:1787827001.957566:0:85228:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ea348 00000080:00010000:1.0:1787827001.957572:0:85228:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787827001.957575:0:85228:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ea348 from it ffffab3ac1563bb0 00010000:00010000:1.0:1787827001.957578:0:85228:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe7600/0xb3f83dad1b7ea348 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91d0404 expref: -99 pid: 85228 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827001.957587:0:85228:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe7600/0xb3f83dad1b7ea348 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91d0404 expref: -99 pid: 85228 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787827001.957609:0:85228:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 44 00000080:00200008:1.0:1787827001.957620:0:85228:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 304000 00000080:00200008:1.0:1787827001.957639:0:85228:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 2047, flags 0, foreign 0 00000002:00010000:1.0:1787827001.957659:0:85228:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: readdir flags 0 00010000:00010000:1.0:1787827001.957667:0:85228:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe7600/0xb3f83dad1b7ea348 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91d0404 expref: -99 pid: 85228 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827001.957678:0:85228:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe7600/0xb3f83dad1b7ea348 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91d0404 expref: -99 pid: 85228 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000100:00100000:1.0:1787827001.958605:0:85228:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607ca5680 pname:cluuid:pid:xid:nid:opc:job bash:7b4e8535-9db7-485d-abf6-bda966b63320:85228:1874669227357568:192.168.203.114@tcp:37:bash.0 00000100:00100000:1.0:1787827001.958755:0:85228:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607ee2000 going to sleep for 11 seconds 00000100:00100000:1.0:1787827001.967082:0:85228:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607ca5680 pname:cluuid:pid:xid:nid:opc:job bash:7b4e8535-9db7-485d-abf6-bda966b63320:85228:1874669227357568:192.168.203.114@tcp:37:bash.0 00010000:00010000:1.0:1787827001.967834:0:85228:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe7600/0xb3f83dad1b7ea348 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91d0404 expref: -99 pid: 85228 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827001.967851:0:85228:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe7600/0xb3f83dad1b7ea348 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91d0404 expref: -99 pid: 85228 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787827001.967882:0:85228:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787827001.967904:0:85228:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827001.967918:0:85228:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe7600/0xb3f83dad1b7ea348 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91d0404 expref: -99 pid: 85228 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787827001.967930:0:85228:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe7600/0xb3f83dad1b7ea348 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91d0404 expref: -99 pid: 85228 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787827001.967937:0:85228:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827001.967942:0:85228:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787827001.967946:0:85228:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ea348 00000080:00010000:1.0:1787827001.967951:0:85228:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ea348 from it ffffab3ac1563cd8 00010000:00010000:1.0:1787827001.967955:0:85228:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe7600/0xb3f83dad1b7ea348 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91d0404 expref: -99 pid: 85228 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787827001.967962:0:85228:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe7600/0xb3f83dad1b7ea348 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91d0404 expref: -99 pid: 85228 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 1 00000080:00200000:1.0:1787827001.968265:0:85228:0:(dir.c:332:ll_iterate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) pos/size0/69632 32bit_api 0 00000080:00200000:1.0:1787827001.968303:0:85228:0:(dir.c:332:ll_iterate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) pos/size18446744073709551614/69632 32bit_api 0 00000080:00200008:1.0:1787827001.968316:0:85228:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 304000 00000080:00200000:3.0:1787827001.973270:0:86842:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:3.0:1787827001.973298:0:86842:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787827001.973317:0:86842:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe7600/0xb3f83dad1b7ea348 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91d0404 expref: -99 pid: 85228 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 1 00010000:00010000:3.0:1787827001.973333:0:86842:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe7600/0xb3f83dad1b7ea348 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91d0404 expref: -99 pid: 85228 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 1 00000080:00200000:3.0:1787827001.973341:0:86842:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787827001.973348:0:86842:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:3.0:1787827001.973352:0:86842:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ea348 00000080:00010000:3.0:1787827001.973359:0:86842:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ea348 from it ffffab3ac1c97b58 00010000:00010000:3.0:1787827001.973364:0:86842:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe7600/0xb3f83dad1b7ea348 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91d0404 expref: -99 pid: 85228 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 1 00010000:00010000:3.0:1787827001.973372:0:86842:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe7600/0xb3f83dad1b7ea348 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91d0404 expref: -99 pid: 85228 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00000080:00200000:3.0:1787827001.973385:0:86842:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:3.0:1787827001.973396:0:86842:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f39.replay-single-*, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), flags=8 00000080:00200000:3.0:1787827001.973401:0:86842:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f39.replay-single-*, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=getattr 00000002:00010000:3.0:1787827001.973412:0:86842:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f39.replay-single-*,[0x0:0x0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 01 00010000:00010000:3.0:1787827001.973453:0:86842:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06600/0xb3f83dad1b7ea34f lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 86842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827001.973460:0:86842:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06600/0xb3f83dad1b7ea34f lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 86842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827001.973470:0:86842:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06600/0xb3f83dad1b7ea34f lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 86842 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787827001.973494:0:86842:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e81e00 pname:cluuid:pid:xid:nid:opc:job checkstat:7b4e8535-9db7-485d-abf6-bda966b63320:86842:1874669227357696:192.168.203.114@tcp:101:checkstat.0 00000100:00100000:3.0:1787827001.973579:0:86842:0:(client.c:2715:ptlrpc_set_wait()) set ffff98560682c300 going to sleep for 11 seconds 00000100:00100000:3.0:1787827001.975475:0:86842:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e81e00 pname:cluuid:pid:xid:nid:opc:job checkstat:7b4e8535-9db7-485d-abf6-bda966b63320:86842:1874669227357696:192.168.203.114@tcp:101:checkstat.0 00010000:00010000:3.0:1787827001.975495:0:86842:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06600/0xb3f83dad1b7ea34f lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 86842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827001.975507:0:86842:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06600/0xb3f83dad1b7ea34f lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 86842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827001.975514:0:86842:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06600/0xb3f83dad1b7ea34f lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 86842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827001.975521:0:86842:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06600/0xb3f83dad1b7ea34f lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 86842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827001.975528:0:86842:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06600/0xb3f83dad1b7ea34f lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 86842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827001.975534:0:86842:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985610e06600 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:3.0:1787827001.975539:0:86842:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06600/0xb3f83dad1b7ea34f lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 86842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827001.975546:0:86842:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06600/0xb3f83dad1b7ea34f lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 86842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827001.975553:0:86842:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06600/0xb3f83dad1b7ea34f lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x20e400000000 pid: 86842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827001.975561:0:86842:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787827001.975567:0:86842:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06600/0xb3f83dad1b7ea34f lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 86842 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827001.975574:0:86842:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06600/0xb3f83dad1b7ea34f lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 86842 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787827001.975589:0:86842:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff985610e81e00 x1874669227357696/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/584 e 0 to 0 dl 1787827017 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'checkstat.0' uid:0 gid:0 projid:0 00000080:00200000:3.0:1787827001.975603:0:86842:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00010000:00010000:3.0:1787827001.975627:0:86842:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe7600/0xb3f83dad1b7ea348 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91d0404 expref: -99 pid: 85228 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00010000:00010000:3.0:1787827001.975639:0:86842:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe7600/0xb3f83dad1b7ea348 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91d0404 expref: -99 pid: 85228 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00000080:00010000:3.0:1787827001.975647:0:86842:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ea348 from it ffffab3ac1c97930 00010000:00010000:3.0:1787827001.975650:0:86842:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe7600/0xb3f83dad1b7ea348 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91d0404 expref: -99 pid: 85228 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00010000:00010000:3.0:1787827001.975658:0:86842:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe7600/0xb3f83dad1b7ea348 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91d0404 expref: -99 pid: 85228 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00010000:00010000:2.0:1787827002.164831:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827003.189998:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827004.213009:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:3.0:1787827004.403757:0:87406:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == replay-single test 41: read from a valid osc while other oscs are invalid ========================================================== 06:36:43 (1787827003) 00000080:00200000:3.0:1787827004.438445:0:87410:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:3.0:1787827004.438473:0:87410:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787827004.438492:0:87410:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe7600/0xb3f83dad1b7ea348 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91d0404 expref: -99 pid: 85228 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00010000:00010000:3.0:1787827004.438507:0:87410:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe7600/0xb3f83dad1b7ea348 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91d0404 expref: -99 pid: 85228 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00000080:00200000:3.0:1787827004.438516:0:87410:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787827004.438624:0:87410:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:3.0:1787827004.438629:0:87410:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ea348 00000080:00010000:3.0:1787827004.438637:0:87410:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ea348 from it ffffab3ac1e2fbf8 00010000:00010000:3.0:1787827004.438641:0:87410:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe7600/0xb3f83dad1b7ea348 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91d0404 expref: -99 pid: 85228 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00010000:00010000:3.0:1787827004.438650:0:87410:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe7600/0xb3f83dad1b7ea348 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91d0404 expref: -99 pid: 85228 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00000080:00200000:3.0:1787827004.438664:0:87410:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:3.0:1787827004.438679:0:87410:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:3.0:1787827004.438686:0:87410:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787827004.438740:0:87410:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe7600/0xb3f83dad1b7ea348 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91d0404 expref: -99 pid: 85228 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00010000:00010000:3.0:1787827004.438750:0:87410:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe7600/0xb3f83dad1b7ea348 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91d0404 expref: -99 pid: 85228 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00000080:00200000:3.0:1787827004.438758:0:87410:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787827004.438763:0:87410:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:3.0:1787827004.438766:0:87410:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ea348 00000080:00010000:3.0:1787827004.438770:0:87410:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ea348 from it ffffab3ac1e2fb38 00010000:00010000:3.0:1787827004.438774:0:87410:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe7600/0xb3f83dad1b7ea348 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91d0404 expref: -99 pid: 85228 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00010000:00010000:3.0:1787827004.438781:0:87410:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe7600/0xb3f83dad1b7ea348 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91d0404 expref: -99 pid: 85228 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 1 00000080:00200000:3.0:1787827004.438789:0:87410:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 3 00000080:00200000:3.0:1787827004.438795:0:87410:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f41.replay-single, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), file ffff9856085b5200, open_flags a141, mode 81b6 opened 0 00000080:00200000:3.0:1787827004.438804:0:87410:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f41.replay-single, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=open|creat 00000002:00010000:3.0:1787827004.438823:0:87410:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f41.replay-single,[0x2000182d1:0x643:0x0]) in obj [0x200000007:0x1:0x0], intent: open|creat flags 0120502 00010000:00010000:3.0:1787827004.438834:0:87410:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe7600/0xb3f83dad1b7ea348 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x5aa6d504d91d0404 expref: -99 pid: 85228 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 1 00010000:00010000:3.0:1787827004.438924:0:87410:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe7600/0xb3f83dad1b7ea348 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91d0404 expref: -99 pid: 85228 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 1 00010000:00010000:3.0:1787827004.438931:0:87410:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787827004.438934:0:87410:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe7600/0xb3f83dad1b7ea348 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91d0404 expref: -99 pid: 85228 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 1 00010000:00010000:3.0:1787827004.438981:0:87410:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06800/0xb3f83dad1b7ea356 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 87410 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827004.438988:0:87410:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06800/0xb3f83dad1b7ea356 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 87410 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:3.0:1787827004.438999:0:87410:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:3.0:1787827004.439011:0:87410:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610e83480 x1874669227357952/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 536/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:3.0:1787827004.439025:0:87410:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06800/0xb3f83dad1b7ea356 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 87410 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787827004.439045:0:87410:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e83480 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:87410:1874669227357952:192.168.203.114@tcp:101:lfs.0 00000100:00100000:3.0:1787827004.439168:0:87410:0:(client.c:2715:ptlrpc_set_wait()) set ffff985605721600 going to sleep for 11 seconds 00000100:00100000:0.0:1787827004.443051:0:87410:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 219043332496 gen 9 00000100:00100000:0.0:1787827004.444116:0:87410:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610e83480 x1874669227357952/t219043332497(219043332497) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 536/608 e 0 to 0 dl 1787827020 ref 3 fl Rpc:RPQU/604/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000100:00100000:0.0:1787827004.444129:0:87410:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e83480 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:87410:1874669227357952:192.168.203.114@tcp:101:lfs.0 00010000:00010000:0.0:1787827004.444147:0:87410:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06800/0xb3f83dad1b7ea356 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91d0d81 expref: -99 pid: 87410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827004.444169:0:87410:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06800/0xb3f83dad1b7ea356 lrc: 4/0,1 mode: --/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91d0d81 expref: -99 pid: 87410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827004.444179:0:87410:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06800/0xb3f83dad1b7ea356 lrc: 4/0,1 mode: --/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91d0d81 expref: -99 pid: 87410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827004.444187:0:87410:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06800/0xb3f83dad1b7ea356 lrc: 4/0,1 mode: PR/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91d0d81 expref: -99 pid: 87410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827004.444194:0:87410:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06800/0xb3f83dad1b7ea356 lrc: 4/0,1 mode: PR/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91d0d81 expref: -99 pid: 87410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827004.444203:0:87410:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06800/0xb3f83dad1b7ea356 lrc: 5/1,1 mode: PR/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91d0d81 expref: -99 pid: 87410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827004.444208:0:87410:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06800/0xb3f83dad1b7ea356 lrc: 5/1,1 mode: PR/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91d0d81 expref: -99 pid: 87410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827004.444213:0:87410:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06800/0xb3f83dad1b7ea356 lrc: 4/1,0 mode: PR/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91d0d81 expref: -99 pid: 87410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787827004.444220:0:87410:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x8000037, status=0 req@ffff985610e83480 x1874669227357952/t219043332497(219043332497) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 536/608 e 0 to 0 dl 1787827020 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00100000:0.0:1787827004.444228:0:87410:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff985610e83480 x1874669227357952/t219043332497(219043332497) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 536/608 e 0 to 0 dl 1787827020 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787827004.444237:0:87410:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06800/0xb3f83dad1b7ea356 lrc: 3/1,0 mode: PR/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91d0d81 expref: -99 pid: 87410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827004.444244:0:87410:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8590033617/1603 (0 0) 00000080:00200000:0.0:1787827004.444249:0:87410:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:0.0:1787827004.444258:0:87410:0:(namei.c:1628:ll_create_it()) VFS Op:name=f41.replay-single, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=open|creat 00000080:00200000:0.0:1787827004.444284:0:87410:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x2000182d1:0x643:0x0](ffff9856102a5348) 00000080:00200000:0.0:1787827004.444292:0:87410:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x2000182d1:0x643:0x0], updating i_size 0 00000080:00200000:0.0:1787827004.444324:0:87410:0:(file.c:6854:ll_layout_conf()) [0x2000182d1:0x643:0x0]: layout version change: 4294967294 -> 4294967295 00000080:00010000:0.0:1787827004.444329:0:87410:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x2000182d1:0x643:0x0](ffff9856102a5348) 00000080:00010000:0.0:1787827004.444332:0:87410:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x2000182d1:0x643:0x0](ffff9856102a5348) for lock 0xb3f83dad1b7ea356 00000080:00010000:0.0:1787827004.444340:0:87410:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x2000182d1:0x643:0x0](ffff9856102a5348) 00000080:00200008:0.0:1787827004.444351:0:87410:0:(file.c:1090:ll_file_open()) START file f41.replay-single:[0x2000182d1:0x643:0x0](ffff9856102a5348), flags 120501 00000002:00100000:0.0:1787827004.444370:0:87410:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff985610e83480 x1874669227357952/t219043332497(219043332497) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 536/608 e 0 to 0 dl 1787827020 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00010000:0.0:1787827004.444400:0:87410:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ea356 from it ffff985610c4a300 00010000:00010000:0.0:1787827004.444405:0:87410:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06800/0xb3f83dad1b7ea356 lrc: 3/1,0 mode: PR/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91d0d81 expref: -99 pid: 87410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827004.444422:0:87410:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06800/0xb3f83dad1b7ea356 lrc: 2/0,0 mode: PR/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91d0d81 expref: -99 pid: 87410 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787827004.444439:0:87410:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x2000182d1:0x643:0x0](ffff9856102a5348), inode mode 81a4 mask 40 00000080:00200080:0.0:1787827004.444459:0:87410:0:(file.c:4655:ll_file_ioctl()) VFS Op:inode=[0x2000182d1:0x643:0x0](ffff9856102a5348) cmd=4008669a arg=17772e0 00000002:00010000:0.0:1787827004.444476:0:87410:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x2000182d1:0x643:0x0]) in obj [0x200000007:0x1:0x0], intent: open flags 040000002 00010000:00010000:0.0:1787827004.444511:0:87410:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57600/0xb3f83dad1b7ea35d lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 87410 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827004.444516:0:87410:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57600/0xb3f83dad1b7ea35d lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 87410 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:0.0:1787827004.444523:0:87410:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:0.0:1787827004.444527:0:87410:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9856085361c0 x1874669227358208/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 648/131856 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:0.0:1787827004.444533:0:87410:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57600/0xb3f83dad1b7ea35d lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 87410 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787827004.444547:0:87410:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085361c0 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:87410:1874669227358208:192.168.203.114@tcp:101:lfs.0 00000100:00100000:0.0:1787827004.444688:0:87410:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5b9c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787827004.447083:0:2377:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.114@tcp (source: 12345-192.168.203.114@tcp) 00000100:00100000:3.0:1787827004.447116:0:15155:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874669235869824 00000100:00100000:3.0:1787827004.447123:0:15155:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874669235869824 00000100:00100000:3.0:1787827004.447166:0:15155:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.114@tcp, seq: 255 00000100:00100000:3.0:1787827004.447177:0:15155:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff985610e80000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb01_000:7b4e8535-9db7-485d-abf6-bda966b63320+5:95558:x1874669235869824:12345-192.168.203.114@tcp:104: 00010000:00010000:3.0:1787827004.447196:0:15155:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06800/0xb3f83dad1b7ea356 lrc: 2/0,0 mode: PR/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x420000000000 nid: local remote: 0x5aa6d504d91d0d81 expref: -99 pid: 87410 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787827004.447436:0:15155:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:3.0:1787827004.447446:0:15155:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff985610e80000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb01_000:7b4e8535-9db7-485d-abf6-bda966b63320+5:95558:x1874669235869824:12345-192.168.203.114@tcp:104: Request processed in 269us (369us total) trans 0 rc 0/0 00000100:00100000:3.0:1787827004.447454:0:15155:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.114@tcp, seq: 255 00010000:00010000:0.0:1787827004.447549:0:79887:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787827004.447556:0:79887:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06800/0xb3f83dad1b7ea356 lrc: 2/0,0 mode: PR/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x420000000000 nid: local remote: 0x5aa6d504d91d0d81 expref: -99 pid: 87410 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787827004.447566:0:79887:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985610e06800 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:0.0:1787827004.447571:0:79887:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06800/0xb3f83dad1b7ea356 lrc: 2/0,0 mode: PR/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x1b rrc: 2 type: IBT gid 0 flags: 0x420400000000 nid: local remote: 0x5aa6d504d91d0d81 expref: -99 pid: 87410 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787827004.447579:0:79887:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06800/0xb3f83dad1b7ea356 lrc: 2/0,0 mode: PR/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x1b rrc: 2 type: IBT gid 0 flags: 0x420400000000 nid: local remote: 0x5aa6d504d91d0d81 expref: -99 pid: 87410 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787827004.447588:0:79887:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06800/0xb3f83dad1b7ea356 lrc: 3/0,0 mode: PR/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x1b rrc: 2 type: IBT gid 0 flags: 0x428400000000 nid: local remote: 0x5aa6d504d91d0d81 expref: -99 pid: 87410 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787827004.447613:0:79887:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x643:0x0]: losing layout lock 00010000:00010000:0.0:1787827004.447653:0:79887:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06800/0xb3f83dad1b7ea356 lrc: 2/0,0 mode: --/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x1b rrc: 2 type: IBT gid 0 flags: 0x4c29400000020 nid: local remote: 0x5aa6d504d91d0d81 expref: -99 pid: 87410 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787827004.447661:0:79887:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787827004.447687:0:79887:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06800/0xb3f83dad1b7ea356 lrc: 1/0,0 mode: --/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x1b rrc: 2 type: IBT gid 0 flags: 0x4c29400000020 nid: local remote: 0x5aa6d504d91d0d81 expref: -99 pid: 87410 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787827004.447720:0:79887:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06800/0xb3f83dad1b7ea356 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x1b rrc: 2 type: IBT gid 0 flags: 0x4c29400000020 nid: local remote: 0x5aa6d504d91d0d81 expref: -99 pid: 87410 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000100:00100000:1.0:1787827004.447727:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608534f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669227358336:192.168.203.114@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787827004.447738:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827004.447741:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827004.447744:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827004.452515:0:87410:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085361c0 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:87410:1874669227358208:192.168.203.114@tcp:101:lfs.0 00010000:00010000:0.0:1787827004.452541:0:87410:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57600/0xb3f83dad1b7ea35d lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 87410 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787827004.452543:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608534f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669227358336:192.168.203.114@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787827004.452572:0:87410:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57600/0xb3f83dad1b7ea35d lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 87410 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827004.452582:0:87410:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57600/0xb3f83dad1b7ea35d lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 87410 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827004.452592:0:87410:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57600/0xb3f83dad1b7ea35d lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 87410 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827004.452601:0:87410:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57600/0xb3f83dad1b7ea35d lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 87410 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827004.452608:0:87410:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985607f57600 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:0.0:1787827004.452613:0:87410:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57600/0xb3f83dad1b7ea35d lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 2 type: IBT flags: 0x206400000000 pid: 87410 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827004.452620:0:87410:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57600/0xb3f83dad1b7ea35d lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 2 type: IBT flags: 0x206400000000 pid: 87410 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827004.452631:0:87410:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57600/0xb3f83dad1b7ea35d lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 2 type: IBT flags: 0x20e400000000 pid: 87410 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827004.452639:0:87410:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787827004.452646:0:87410:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57600/0xb3f83dad1b7ea35d lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 87410 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827004.452654:0:87410:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57600/0xb3f83dad1b7ea35d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 87410 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787827004.452683:0:87410:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x1 disposition=0x800002b, status=0 req@ffff9856085361c0 x1874669227358208/t219043332498(219043332498) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 648/664 e 0 to 0 dl 1787827020 ref 2 fl Complete:RPQU/604/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00100000:0.0:1787827004.452728:0:87410:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9856085361c0 x1874669227358208/t219043332498(219043332498) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 648/664 e 0 to 0 dl 1787827020 ref 2 fl Complete:RPQU/604/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787827004.452744:0:87410:0:(namei.c:952:ll_intent_lock()) intent lock 1 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:0.0:1787827004.452788:0:87410:0:(vvp_io.c:1885:vvp_io_init()) [0x2000182d1:0x643:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:0.0:1787827004.452798:0:87410:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x643:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787827004.452844:0:87410:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x2000182d1:0x643:0x0], updating i_size 0 00000002:00100000:0.0:1787827004.452863:0:87410:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9856085361c0 x1874669227358208/t219043332498(219043332498) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 576/664 e 0 to 0 dl 1787827020 ref 2 fl Complete:RPQU/604/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00100000:0.0:1787827004.452900:0:87410:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9856085361c0 x1874669227358208/t219043332498(219043332498) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 576/664 e 0 to 0 dl 1787827020 ref 2 fl Complete:RPQU/604/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000020:00100000:0.0:1787827004.452911:0:87410:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:0.0:1787827004.452916:0:87410:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985608535e00 x1874669227358592/t0(0) o35->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:0.0:1787827004.452938:0:87410:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608535e00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:87410:1874669227358592:192.168.203.114@tcp:35:lfs.0 00000100:00100000:0.0:1787827004.453252:0:87410:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5be40 going to sleep for 11 seconds 00000100:00100000:0.0:1787827004.456043:0:87410:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608535e00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:87410:1874669227358592:192.168.203.114@tcp:35:lfs.0 00000002:00100000:0.0:1787827004.456085:0:87410:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9856085361c0 x1874669227358208/t219043332498(219043332498) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 576/664 e 0 to 0 dl 1787827020 ref 2 fl Complete:RQU/604/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00010000:0.0:1787827004.456122:0:87410:0:(file.c:7076:ll_layout_intent()) ### lustre: requeue layout lock for file [0x2000182d1:0x643:0x0](ffff9856102a5348) 00000002:00010000:0.0:1787827004.456135:0:87410:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x2000182d1:0x643:0x0]) in obj [0x2000182d1:0x643:0x0], intent: layout flags 0 00010000:00010000:0.0:1787827004.456178:0:87410:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57800/0xb3f83dad1b7ea364 lrc: 3/1,0 mode: --/CR res: [0x2000182d1:0x643:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 87410 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827004.456187:0:87410:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57800/0xb3f83dad1b7ea364 lrc: 3/1,0 mode: --/CR res: [0x2000182d1:0x643:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x0 pid: 87410 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827004.456195:0:87410:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57800/0xb3f83dad1b7ea364 lrc: 3/1,0 mode: --/CR res: [0x2000182d1:0x643:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x0 pid: 87410 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787827004.456211:0:87410:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085352c0 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:87410:1874669227358720:192.168.203.114@tcp:101:lfs.0 00000100:00100000:0.0:1787827004.456282:0:87410:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5be40 going to sleep for 11 seconds 00000100:00100000:0.0:1787827004.458118:0:87410:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085352c0 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:87410:1874669227358720:192.168.203.114@tcp:101:lfs.0 00010000:00010000:0.0:1787827004.458143:0:87410:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57800/0xb3f83dad1b7ea364 lrc: 4/1,0 mode: CR/CR res: [0x2000182d1:0x643:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91d0da4 expref: -99 pid: 87410 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827004.458157:0:87410:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57800/0xb3f83dad1b7ea364 lrc: 4/1,0 mode: CR/CR res: [0x2000182d1:0x643:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91d0da4 expref: -99 pid: 87410 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787827004.458171:0:87410:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x400 disposition=0x1, status=0 req@ffff9856085352c0 x1874669227358720/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 376/400 e 0 to 0 dl 1787827020 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787827004.458183:0:87410:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: layout, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57800/0xb3f83dad1b7ea364 lrc: 3/1,0 mode: CR/CR res: [0x2000182d1:0x643:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91d0da4 expref: -99 pid: 87410 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00000002:00010000:0.0:1787827004.458196:0:87410:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57800/0xb3f83dad1b7ea364 lrc: 3/1,0 mode: CR/CR res: [0x2000182d1:0x643:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91d0da4 expref: -99 pid: 87410 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827004.458208:0:87410:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 16 res 8590033617/1603 (0 0) 00000080:00200000:0.0:1787827004.458214:0:87410:0:(namei.c:952:ll_intent_lock()) intent lock 1024 on i1 [0x2000182d1:0x643:0x0] suppgids 0 0: rc 0 00000080:00010000:0.0:1787827004.458228:0:87410:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x2000182d1:0x643:0x0](ffff9856102a5348) for lock 0xb3f83dad1b7ea364 00000080:00010000:0.0:1787827004.458235:0:87410:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ea364 from it ffffab3ac1e2fc38 00010000:00010000:0.0:1787827004.458239:0:87410:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57800/0xb3f83dad1b7ea364 lrc: 3/1,0 mode: CR/CR res: [0x2000182d1:0x643:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91d0da4 expref: -99 pid: 87410 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827004.458248:0:87410:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57800/0xb3f83dad1b7ea364 lrc: 2/0,0 mode: CR/CR res: [0x2000182d1:0x643:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91d0da4 expref: -99 pid: 87410 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787827004.458271:0:87410:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57800/0xb3f83dad1b7ea364 lrc: 2/1,0 mode: CR/CR res: [0x2000182d1:0x643:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91d0da4 expref: -99 pid: 87410 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787827004.458282:0:87410:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57800/0xb3f83dad1b7ea364 lrc: 2/1,0 mode: CR/CR res: [0x2000182d1:0x643:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91d0da4 expref: -99 pid: 87410 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00010000:0.0:1787827004.458290:0:87410:0:(file.c:6959:ll_layout_lock_set()) ### file [0x2000182d1:0x643:0x0](00000000c93f57fd) being reconfigured ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57800/0xb3f83dad1b7ea364 lrc: 3/1,0 mode: CR/CR res: [0x2000182d1:0x643:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91d0da4 expref: -99 pid: 87410 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787827004.458340:0:87410:0:(file.c:6854:ll_layout_conf()) [0x2000182d1:0x643:0x0]: layout version change: 4294967294 -> 0 00010000:00010000:0.0:1787827004.458347:0:87410:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57800/0xb3f83dad1b7ea364 lrc: 3/1,0 mode: CR/CR res: [0x2000182d1:0x643:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91d0da4 expref: -99 pid: 87410 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787827004.458356:0:87410:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57800/0xb3f83dad1b7ea364 lrc: 2/0,0 mode: CR/CR res: [0x2000182d1:0x643:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91d0da4 expref: -99 pid: 87410 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00000080:00200008:0.0:1787827004.458414:0:87410:0:(file.c:479:ll_file_release()) START file f41.replay-single:[0x2000182d1:0x643:0x0](ffff9856102a5348), flags 100001 00000002:00100000:0.0:1787827004.458452:0:87410:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff985610e83480 x1874669227357952/t219043332497(219043332497) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 536/608 e 0 to 0 dl 1787827020 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:0 00000020:00100000:0.0:1787827004.458464:0:87410:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:0.0:1787827004.458470:0:87410:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9856085352c0 x1874669227358976/t0(0) o35->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:0.0:1787827004.458489:0:87410:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085352c0 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:87410:1874669227358976:192.168.203.114@tcp:35:lfs.0 00000100:00100000:0.0:1787827004.458576:0:87410:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5be40 going to sleep for 11 seconds 00000100:00100000:1.0:1787827004.460404:0:87410:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085352c0 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:87410:1874669227358976:192.168.203.114@tcp:35:lfs.0 00000002:00100000:1.0:1787827004.460431:0:87410:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff985610e83480 x1874669227357952/t219043332497(219043332497) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 536/608 e 0 to 0 dl 1787827020 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787827004.482639:0:87423:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:0.0:1787827004.482674:0:87423:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827004.482768:0:87423:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57600/0xb3f83dad1b7ea36b lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 87423 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827004.482786:0:87423:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57600/0xb3f83dad1b7ea36b lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 87423 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827004.482797:0:87423:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57600/0xb3f83dad1b7ea36b lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 87423 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787827004.482826:0:87423:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608536d00 pname:cluuid:pid:xid:nid:opc:job dd:7b4e8535-9db7-485d-abf6-bda966b63320:87423:1874669227359104:192.168.203.114@tcp:101:dd.0 00000100:00100000:0.0:1787827004.482917:0:87423:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5b180 going to sleep for 11 seconds 00000100:00100000:3.0:1787827004.485156:0:87423:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608536d00 pname:cluuid:pid:xid:nid:opc:job dd:7b4e8535-9db7-485d-abf6-bda966b63320:87423:1874669227359104:192.168.203.114@tcp:101:dd.0 00010000:00010000:3.0:1787827004.485990:0:87423:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57600/0xb3f83dad1b7ea36b lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91d0db2 expref: -99 pid: 87423 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827004.486003:0:87423:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57600/0xb3f83dad1b7ea36b lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91d0db2 expref: -99 pid: 87423 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827004.486015:0:87423:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57600/0xb3f83dad1b7ea36b lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91d0db2 expref: -99 pid: 87423 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827004.486032:0:87423:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57600/0xb3f83dad1b7ea36b lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91d0db2 expref: -99 pid: 87423 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827004.486044:0:87423:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57600/0xb3f83dad1b7ea36b lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91d0db2 expref: -99 pid: 87423 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827004.486052:0:87423:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57600/0xb3f83dad1b7ea36b lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91d0db2 expref: -99 pid: 87423 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827004.486060:0:87423:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57600/0xb3f83dad1b7ea36b lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91d0db2 expref: -99 pid: 87423 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787827004.486074:0:87423:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff985608536d00 x1874669227359104/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 576/688 e 0 to 0 dl 1787827020 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787827004.486089:0:87423:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57600/0xb3f83dad1b7ea36b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91d0db2 expref: -99 pid: 87423 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827004.486101:0:87423:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:3.0:1787827004.486108:0:87423:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:3.0:1787827004.486131:0:87423:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 69632 00000080:00010000:3.0:1787827004.486136:0:87423:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ea36b 00000080:00010000:3.0:1787827004.486141:0:87423:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:3.0:1787827004.486145:0:87423:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ea36b from it ffffab3ac1dcfbf8 00010000:00010000:3.0:1787827004.486148:0:87423:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57600/0xb3f83dad1b7ea36b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91d0db2 expref: -99 pid: 87423 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827004.486162:0:87423:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57600/0xb3f83dad1b7ea36b lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91d0db2 expref: -99 pid: 87423 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787827004.486182:0:87423:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:3.0:1787827004.486200:0:87423:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:3.0:1787827004.486220:0:87423:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787827004.486229:0:87423:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57600/0xb3f83dad1b7ea36b lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91d0db2 expref: -99 pid: 87423 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787827004.486239:0:87423:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57600/0xb3f83dad1b7ea36b lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91d0db2 expref: -99 pid: 87423 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787827004.486245:0:87423:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787827004.486250:0:87423:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:3.0:1787827004.486252:0:87423:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ea36b 00000080:00010000:3.0:1787827004.486256:0:87423:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ea36b from it ffffab3ac1dcfb38 00010000:00010000:3.0:1787827004.486259:0:87423:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57600/0xb3f83dad1b7ea36b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91d0db2 expref: -99 pid: 87423 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787827004.486267:0:87423:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57600/0xb3f83dad1b7ea36b lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91d0db2 expref: -99 pid: 87423 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:3.0:1787827004.486277:0:87423:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 3 00000080:00200000:3.0:1787827004.486283:0:87423:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f41.replay-single, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), file ffff9856031a2600, open_flags 8241, mode 81b6 opened 0 00000080:00200000:3.0:1787827004.486291:0:87423:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f41.replay-single, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=open|creat 00000002:00010000:3.0:1787827004.486309:0:87423:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f41.replay-single,[0x2000182d1:0x644:0x0]) in obj [0x200000007:0x1:0x0], intent: open|creat flags 0101102 00010000:00010000:3.0:1787827004.486321:0:87423:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57600/0xb3f83dad1b7ea36b lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x5aa6d504d91d0db2 expref: -99 pid: 87423 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787827004.486362:0:87423:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57600/0xb3f83dad1b7ea36b lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91d0db2 expref: -99 pid: 87423 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787827004.486369:0:87423:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787827004.486372:0:87423:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57600/0xb3f83dad1b7ea36b lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91d0db2 expref: -99 pid: 87423 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787827004.486402:0:87423:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07200/0xb3f83dad1b7ea372 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 87423 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827004.486444:0:87423:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07200/0xb3f83dad1b7ea372 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 87423 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:3.0:1787827004.486456:0:87423:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:3.0:1787827004.486463:0:87423:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610e812c0 x1874669227359360/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:3.0:1787827004.486472:0:87423:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07200/0xb3f83dad1b7ea372 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 87423 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787827004.486490:0:87423:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e812c0 pname:cluuid:pid:xid:nid:opc:job dd:7b4e8535-9db7-485d-abf6-bda966b63320:87423:1874669227359360:192.168.203.114@tcp:101:dd.0 00000100:00100000:3.0:1787827004.486594:0:87423:0:(client.c:2715:ptlrpc_set_wait()) set ffff985605721a80 going to sleep for 11 seconds 00000100:00100000:1.0:1787827004.490874:0:87423:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e812c0 pname:cluuid:pid:xid:nid:opc:job dd:7b4e8535-9db7-485d-abf6-bda966b63320:87423:1874669227359360:192.168.203.114@tcp:101:dd.0 00010000:00010000:1.0:1787827004.490899:0:87423:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07200/0xb3f83dad1b7ea372 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91d0dce expref: -99 pid: 87423 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827004.490919:0:87423:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07200/0xb3f83dad1b7ea372 lrc: 4/0,1 mode: --/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91d0dce expref: -99 pid: 87423 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827004.490928:0:87423:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07200/0xb3f83dad1b7ea372 lrc: 4/0,1 mode: --/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91d0dce expref: -99 pid: 87423 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827004.490938:0:87423:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07200/0xb3f83dad1b7ea372 lrc: 4/0,1 mode: PR/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91d0dce expref: -99 pid: 87423 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827004.490949:0:87423:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07200/0xb3f83dad1b7ea372 lrc: 4/0,1 mode: PR/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91d0dce expref: -99 pid: 87423 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827004.490974:0:87423:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07200/0xb3f83dad1b7ea372 lrc: 5/1,1 mode: PR/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91d0dce expref: -99 pid: 87423 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827004.490983:0:87423:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07200/0xb3f83dad1b7ea372 lrc: 5/1,1 mode: PR/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91d0dce expref: -99 pid: 87423 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827004.490993:0:87423:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07200/0xb3f83dad1b7ea372 lrc: 4/1,0 mode: PR/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91d0dce expref: -99 pid: 87423 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787827004.491004:0:87423:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x2b, status=0 req@ffff985610e812c0 x1874669227359360/t219043332501(219043332501) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/664 e 0 to 0 dl 1787827020 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00000002:00100000:1.0:1787827004.491016:0:87423:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff985610e812c0 x1874669227359360/t219043332501(219043332501) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/664 e 0 to 0 dl 1787827020 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787827004.491028:0:87423:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07200/0xb3f83dad1b7ea372 lrc: 3/1,0 mode: PR/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91d0dce expref: -99 pid: 87423 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:1.0:1787827004.491040:0:87423:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07200/0xb3f83dad1b7ea372 lrc: 3/1,0 mode: PR/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91d0dce expref: -99 pid: 87423 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827004.491049:0:87423:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8590033617/1603 (0 0) 00000080:00200000:1.0:1787827004.491054:0:87423:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:1.0:1787827004.491093:0:87423:0:(namei.c:147:ll_iget()) got inode: [0x2000182d1:0x643:0x0](ffff9856102a5348): rc = 0 00000080:00200000:1.0:1787827004.491105:0:87423:0:(file.c:6854:ll_layout_conf()) [0x2000182d1:0x643:0x0]: layout version change: 0 -> 0 00000080:00010000:1.0:1787827004.491109:0:87423:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x2000182d1:0x643:0x0](ffff9856102a5348) for lock 0xb3f83dad1b7ea372 00000080:00010000:1.0:1787827004.491118:0:87423:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x2000182d1:0x643:0x0](ffff9856102a5348) 00000080:00200008:1.0:1787827004.491129:0:87423:0:(file.c:1090:ll_file_open()) START file f41.replay-single:[0x2000182d1:0x643:0x0](ffff9856102a5348), flags 101101 00000002:00100000:1.0:1787827004.491148:0:87423:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff985610e812c0 x1874669227359360/t219043332501(219043332501) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787827020 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00000080:00010000:1.0:1787827004.491178:0:87423:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ea372 from it ffff985610c4aa80 00010000:00010000:1.0:1787827004.491182:0:87423:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07200/0xb3f83dad1b7ea372 lrc: 3/1,0 mode: PR/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91d0dce expref: -99 pid: 87423 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827004.491191:0:87423:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07200/0xb3f83dad1b7ea372 lrc: 2/0,0 mode: PR/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91d0dce expref: -99 pid: 87423 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787827004.491209:0:87423:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x2000182d1:0x643:0x0](ffff9856102a5348), inode mode 81a4 mask 42 00000002:00010000:1.0:1787827004.491231:0:87423:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x0:0x0:0x0]) in obj [0x2000182d1:0x643:0x0], intent: getxattr flags 0 00010000:00010000:1.0:1787827004.492201:0:87423:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe6e00/0xb3f83dad1b7ea379 lrc: 3/1,0 mode: --/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 87423 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827004.492545:0:87423:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe6e00/0xb3f83dad1b7ea379 lrc: 3/1,0 mode: --/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x0 pid: 87423 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827004.492556:0:87423:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe6e00/0xb3f83dad1b7ea379 lrc: 3/1,0 mode: --/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x0 pid: 87423 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787827004.492578:0:87423:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607ca5a40 pname:cluuid:pid:xid:nid:opc:job dd:7b4e8535-9db7-485d-abf6-bda966b63320:87423:1874669227359488:192.168.203.114@tcp:101:dd.0 00000100:00100000:1.0:1787827004.492644:0:87423:0:(client.c:2715:ptlrpc_set_wait()) set ffff9856055dc9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827004.497112:0:87423:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607ca5a40 pname:cluuid:pid:xid:nid:opc:job dd:7b4e8535-9db7-485d-abf6-bda966b63320:87423:1874669227359488:192.168.203.114@tcp:101:dd.0 00010000:00010000:1.0:1787827004.497139:0:87423:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe6e00/0xb3f83dad1b7ea379 lrc: 4/1,0 mode: --/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91d0de3 expref: -99 pid: 87423 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827004.497155:0:87423:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe6e00/0xb3f83dad1b7ea379 lrc: 4/1,0 mode: PR/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91d0de3 expref: -99 pid: 87423 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827004.497165:0:87423:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe6e00/0xb3f83dad1b7ea379 lrc: 4/1,0 mode: PR/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91d0de3 expref: -99 pid: 87423 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787827004.497178:0:87423:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x80 disposition=0x0, status=0 req@ffff985607ca5a40 x1874669227359488/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 568/1152 e 0 to 0 dl 1787827020 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787827004.497190:0:87423:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe6e00/0xb3f83dad1b7ea379 lrc: 3/1,0 mode: PR/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91d0de3 expref: -99 pid: 87423 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827004.497203:0:87423:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8590033617/1603 (0 0) 00000080:00200000:1.0:1787827004.497209:0:87423:0:(namei.c:952:ll_intent_lock()) intent lock 128 on i1 [0x2000182d1:0x643:0x0] suppgids 0 -1: rc 0 00000080:00010000:1.0:1787827004.497230:0:87423:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x2000182d1:0x643:0x0](ffff9856102a5348) for lock 0xb3f83dad1b7ea379 00000080:00010000:1.0:1787827004.497238:0:87423:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ea379 from it ffffab3ac1dcfa38 00010000:00010000:1.0:1787827004.497242:0:87423:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe6e00/0xb3f83dad1b7ea379 lrc: 3/1,0 mode: PR/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91d0de3 expref: -99 pid: 87423 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827004.497251:0:87423:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe6e00/0xb3f83dad1b7ea379 lrc: 2/0,0 mode: PR/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91d0de3 expref: -99 pid: 87423 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:1.0:1787827004.497285:0:87423:0:(llite_lib.c:2427:ll_setattr_raw()) START file f41.replay-single:[0x2000182d1:0x643:0x0](ffff9856102a5348) current size 0, valid attrs a068, mode 0, uid 2, gid 0, new size 0, atime 1787827004.000000000, mtime 1787827004.000000000, ctime 1787827004.000000000 00010000:00010000:1.0:1787827004.497311:0:87423:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07200/0xb3f83dad1b7ea372 lrc: 2/0,0 mode: PR/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x5aa6d504d91d0dce expref: -99 pid: 87423 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827004.497327:0:87423:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57800/0xb3f83dad1b7ea364 lrc: 2/0,0 mode: CR/CR res: [0x2000182d1:0x643:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91d0da4 expref: -99 pid: 87410 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827004.497360:0:87423:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07200/0xb3f83dad1b7ea372 lrc: 1/0,0 mode: --/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x5aa6d504d91d0dce expref: -99 pid: 87423 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827004.497367:0:87423:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787827004.497370:0:87423:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07200/0xb3f83dad1b7ea372 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x5aa6d504d91d0dce expref: -99 pid: 87423 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000020:00100000:1.0:1787827004.497391:0:87423:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827004.497398:0:87423:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985607ca5a40 x1874669227359744/t0(0) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 488/512 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827004.497418:0:87423:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607ca5a40 pname:cluuid:pid:xid:nid:opc:job dd:7b4e8535-9db7-485d-abf6-bda966b63320:87423:1874669227359744:192.168.203.114@tcp:36:dd.0 00000100:00100000:1.0:1787827004.497506:0:87423:0:(client.c:2715:ptlrpc_set_wait()) set ffff9856055dc9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827004.498873:0:87423:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607ca5a40 pname:cluuid:pid:xid:nid:opc:job dd:7b4e8535-9db7-485d-abf6-bda966b63320:87423:1874669227359744:192.168.203.114@tcp:36:dd.0 00000080:00200000:1.0:1787827004.498904:0:87423:0:(file.c:1848:ll_io_set_mirror()) f41.replay-single: desiginated mirror: 0 00000080:00200000:1.0:1787827004.498911:0:87423:0:(vvp_io.c:1885:vvp_io_init()) [0x2000182d1:0x643:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787827004.498927:0:87423:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787827004.498950:0:87423:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [0, 18446744073709551615] 00000008:00010000:1.0:1787827004.498974:0:87423:0:(lustre_osc.h:166:osc_enq2ldlm_flags()) flags: 4 00000008:00010000:1.0:1787827004.498979:0:87423:0:(osc_lock.c:1320:osc_lock_init()) ### lock ffff985607c769b0, osc lock ffff985602f8f380, flags 0x80000000 00010000:00010000:1.0:1787827004.499022:0:87423:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-OST0000-osc-ffff985602bc9800 lock: 00000000c1d2050c/0xb3f83dad1b7ea380 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x9a2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 87423 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827004.499034:0:87423:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x80000000 ns: lustre-OST0000-osc-ffff985602bc9800 lock: 00000000c1d2050c/0xb3f83dad1b7ea380 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x9a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 87423 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827004.499045:0:87423:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-OST0000-osc-ffff985602bc9800 lock: 00000000c1d2050c/0xb3f83dad1b7ea380 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x9a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 87423 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787827004.499063:0:87423:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607ca6d00 pname:cluuid:pid:xid:nid:opc:job dd:7b4e8535-9db7-485d-abf6-bda966b63320:87423:1874669227359872:192.168.203.114@tcp:101:dd.0 00000100:00100000:1.0:1787827004.499147:0:87423:0:(client.c:2715:ptlrpc_set_wait()) set ffff9856055dc9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827004.500326:0:87423:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607ca6d00 pname:cluuid:pid:xid:nid:opc:job dd:7b4e8535-9db7-485d-abf6-bda966b63320:87423:1874669227359872:192.168.203.114@tcp:101:dd.0 00010000:00010000:1.0:1787827004.500350:0:87423:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0000-osc-ffff985602bc9800 lock: 00000000c1d2050c/0xb3f83dad1b7ea380 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x9a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000080000000 nid: local remote: 0x5aa6d504d91d0df1 expref: -99 pid: 87423 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827004.500360:0:87423:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-OST0000-osc-ffff985602bc9800 lock: 00000000c1d2050c/0xb3f83dad1b7ea380 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x9a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x80000000 nid: local remote: 0x5aa6d504d91d0df1 expref: -99 pid: 87423 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827004.500369:0:87423:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-OST0000-osc-ffff985602bc9800 lock: 00000000c1d2050c/0xb3f83dad1b7ea380 lrc: 5/0,2 mode: PW/PW res: [0x280000401:0x9a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000080000000 nid: local remote: 0x5aa6d504d91d0df1 expref: -99 pid: 87423 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827004.500377:0:87423:0:(osc_lock.c:192:osc_lock_lvb_update()) ### acquired size 0, setting rss=0; kms=0, end=18446744073709551615 ns: lustre-OST0000-osc-ffff985602bc9800 lock: 00000000c1d2050c/0xb3f83dad1b7ea380 lrc: 4/0,2 mode: PW/PW res: [0x280000401:0x9a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020080000000 nid: local remote: 0x5aa6d504d91d0df1 expref: -99 pid: 87423 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827004.500385:0:87423:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-OST0000-osc-ffff985602bc9800 lock: 00000000c1d2050c/0xb3f83dad1b7ea380 lrc: 5/0,2 mode: PW/PW res: [0x280000401:0x9a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020080000000 nid: local remote: 0x5aa6d504d91d0df1 expref: -99 pid: 87423 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827004.500390:0:87423:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-OST0000-osc-ffff985602bc9800 lock: 00000000c1d2050c/0xb3f83dad1b7ea380 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x9a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020080000000 nid: local remote: 0x5aa6d504d91d0df1 expref: -99 pid: 87423 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787827004.500449:0:2383:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1787827004.500461:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607ca6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669227360000:192.168.203.114@tcp:10:dd.0 00000100:00100000:1.0:1787827004.502479:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607ca6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669227360000:192.168.203.114@tcp:10:dd.0 00000080:00200000:3.0:1787827004.502535:0:87423:0:(file.c:1775:ll_merge_attr_nolock()) [0x2000182d1:0x643:0x0] updating i_size 0 i_blocks 0 00000020:00010000:3.0:1787827004.502542:0:87423:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff985605747128 (ffff985610fe1c68/0) at cl_lock_release():245 00000020:00010000:3.0:1787827004.502546:0:87423:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff985605747128 (ffff985610fe1c68/0) at cl_lock_cancel():142 00000020:00010000:3.0:1787827004.502549:0:87423:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff985607c769b0 (ffff985610fe1cf8/1) at cl_lock_cancel():142 00010000:00010000:3.0:1787827004.502557:0:87423:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-OST0000-osc-ffff985602bc9800 lock: 00000000c1d2050c/0xb3f83dad1b7ea380 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x9a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020080000000 nid: local remote: 0x5aa6d504d91d0df1 expref: -99 pid: 87423 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827004.502570:0:87423:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0000-osc-ffff985602bc9800 lock: 00000000c1d2050c/0xb3f83dad1b7ea380 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x9a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020080000000 nid: local remote: 0x5aa6d504d91d0df1 expref: -99 pid: 87423 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000020:00010000:3.0:1787827004.502584:0:87423:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff985605747128 (ffff985610fe1c68/0) at cl_lock_fini():72 00000020:00010000:3.0:1787827004.502587:0:87423:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff985607c769b0 (ffff985610fe1c68/1) at cl_lock_fini():72 00000080:00200000:3.0:1787827004.502601:0:87423:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x643:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200008:3.0:1787827004.502612:0:87423:0:(llite_lib.c:2686:ll_setattr_raw()) COMPLETED file f41.replay-single:[0x2000182d1:0x643:0x0](ffff9856102a5348) current size 0, valid attrs 1000a168, mode 0, uid 2, gid 0, new size 0, atime 1787827004.000000000, mtime 1787827004.000000000, ctime 1787827004.000000000, rc 0 00000080:00200008:3.0:1787827004.502725:0:87423:0:(file.c:2652:do_file_write_iter()) START file f41.replay-single:[0x2000182d1:0x643:0x0], ppos: 0, count: 4096 00000080:00200000:3.0:1787827004.502735:0:87423:0:(file.c:2090:ll_file_io_generic()) f41.replay-single: write ppos: 0, bytes: 4096 00000080:00200000:3.0:1787827004.502740:0:87423:0:(file.c:1848:ll_io_set_mirror()) f41.replay-single: desiginated mirror: 0 00000020:00200000:3.0:1787827004.502747:0:87423:0:(cl_io.c:218:cl_io_rw_init()) header@000000004f485c4c[0x110, 1, [0x2000182d1:0x643:0x0] hash] 00000020:00200000:3.0:1787827004.502749:0:87423:0:(cl_io.c:218:cl_io_rw_init()) io range: 2 [0, 4096) 0 0 00000080:00200000:3.0:1787827004.502753:0:87423:0:(vvp_io.c:1885:vvp_io_init()) [0x2000182d1:0x643:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1787827004.502841:0:87423:0:(file.c:2170:ll_file_io_generic()) Range lock [0, 0] 00020000:00200000:0.0:1787827004.502849:0:87423:0:(lov_io.c:1445:lov_io_rw_iter_init()) stripe: 0 chunk: [0, 4096) 4096, 4096 00020000:00200000:0.0:1787827004.502854:0:87423:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1787827004.502869:0:87423:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [0, 0] 00000008:00010000:0.0:1787827004.502880:0:87423:0:(lustre_osc.h:166:osc_enq2ldlm_flags()) flags: 0 00000008:00010000:0.0:1787827004.502885:0:87423:0:(osc_lock.c:1320:osc_lock_init()) ### lock ffff985607fa07b0, osc lock ffff985611951e78, flags 0x0 00010000:00010000:0.0:1787827004.502903:0:87423:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-OST0000-osc-ffff985602bc9800 lock: 00000000c1d2050c/0xb3f83dad1b7ea380 lrc: 2/0,1 mode: PW/PW res: [0x280000401:0x9a2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020080000000 nid: local remote: 0x5aa6d504d91d0df1 expref: -99 pid: 87423 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787827004.502914:0:87423:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 4095) ns: lustre-OST0000-osc-ffff985602bc9800 lock: 00000000c1d2050c/0xb3f83dad1b7ea380 lrc: 2/0,1 mode: PW/PW res: [0x280000401:0x9a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020080000000 nid: local remote: 0x5aa6d504d91d0df1 expref: -99 pid: 87423 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787827004.502926:0:87423:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-OST0000-osc-ffff985602bc9800 lock: 00000000c1d2050c/0xb3f83dad1b7ea380 lrc: 6/0,2 mode: PW/PW res: [0x280000401:0x9a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020080000000 nid: local remote: 0x5aa6d504d91d0df1 expref: -99 pid: 87423 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787827004.502935:0:87423:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-OST0000-osc-ffff985602bc9800 lock: 00000000c1d2050c/0xb3f83dad1b7ea380 lrc: 6/0,2 mode: PW/PW res: [0x280000401:0x9a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020080000000 nid: local remote: 0x5aa6d504d91d0df1 expref: -99 pid: 87423 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787827004.502943:0:87423:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-OST0000-osc-ffff985602bc9800 lock: 00000000c1d2050c/0xb3f83dad1b7ea380 lrc: 5/0,1 mode: PW/PW res: [0x280000401:0x9a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020080000000 nid: local remote: 0x5aa6d504d91d0df1 expref: -99 pid: 87423 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787827004.502969:0:87423:0:(vvp_io.c:1332:vvp_io_write_start()) f41.replay-single: write [0, 4096) append 0 append lockpos 0 00000080:00200000:0.0:1787827004.502982:0:87423:0:(rw26.c:681:ll_write_begin()) Writing 0 of 0 to 4096 bytes 00000080:00200000:0.0:1787827004.503014:0:87423:0:(rw26.c:875:ll_write_end()) pos 0, len 4096, copied 4096 00000080:00200000:0.0:1787827004.503021:0:87423:0:(rw26.c:933:ll_write_end()) page@00000000d9996880[3 000000008e5ad20d 2 1 000000001e78f4e9 0 0 0] 00000080:00200000:0.0:1787827004.503025:0:87423:0:(rw26.c:933:ll_write_end()) vmpage @0000000008e49fbc 17ffffc0001001 3:0 ffff98560558a488 0 lru 00000080:00200000:0.0:1787827004.503030:0:87423:0:(rw26.c:933:ll_write_end()) osc-page@000000005b7fefa3 0: 1< 0 - - > 2< 0 0 0 0x0 0x0 | 000000003c9253a3 00000000b78fc1fd > 3< 0 0 > 4< 0 0 8 484638720 - | - - - - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827004.503032:0:87423:0:(rw26.c:933:ll_write_end()) end page@00000000d9996880 00000080:00200000:0.0:1787827004.503033:0:87423:0:(rw26.c:933:ll_write_end()) queued cl_page: 1. 00000080:00200000:0.0:1787827004.503036:0:87423:0:(vvp_io.c:1214:vvp_io_write_commit()) commit async pages: 1, from 0, to 4096 prio 0 00000008:00010000:0.0:1787827004.503052:0:87423:0:(osc_cache.c:860:osc_extent_find()) ### extent: 00000000d15f490a ns: lustre-OST0000-osc-ffff985602bc9800 lock: 00000000c1d2050c/0xb3f83dad1b7ea380 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x9a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020080000000 nid: local remote: 0x5aa6d504d91d0df1 expref: -99 pid: 87423 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787827004.503066:0:87423:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9856102a54f8, count 1, dirtied 1 00000080:00200000:0.0:1787827004.503070:0:87423:0:(vvp_io.c:1237:vvp_io_write_commit()) Committed 1 pages 4096 bytes, tot: 4096 00000080:00200000:0.0:1787827004.503073:0:87423:0:(vvp_io.c:1398:vvp_io_write_start()) f41.replay-single: write bytes 0, result: 4096 00000020:00010000:0.0:1787827004.503085:0:87423:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff985605747128 (ffff985610fe1c68/0) at cl_lock_release():245 00000020:00010000:0.0:1787827004.503088:0:87423:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff985605747128 (ffff985610fe1c68/0) at cl_lock_cancel():142 00000020:00010000:0.0:1787827004.503090:0:87423:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff985607fa07b0 (ffff985610fe1488/1) at cl_lock_cancel():142 00010000:00010000:0.0:1787827004.503094:0:87423:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-OST0000-osc-ffff985602bc9800 lock: 00000000c1d2050c/0xb3f83dad1b7ea380 lrc: 5/0,1 mode: PW/PW res: [0x280000401:0x9a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020080000000 nid: local remote: 0x5aa6d504d91d0df1 expref: -99 pid: 87423 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787827004.503103:0:87423:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0000-osc-ffff985602bc9800 lock: 00000000c1d2050c/0xb3f83dad1b7ea380 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x9a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020080000000 nid: local remote: 0x5aa6d504d91d0df1 expref: -99 pid: 87423 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000020:00010000:0.0:1787827004.503113:0:87423:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff985605747128 (ffff985610fe1c68/0) at cl_lock_fini():72 00000020:00010000:0.0:1787827004.503116:0:87423:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff985607fa07b0 (ffff985610fe1c68/1) at cl_lock_fini():72 00000080:00200000:0.0:1787827004.503122:0:87423:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000080:00200000:0.0:1787827004.503125:0:87423:0:(file.c:2207:ll_file_io_generic()) Range unlock [0, 0] 00000080:00200000:0.0:1787827004.503129:0:87423:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x643:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787827004.503131:0:87423:0:(file.c:2240:ll_file_io_generic()) f41.replay-single: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:0.0:1787827004.503135:0:87423:0:(file.c:1775:ll_merge_attr_nolock()) [0x2000182d1:0x643:0x0] updating i_size 4096 i_blocks 0 00000080:00200000:0.0:1787827004.503139:0:87423:0:(file.c:2326:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:0.0:1787827004.503165:0:87423:0:(file.c:479:ll_file_release()) START file f41.replay-single:[0x2000182d1:0x643:0x0](ffff9856102a5348), flags 100001 00000002:00100000:0.0:1787827004.503238:0:87423:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff985610e812c0 x1874669227359360/t219043332501(219043332501) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787827020 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00000020:00100000:0.0:1787827004.503247:0:87423:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:0.0:1787827004.503251:0:87423:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985608535680 x1874669227360256/t0(0) o35->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:0.0:1787827004.503263:0:87423:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608535680 pname:cluuid:pid:xid:nid:opc:job dd:7b4e8535-9db7-485d-abf6-bda966b63320:87423:1874669227360256:192.168.203.114@tcp:35:dd.0 00000100:00100000:0.0:1787827004.503321:0:87423:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5b780 going to sleep for 11 seconds 00000100:00100000:0.0:1787827004.504903:0:87423:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608535680 pname:cluuid:pid:xid:nid:opc:job dd:7b4e8535-9db7-485d-abf6-bda966b63320:87423:1874669227360256:192.168.203.114@tcp:35:dd.0 00000002:00100000:0.0:1787827004.504926:0:87423:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff985610e812c0 x1874669227359360/t219043332501(219043332501) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787827020 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00010000:00010000:2.0:1787827004.514780:0:87426:0:(ldlm_resource.c:195:lru_size_store()) dropping all unused locks from namespace lustre-OST0000-osc-ffff985602bc9800 00010000:00010000:2.0:1787827004.514825:0:87426:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 8/8 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1787827004.514890:0:87427:0:(ldlm_resource.c:195:lru_size_store()) dropping all unused locks from namespace lustre-OST0001-osc-ffff985602bc9800 00010000:00010000:1.0:1787827004.514909:0:87427:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 4/12 locks to regular blp list, 2 blwis in pool 00010000:00010000:3.0:1787827004.514993:0:33341:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 8 locks of 4 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1787827004.515002:0:33341:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0000-osc-ffff985602bc9800 lock: 00000000a837b8ac/0xb3f83dad1b7df9b9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x442:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 7925760->7929855) gid 0 flags: 0x28400020000 nid: local remote: 0x5aa6d504d919c3eb expref: -99 pid: 57871 timeout: 0 lvb_type: 1 lru_score: 52 lru_type: 0 00010000:00010000:3.0:1787827004.515034:0:33341:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0000-osc-ffff985602bc9800 lock: 000000002b59de21/0xb3f83dad1b7df8c4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x402:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800028480000000 nid: local remote: 0x5aa6d504d919b8a7 expref: -99 pid: 54561 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000080:00200000:3.0:1787827004.515061:0:33341:0:(vvp_io.c:1885:vvp_io_init()) [0x200001b71:0x12e:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00010000:00010000:2.0:1787827004.515086:0:15163:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 4 locks of 0 total in blp. (0 blwis in pool) 00000080:00200000:3.0:1787827004.515111:0:33341:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b71:0x12e:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 1 00010000:00010000:3.0:1787827004.515127:0:33341:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0000-osc-ffff985602bc9800 lock: 000000000ce4cb39/0xb3f83dad1b7df4c6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x302:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800028480000000 nid: local remote: 0x5aa6d504d9198ad3 expref: -99 pid: 42689 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000080:00200000:3.0:1787827004.515145:0:33341:0:(vvp_io.c:1885:vvp_io_init()) [0x200001b71:0x115:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:3.0:1787827004.515165:0:33341:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b71:0x115:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 1 00010000:00010000:3.0:1787827004.515177:0:33341:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0000-osc-ffff985602bc9800 lock: 00000000fa88f276/0xb3f83dad1b7dd16b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x184:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800028400000000 nid: local remote: 0x5aa6d504d918c3d1 expref: -99 pid: 29090 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787827004.515187:0:15163:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0001-osc-ffff985602bc9800 lock: 0000000050203c77/0xb3f83dad1b7df505 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x302:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800028400000000 nid: local remote: 0x5aa6d504d9198bc8 expref: -99 pid: 42689 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000080:00200000:3.0:1787827004.515189:0:33341:0:(vvp_io.c:1885:vvp_io_init()) [0x200001b71:0x19:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:3.0:1787827004.515211:0:33341:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b71:0x19:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 1 00010000:00010000:3.0:1787827004.515222:0:33341:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0000-osc-ffff985602bc9800 lock: 00000000aad5fbf4/0xb3f83dad1b7dd141 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x183:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800028400000000 nid: local remote: 0x5aa6d504d918c361 expref: -99 pid: 29090 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787827004.515246:0:33341:0:(vvp_io.c:1885:vvp_io_init()) [0x200001b71:0x17:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:2.0:1787827004.515254:0:15163:0:(vvp_io.c:1885:vvp_io_init()) [0x200001b71:0x118:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:3.0:1787827004.515267:0:33341:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b71:0x17:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 1 00010000:00010000:3.0:1787827004.515281:0:33341:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0000-osc-ffff985602bc9800 lock: 0000000072190817/0xb3f83dad1b7dd117 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x182:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800028400000000 nid: local remote: 0x5aa6d504d918c2f1 expref: -99 pid: 29090 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787827004.515283:0:15163:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b71:0x118:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 1 00000080:00200000:3.0:1787827004.515301:0:33341:0:(vvp_io.c:1885:vvp_io_init()) [0x200001b71:0x15:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00010000:00010000:2.0:1787827004.515307:0:15163:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0001-osc-ffff985602bc9800 lock: 000000000b92e2ef/0xb3f83dad1b7dd180 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x184:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800028400000000 nid: local remote: 0x5aa6d504d918c409 expref: -99 pid: 29090 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787827004.515322:0:33341:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b71:0x15:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 1 00000080:00200000:2.0:1787827004.515327:0:15163:0:(vvp_io.c:1885:vvp_io_init()) [0x200001b71:0x1a:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00010000:00010000:3.0:1787827004.515336:0:33341:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0000-osc-ffff985602bc9800 lock: 00000000c1d2050c/0xb3f83dad1b7ea380 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x9a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800028480000000 nid: local remote: 0x5aa6d504d91d0df1 expref: -99 pid: 87423 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000008:00010000:3.0:1787827004.515356:0:33341:0:(osc_cache.c:2059:try_to_add_extent_for_io()) ### extent: 00000000d15f490a ns: lustre-OST0000-osc-ffff985602bc9800 lock: 00000000c1d2050c/0xb3f83dad1b7ea380 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x9a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800029480000000 nid: local remote: 0x5aa6d504d91d0df1 expref: -99 pid: 87423 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1787827004.515366:0:15163:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b71:0x1a:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 1 00000008:00010000:3.0:1787827004.515372:0:33341:0:(osc_cache.c:1176:osc_extent_make_ready()) ### extent: 00000000d15f490a ns: lustre-OST0000-osc-ffff985602bc9800 lock: 00000000c1d2050c/0xb3f83dad1b7ea380 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x9a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800029480000000 nid: local remote: 0x5aa6d504d91d0df1 expref: -99 pid: 87423 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1787827004.515379:0:15163:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0001-osc-ffff985602bc9800 lock: 00000000628fe1b6/0xb3f83dad1b7dd156 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x183:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800028400000000 nid: local remote: 0x5aa6d504d918c399 expref: -99 pid: 29090 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787827004.515392:0:15163:0:(vvp_io.c:1885:vvp_io_init()) [0x200001b71:0x18:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:2.0:1787827004.515407:0:15163:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b71:0x18:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 1 00010000:00010000:2.0:1787827004.515414:0:15163:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0001-osc-ffff985602bc9800 lock: 000000006ff65a4d/0xb3f83dad1b7dd12c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x182:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800028400000000 nid: local remote: 0x5aa6d504d918c329 expref: -99 pid: 29090 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787827004.515422:0:15163:0:(vvp_io.c:1885:vvp_io_init()) [0x200001b71:0x16:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00010000:00010000:3.0:1787827004.515424:0:33341:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 4095) ns: lustre-OST0000-osc-ffff985602bc9800 lock: 00000000c1d2050c/0xb3f83dad1b7ea380 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x9a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800029480000000 nid: local remote: 0x5aa6d504d91d0df1 expref: -99 pid: 87423 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1787827004.515434:0:15163:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b71:0x16:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 1 00010000:00010000:2.0:1787827004.515459:0:15163:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-OST0001-osc-ffff985602bc9800 lock: 0000000050203c77/0xb3f83dad1b7df505 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x302:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x804869400000020 nid: local remote: 0x5aa6d504d9198bc8 expref: -99 pid: 42689 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1787827004.515474:0:15163:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-OST0001-osc-ffff985602bc9800 lock: 000000000b92e2ef/0xb3f83dad1b7dd180 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x184:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x804869400000020 nid: local remote: 0x5aa6d504d918c409 expref: -99 pid: 29090 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787827004.515485:0:15163:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-OST0001-osc-ffff985602bc9800 lock: 00000000628fe1b6/0xb3f83dad1b7dd156 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x183:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x804869400000020 nid: local remote: 0x5aa6d504d918c399 expref: -99 pid: 29090 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787827004.515491:0:15163:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-OST0001-osc-ffff985602bc9800 lock: 000000006ff65a4d/0xb3f83dad1b7dd12c lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x182:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x804869400000020 nid: local remote: 0x5aa6d504d918c329 expref: -99 pid: 29090 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787827004.515497:0:15163:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 4 locks packed 00000008:00100000:3.0:1787827004.515502:0:33341:0:(osc_request.c:2124:osc_brw_prep_request()) brw rpc ffff985610e821c0 - object 0x280000401:2466 offset 0<>4096 00000100:00100000:2.0:1787827004.515519:0:15163:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ldlm_bl_01:7b4e8535-9db7-485d-abf6-bda966b63320:15163:1874669227360384:192.168.203.114@tcp:103:ldlm_bl.0 00000100:00100000:3.0:1787827004.515552:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e821c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669227360512:192.168.203.114@tcp:4:dd.0 00000100:00100000:2.0:1787827004.515587:0:15163:0:(client.c:2715:ptlrpc_set_wait()) set ffff98560674c9c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787827004.517412:0:15163:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ldlm_bl_01:7b4e8535-9db7-485d-abf6-bda966b63320:15163:1874669227360384:192.168.203.114@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787827004.517434:0:15163:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0001-osc-ffff985602bc9800 lock: 0000000050203c77/0xb3f83dad1b7df505 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x302:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x804869400000020 nid: local remote: 0x5aa6d504d9198bc8 expref: -99 pid: 42689 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787827004.517454:0:15163:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0001-osc-ffff985602bc9800 lock: 000000000b92e2ef/0xb3f83dad1b7dd180 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x184:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x804869400000020 nid: local remote: 0x5aa6d504d918c409 expref: -99 pid: 29090 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787827004.517466:0:15163:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0001-osc-ffff985602bc9800 lock: 00000000628fe1b6/0xb3f83dad1b7dd156 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x183:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x804869400000020 nid: local remote: 0x5aa6d504d918c399 expref: -99 pid: 29090 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787827004.517478:0:15163:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0001-osc-ffff985602bc9800 lock: 000000006ff65a4d/0xb3f83dad1b7dd12c lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x182:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x804869400000020 nid: local remote: 0x5aa6d504d918c329 expref: -99 pid: 29090 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787827004.517510:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827004.517512:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827004.517514:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000008:00010000:3.0:1787827004.519190:0:2386:0:(osc_cache.c:910:osc_extent_finish()) ### extent: 00000000d15f490a ns: lustre-OST0000-osc-ffff985602bc9800 lock: 00000000c1d2050c/0xb3f83dad1b7ea380 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x9a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800029480000000 nid: local remote: 0x5aa6d504d91d0df1 expref: -99 pid: 87423 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000008:00010000:3.0:1787827004.519237:0:2386:0:(osc_cache.c:498:__osc_extent_remove()) ### extent: 00000000d15f490a ns: lustre-OST0000-osc-ffff985602bc9800 lock: 00000000c1d2050c/0xb3f83dad1b7ea380 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x9a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800029480000000 nid: local remote: 0x5aa6d504d91d0df1 expref: -99 pid: 87423 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000100:00100000:3.0:1787827004.519257:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e821c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669227360512:192.168.203.114@tcp:4:dd.0 00000080:00200000:1.0:1787827004.519288:0:33341:0:(vvp_io.c:1885:vvp_io_init()) [0x2000182d1:0x643:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787827004.519331:0:33341:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x643:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 1 00010000:00010000:1.0:1787827004.519354:0:33341:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0000-osc-ffff985602bc9800 lock: 00000000a906a3ef/0xb3f83dad1b7df9f1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x462:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800028400000000 nid: local remote: 0x5aa6d504d919c836 expref: -99 pid: 59807 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787827004.519443:0:33341:0:(vvp_io.c:1885:vvp_io_init()) [0x20000fe01:0x1:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787827004.519468:0:33341:0:(vvp_io.c:304:vvp_io_fini()) [0x20000fe01:0x1:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 1 00010000:00010000:1.0:1787827004.519500:0:33341:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-OST0000-osc-ffff985602bc9800 lock: 00000000a837b8ac/0xb3f83dad1b7df9b9 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x442:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 7925760->7929855) gid 0 flags: 0x4829400020020 nid: local remote: 0x5aa6d504d919c3eb expref: -99 pid: 57871 timeout: 0 lvb_type: 1 lru_score: 52 lru_type: 0 00010000:00010000:1.0:1787827004.519510:0:33341:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-OST0000-osc-ffff985602bc9800 lock: 000000002b59de21/0xb3f83dad1b7df8c4 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x402:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804869480000020 nid: local remote: 0x5aa6d504d919b8a7 expref: -99 pid: 54561 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787827004.519519:0:33341:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-OST0000-osc-ffff985602bc9800 lock: 000000000ce4cb39/0xb3f83dad1b7df4c6 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x302:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804869480000020 nid: local remote: 0x5aa6d504d9198ad3 expref: -99 pid: 42689 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787827004.519526:0:33341:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-OST0000-osc-ffff985602bc9800 lock: 00000000fa88f276/0xb3f83dad1b7dd16b lrc: 1/0,0 mode: --/PW res: [0x280000401:0x184:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x804869400000020 nid: local remote: 0x5aa6d504d918c3d1 expref: -99 pid: 29090 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827004.519534:0:33341:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-OST0000-osc-ffff985602bc9800 lock: 00000000aad5fbf4/0xb3f83dad1b7dd141 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x183:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x804869400000020 nid: local remote: 0x5aa6d504d918c361 expref: -99 pid: 29090 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827004.519541:0:33341:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-OST0000-osc-ffff985602bc9800 lock: 0000000072190817/0xb3f83dad1b7dd117 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x182:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x804869400000020 nid: local remote: 0x5aa6d504d918c2f1 expref: -99 pid: 29090 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827004.519549:0:33341:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-OST0000-osc-ffff985602bc9800 lock: 00000000c1d2050c/0xb3f83dad1b7ea380 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x9a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804869480000020 nid: local remote: 0x5aa6d504d91d0df1 expref: -99 pid: 87423 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787827004.519557:0:33341:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-OST0000-osc-ffff985602bc9800 lock: 00000000a906a3ef/0xb3f83dad1b7df9f1 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x462:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x804869400000020 nid: local remote: 0x5aa6d504d919c836 expref: -99 pid: 59807 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827004.519564:0:33341:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 8 locks packed 00000100:00100000:1.0:1787827004.519581:0:33341:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607ca70c0 pname:cluuid:pid:xid:nid:opc:job ldlm_bl_03:7b4e8535-9db7-485d-abf6-bda966b63320:33341:1874669227360640:192.168.203.114@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1787827004.519648:0:33341:0:(client.c:2715:ptlrpc_set_wait()) set ffff9856055dc9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827004.523054:0:33341:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607ca70c0 pname:cluuid:pid:xid:nid:opc:job ldlm_bl_03:7b4e8535-9db7-485d-abf6-bda966b63320:33341:1874669227360640:192.168.203.114@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787827004.523083:0:33341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0000-osc-ffff985602bc9800 lock: 00000000a837b8ac/0xb3f83dad1b7df9b9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x442:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 7925760->7929855) gid 0 flags: 0x4829400020020 nid: local remote: 0x5aa6d504d919c3eb expref: -99 pid: 57871 timeout: 0 lvb_type: 1 lru_score: 52 lru_type: 0 00010000:00010000:1.0:1787827004.523107:0:33341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0000-osc-ffff985602bc9800 lock: 000000002b59de21/0xb3f83dad1b7df8c4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x402:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804869480000020 nid: local remote: 0x5aa6d504d919b8a7 expref: -99 pid: 54561 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787827004.523120:0:33341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0000-osc-ffff985602bc9800 lock: 000000000ce4cb39/0xb3f83dad1b7df4c6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x302:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804869480000020 nid: local remote: 0x5aa6d504d9198ad3 expref: -99 pid: 42689 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787827004.523132:0:33341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0000-osc-ffff985602bc9800 lock: 00000000fa88f276/0xb3f83dad1b7dd16b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x184:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x804869400000020 nid: local remote: 0x5aa6d504d918c3d1 expref: -99 pid: 29090 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827004.523144:0:33341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0000-osc-ffff985602bc9800 lock: 00000000aad5fbf4/0xb3f83dad1b7dd141 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x183:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x804869400000020 nid: local remote: 0x5aa6d504d918c361 expref: -99 pid: 29090 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827004.523155:0:33341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0000-osc-ffff985602bc9800 lock: 0000000072190817/0xb3f83dad1b7dd117 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x182:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x804869400000020 nid: local remote: 0x5aa6d504d918c2f1 expref: -99 pid: 29090 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827004.523168:0:33341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0000-osc-ffff985602bc9800 lock: 00000000c1d2050c/0xb3f83dad1b7ea380 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x9a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804869480000020 nid: local remote: 0x5aa6d504d91d0df1 expref: -99 pid: 87423 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787827004.523186:0:33341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0000-osc-ffff985602bc9800 lock: 00000000a906a3ef/0xb3f83dad1b7df9f1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x462:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x804869400000020 nid: local remote: 0x5aa6d504d919c836 expref: -99 pid: 59807 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827004.523206:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827004.523208:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827004.523210:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827005.236920:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200000:2.0:1787827005.300388:0:87460:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:2.0:1787827005.300418:0:87460:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827005.300474:0:87460:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8e00/0xb3f83dad1b7ea387 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 87460 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827005.300482:0:87460:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8e00/0xb3f83dad1b7ea387 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 87460 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827005.300490:0:87460:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8e00/0xb3f83dad1b7ea387 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 87460 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787827005.300651:0:87460:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job dd:7b4e8535-9db7-485d-abf6-bda966b63320:87460:1874669227360768:192.168.203.114@tcp:101:dd.0 00000100:00100000:2.0:1787827005.300923:0:87460:0:(client.c:2715:ptlrpc_set_wait()) set ffff98560674c300 going to sleep for 11 seconds 00000100:00100000:3.0:1787827005.303171:0:87460:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job dd:7b4e8535-9db7-485d-abf6-bda966b63320:87460:1874669227360768:192.168.203.114@tcp:101:dd.0 00010000:00010000:3.0:1787827005.303196:0:87460:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8e00/0xb3f83dad1b7ea387 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91d0dff expref: -99 pid: 87460 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827005.303207:0:87460:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8e00/0xb3f83dad1b7ea387 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91d0dff expref: -99 pid: 87460 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827005.303218:0:87460:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8e00/0xb3f83dad1b7ea387 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91d0dff expref: -99 pid: 87460 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827005.303228:0:87460:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8e00/0xb3f83dad1b7ea387 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91d0dff expref: -99 pid: 87460 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827005.303239:0:87460:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8e00/0xb3f83dad1b7ea387 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91d0dff expref: -99 pid: 87460 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827005.303246:0:87460:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8e00/0xb3f83dad1b7ea387 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91d0dff expref: -99 pid: 87460 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827005.303253:0:87460:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8e00/0xb3f83dad1b7ea387 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91d0dff expref: -99 pid: 87460 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787827005.303268:0:87460:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9856140d83c0 x1874669227360768/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 576/688 e 0 to 0 dl 1787827021 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787827005.303281:0:87460:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8e00/0xb3f83dad1b7ea387 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91d0dff expref: -99 pid: 87460 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827005.303292:0:87460:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:3.0:1787827005.303299:0:87460:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:3.0:1787827005.303322:0:87460:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 69632 00000080:00010000:3.0:1787827005.303327:0:87460:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ea387 00000080:00010000:3.0:1787827005.303332:0:87460:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:3.0:1787827005.303334:0:87460:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ea387 from it ffffab3ac1c77bf8 00010000:00010000:3.0:1787827005.303337:0:87460:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8e00/0xb3f83dad1b7ea387 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91d0dff expref: -99 pid: 87460 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827005.303345:0:87460:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8e00/0xb3f83dad1b7ea387 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91d0dff expref: -99 pid: 87460 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787827005.303377:0:87460:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:3.0:1787827005.303394:0:87460:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f41.replay-single, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), file ffff985602b82f00, open_flags 8000, mode 0 opened 0 00000080:00200000:3.0:1787827005.303400:0:87460:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f41.replay-single, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=open 00000002:00010000:3.0:1787827005.303420:0:87460:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f41.replay-single,[0x0:0x0:0x0]) in obj [0x200000007:0x1:0x0], intent: open flags 0100001 00010000:00010000:3.0:1787827005.303505:0:87460:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06000/0xb3f83dad1b7ea38e lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 87460 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827005.303512:0:87460:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06000/0xb3f83dad1b7ea38e lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 87460 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:3.0:1787827005.303525:0:87460:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:3.0:1787827005.303532:0:87460:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610e83840 x1874669227361024/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/131856 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:3.0:1787827005.303539:0:87460:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06000/0xb3f83dad1b7ea38e lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 87460 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787827005.303555:0:87460:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e83840 pname:cluuid:pid:xid:nid:opc:job dd:7b4e8535-9db7-485d-abf6-bda966b63320:87460:1874669227361024:192.168.203.114@tcp:101:dd.0 00000100:00100000:3.0:1787827005.303722:0:87460:0:(client.c:2715:ptlrpc_set_wait()) set ffff985605721cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1787827005.306080:0:87460:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e83840 pname:cluuid:pid:xid:nid:opc:job dd:7b4e8535-9db7-485d-abf6-bda966b63320:87460:1874669227361024:192.168.203.114@tcp:101:dd.0 00010000:00010000:3.0:1787827005.306097:0:87460:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06000/0xb3f83dad1b7ea38e lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91d0e1b expref: -99 pid: 87460 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827005.306109:0:87460:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06000/0xb3f83dad1b7ea38e lrc: 4/1,0 mode: --/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91d0e1b expref: -99 pid: 87460 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827005.306114:0:87460:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06000/0xb3f83dad1b7ea38e lrc: 4/1,0 mode: --/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91d0e1b expref: -99 pid: 87460 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827005.306121:0:87460:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06000/0xb3f83dad1b7ea38e lrc: 4/1,0 mode: PR/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91d0e1b expref: -99 pid: 87460 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827005.306128:0:87460:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06000/0xb3f83dad1b7ea38e lrc: 4/1,0 mode: PR/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91d0e1b expref: -99 pid: 87460 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827005.306139:0:87460:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06000/0xb3f83dad1b7ea38e lrc: 5/2,0 mode: PR/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91d0e1b expref: -99 pid: 87460 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827005.306151:0:87460:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06000/0xb3f83dad1b7ea38e lrc: 5/2,0 mode: PR/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91d0e1b expref: -99 pid: 87460 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827005.306159:0:87460:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06000/0xb3f83dad1b7ea38e lrc: 4/1,0 mode: PR/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91d0e1b expref: -99 pid: 87460 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787827005.306170:0:87460:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x1 disposition=0x2b, status=0 req@ffff985610e83840 x1874669227361024/t219043332505(219043332505) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/664 e 0 to 0 dl 1787827021 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00000002:00100000:3.0:1787827005.306182:0:87460:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff985610e83840 x1874669227361024/t219043332505(219043332505) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 664/664 e 0 to 0 dl 1787827021 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787827005.306194:0:87460:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06000/0xb3f83dad1b7ea38e lrc: 3/1,0 mode: PR/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91d0e1b expref: -99 pid: 87460 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:3.0:1787827005.306207:0:87460:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06000/0xb3f83dad1b7ea38e lrc: 3/1,0 mode: PR/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91d0e1b expref: -99 pid: 87460 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827005.306214:0:87460:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8590033617/1603 (0 0) 00000080:00200000:3.0:1787827005.306219:0:87460:0:(namei.c:952:ll_intent_lock()) intent lock 1 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:3.0:1787827005.306252:0:87460:0:(namei.c:147:ll_iget()) got inode: [0x2000182d1:0x643:0x0](ffff9856102a5348): rc = 0 00000080:00200000:3.0:1787827005.306260:0:87460:0:(file.c:6854:ll_layout_conf()) [0x2000182d1:0x643:0x0]: layout version change: 0 -> 0 00000080:00010000:3.0:1787827005.306264:0:87460:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x2000182d1:0x643:0x0](ffff9856102a5348) for lock 0xb3f83dad1b7ea38e 00000080:00010000:3.0:1787827005.306271:0:87460:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x2000182d1:0x643:0x0](ffff9856102a5348) 00000080:00200008:3.0:1787827005.306281:0:87460:0:(file.c:1090:ll_file_open()) START file f41.replay-single:[0x2000182d1:0x643:0x0](ffff9856102a5348), flags 100000 00000002:00100000:3.0:1787827005.306295:0:87460:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff985610e83840 x1874669227361024/t219043332505(219043332505) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787827021 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00000080:00010000:3.0:1787827005.306332:0:87460:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ea38e from it ffff985610c4aea0 00010000:00010000:3.0:1787827005.306335:0:87460:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06000/0xb3f83dad1b7ea38e lrc: 3/1,0 mode: PR/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91d0e1b expref: -99 pid: 87460 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827005.306341:0:87460:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06000/0xb3f83dad1b7ea38e lrc: 2/0,0 mode: PR/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91d0e1b expref: -99 pid: 87460 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787827005.306418:0:87460:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x2000182d1:0x643:0x0](ffff9856102a5348), inode mode 81a4 mask 44 00000080:00200008:3.0:1787827005.306447:0:87460:0:(file.c:5224:ll_file_seek()) START file f41.replay-single:[0x2000182d1:0x643:0x0], offset: 0, type: SEEK_CUR 00000080:00200008:3.0:1787827005.306450:0:87460:0:(file.c:5262:ll_file_seek()) COMPLETED file f41.replay-single:[0x2000182d1:0x643:0x0], offset: 0, type: SEEK_CUR, rc = 0 00000080:00200008:3.0:1787827005.306521:0:87460:0:(file.c:2485:do_file_read_iter()) START file f41.replay-single:[0x2000182d1:0x643:0x0], ppos: 0, count: 4096 00000080:00200000:3.0:1787827005.306540:0:87460:0:(rw.c:1947:ll_readpage()) fast read pgno: 0 00000080:00200000:3.0:1787827005.306551:0:87460:0:(file.c:2090:ll_file_io_generic()) f41.replay-single: read ppos: 0, bytes: 4096 00000080:00200000:3.0:1787827005.306555:0:87460:0:(file.c:1848:ll_io_set_mirror()) f41.replay-single: desiginated mirror: 0 00000020:00200000:3.0:1787827005.306562:0:87460:0:(cl_io.c:218:cl_io_rw_init()) header@000000004f485c4c[0x110, 1, [0x2000182d1:0x643:0x0] hash] 00000020:00200000:3.0:1787827005.306565:0:87460:0:(cl_io.c:218:cl_io_rw_init()) io range: 1 [0, 4096) 0 0 00000080:00200000:3.0:1787827005.306570:0:87460:0:(vvp_io.c:1885:vvp_io_init()) [0x2000182d1:0x643:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787827005.306586:0:87460:0:(lov_io.c:1445:lov_io_rw_iter_init()) stripe: 0 chunk: [0, 4096) 4096, 4096 00020000:00200000:3.0:1787827005.306591:0:87460:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787827005.306608:0:87460:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 0 [0, 0] 00000008:00010000:3.0:1787827005.306622:0:87460:0:(lustre_osc.h:166:osc_enq2ldlm_flags()) flags: 0 00000008:00010000:3.0:1787827005.306626:0:87460:0:(osc_lock.c:1320:osc_lock_init()) ### lock ffff9856119e08b0, osc lock ffff98560897f5f0, flags 0x0 00010000:00010000:3.0:1787827005.306666:0:87460:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0000-osc-ffff985602bc9800 lock: 0000000021ff82c6/0xb3f83dad1b7ea395 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x9a2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 87460 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827005.306676:0:87460:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x0 ns: lustre-OST0000-osc-ffff985602bc9800 lock: 0000000021ff82c6/0xb3f83dad1b7ea395 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x9a2:0x0].0x0 rrc: 2 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 87460 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827005.306685:0:87460:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-OST0000-osc-ffff985602bc9800 lock: 0000000021ff82c6/0xb3f83dad1b7ea395 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x9a2:0x0].0x0 rrc: 2 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 87460 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787827005.307087:0:87460:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e81a40 pname:cluuid:pid:xid:nid:opc:job dd:7b4e8535-9db7-485d-abf6-bda966b63320:87460:1874669227361152:192.168.203.114@tcp:101:dd.0 00000100:00100000:3.0:1787827005.307153:0:87460:0:(client.c:2715:ptlrpc_set_wait()) set ffff985605721cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1787827005.310020:0:87460:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e81a40 pname:cluuid:pid:xid:nid:opc:job dd:7b4e8535-9db7-485d-abf6-bda966b63320:87460:1874669227361152:192.168.203.114@tcp:101:dd.0 00010000:00010000:3.0:1787827005.310049:0:87460:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-OST0000-osc-ffff985602bc9800 lock: 0000000021ff82c6/0xb3f83dad1b7ea395 lrc: 4/1,0 mode: --/PR res: [0x280000401:0x9a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91d0e29 expref: -99 pid: 87460 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827005.310066:0:87460:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0000-osc-ffff985602bc9800 lock: 0000000021ff82c6/0xb3f83dad1b7ea395 lrc: 4/1,0 mode: PR/PR res: [0x280000401:0x9a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91d0e29 expref: -99 pid: 87460 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827005.310077:0:87460:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-OST0000-osc-ffff985602bc9800 lock: 0000000021ff82c6/0xb3f83dad1b7ea395 lrc: 4/1,0 mode: PR/PR res: [0x280000401:0x9a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91d0e29 expref: -99 pid: 87460 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827005.310090:0:87460:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0000-osc-ffff985602bc9800 lock: 0000000021ff82c6/0xb3f83dad1b7ea395 lrc: 5/2,0 mode: PR/PR res: [0x280000401:0x9a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91d0e29 expref: -99 pid: 87460 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:3.0:1787827005.310102:0:87460:0:(osc_lock.c:192:osc_lock_lvb_update()) ### acquired size 4096, setting rss=4096; kms=4096, end=18446744073709551615 ns: lustre-OST0000-osc-ffff985602bc9800 lock: 0000000021ff82c6/0xb3f83dad1b7ea395 lrc: 4/2,0 mode: PR/PR res: [0x280000401:0x9a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91d0e29 expref: -99 pid: 87460 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827005.310116:0:87460:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff985602bc9800 lock: 0000000021ff82c6/0xb3f83dad1b7ea395 lrc: 5/2,0 mode: PR/PR res: [0x280000401:0x9a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x810020000000000 nid: local remote: 0x5aa6d504d91d0e29 expref: -99 pid: 87460 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827005.310123:0:87460:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-OST0000-osc-ffff985602bc9800 lock: 0000000021ff82c6/0xb3f83dad1b7ea395 lrc: 4/1,0 mode: PR/PR res: [0x280000401:0x9a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x810020000000000 nid: local remote: 0x5aa6d504d91d0e29 expref: -99 pid: 87460 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:3.0:1787827005.310150:0:87460:0:(vvp_io.c:874:vvp_io_read_start()) f41.replay-single: read [0, 4096) 00000080:00200000:3.0:1787827005.310169:0:87460:0:(vvp_io.c:194:vvp_prep_size()) [0x2000182d1:0x643:0x0] updating i_size 4096 00000080:00200000:3.0:1787827005.310174:0:87460:0:(vvp_io.c:932:vvp_io_read_start()) count:4096 iocb pos:0 00000008:00010000:3.0:1787827005.310218:0:87460:0:(osc_cache.c:3109:osc_queue_sync_pages()) ### no dlm lock: extent: ffff98560511aa80 00000008:00010000:3.0:1787827005.310250:0:50252:0:(osc_cache.c:2059:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff98560511aa80 00010000:00010000:3.0:1787827005.310277:0:50252:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 4095) ns: lustre-OST0000-osc-ffff985602bc9800 lock: 0000000021ff82c6/0xb3f83dad1b7ea395 lrc: 4/1,0 mode: PR/PR res: [0x280000401:0x9a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000000000 nid: local remote: 0x5aa6d504d91d0e29 expref: -99 pid: 87460 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00100000:3.0:1787827005.310309:0:50252:0:(osc_request.c:2124:osc_brw_prep_request()) brw rpc ffff985610e81a40 - object 0x280000401:2466 offset 0<>4096 00000100:00100000:3.0:1787827005.310351:0:2386:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1787827005.310499:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610e81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669227361280:192.168.203.114@tcp:3:dd.0 00000008:00010000:3.0:1787827005.312326:0:2386:0:(osc_cache.c:910:osc_extent_finish()) ### no dlm lock: extent: ffff98560511aa80 00000020:00200000:3.0:1787827005.312337:0:2386:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:3.0:1787827005.312470:0:2386:0:(osc_cache.c:498:__osc_extent_remove()) ### no dlm lock: extent: ffff98560511aa80 00000100:00100000:3.0:1787827005.312494:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610e81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669227361280:192.168.203.114@tcp:3:dd.0 00000020:00010000:2.0:1787827005.312664:0:87460:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff985610d29998 (ffff9856420e7908/0) at cl_lock_release():245 00000020:00010000:2.0:1787827005.312669:0:87460:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff985610d29998 (ffff9856420e7908/0) at cl_lock_cancel():142 00000020:00010000:2.0:1787827005.312672:0:87460:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9856119e08b0 (ffff985610fe1908/1) at cl_lock_cancel():142 00010000:00010000:2.0:1787827005.312682:0:87460:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff985602bc9800 lock: 0000000021ff82c6/0xb3f83dad1b7ea395 lrc: 4/1,0 mode: PR/PR res: [0x280000401:0x9a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x810020000000000 nid: local remote: 0x5aa6d504d91d0e29 expref: -99 pid: 87460 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827005.312767:0:87460:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0000-osc-ffff985602bc9800 lock: 0000000021ff82c6/0xb3f83dad1b7ea395 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x9a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000000000 nid: local remote: 0x5aa6d504d91d0e29 expref: -99 pid: 87460 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000020:00010000:2.0:1787827005.312844:0:87460:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff985610d29998 (ffff9856420e7908/0) at cl_lock_fini():72 00000020:00010000:2.0:1787827005.312848:0:87460:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9856119e08b0 (ffff9856420e7908/1) at cl_lock_fini():72 00000080:00200000:2.0:1787827005.312857:0:87460:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000080:00200000:2.0:1787827005.312887:0:87460:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x643:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827005.312891:0:87460:0:(file.c:2240:ll_file_io_generic()) f41.replay-single: 1 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1787827005.312896:0:87460:0:(file.c:2326:ll_file_io_generic()) iot: 1, result: 4096 00000080:00200008:2.0:1787827005.312934:0:87460:0:(file.c:479:ll_file_release()) START file f41.replay-single:[0x2000182d1:0x643:0x0](ffff9856102a5348), flags 100000 00000002:00100000:2.0:1787827005.313001:0:87460:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff985610e83840 x1874669227361024/t219043332505(219043332505) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787827021 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00000020:00100000:2.0:1787827005.313012:0:87460:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:2.0:1787827005.313018:0:87460:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9856140d8b40 x1874669227361536/t0(0) o35->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:2.0:1787827005.313039:0:87460:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job dd:7b4e8535-9db7-485d-abf6-bda966b63320:87460:1874669227361536:192.168.203.114@tcp:35:dd.0 00000100:00100000:2.0:1787827005.313141:0:87460:0:(client.c:2715:ptlrpc_set_wait()) set ffff98560674c3c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787827005.314582:0:87460:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job dd:7b4e8535-9db7-485d-abf6-bda966b63320:87460:1874669227361536:192.168.203.114@tcp:35:dd.0 00000002:00100000:0.0:1787827005.314608:0:87460:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff985610e83840 x1874669227361024/t219043332505(219043332505) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/664 e 0 to 0 dl 1787827021 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00010000:00010000:0.0:1787827006.260932:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827006.708848:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607ca4f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669227361792:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787827006.708934:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607ca5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2383:1874669227361664:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827006.708958:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607ca6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669227362048:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787827006.708986:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607ca61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669227361920:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787827006.709013:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607ca4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669227362176:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827006.712297:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607ca4f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669227361792:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787827006.712299:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607ca5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2383:1874669227361664:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827006.712326:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607ca6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669227362048:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787827006.712329:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607ca61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669227361920:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787827006.712340:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607ca4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669227362176:192.168.203.114@tcp:400:kworker.0 00010000:00010000:0.0:1787827007.284852:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:3.0:1787827008.227165:0:88031:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == replay-single test 42: recovery after ost failure ===== 06:36:47 (1787827007) 00000080:00200008:2.0:1787827008.249986:0:88035:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:2.0:1787827008.249996:0:88035:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:2.0:1787827008.250015:0:88035:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827008.250032:0:88035:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8e00/0xb3f83dad1b7ea387 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91d0dff expref: -99 pid: 87460 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787827008.250048:0:88035:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8e00/0xb3f83dad1b7ea387 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91d0dff expref: -99 pid: 87460 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787827008.250057:0:88035:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827008.250064:0:88035:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:2.0:1787827008.250068:0:88035:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ea387 00000080:00010000:2.0:1787827008.250075:0:88035:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ea387 from it ffffab3ac0dd3cb8 00010000:00010000:2.0:1787827008.250079:0:88035:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8e00/0xb3f83dad1b7ea387 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91d0dff expref: -99 pid: 87460 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787827008.250088:0:88035:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8e00/0xb3f83dad1b7ea387 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91d0dff expref: -99 pid: 87460 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1787827008.250119:0:88035:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:2.0:1787827008.250127:0:88035:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827008.250134:0:88035:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8e00/0xb3f83dad1b7ea387 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91d0dff expref: -99 pid: 87460 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1787827008.250142:0:88035:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8e00/0xb3f83dad1b7ea387 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91d0dff expref: -99 pid: 87460 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1787827008.250148:0:88035:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827008.250152:0:88035:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:2.0:1787827008.250155:0:88035:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ea387 00000080:00010000:2.0:1787827008.250159:0:88035:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ea387 from it ffffab3ac0dd3bb0 00010000:00010000:2.0:1787827008.250162:0:88035:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8e00/0xb3f83dad1b7ea387 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91d0dff expref: -99 pid: 87460 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1787827008.250171:0:88035:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8e00/0xb3f83dad1b7ea387 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91d0dff expref: -99 pid: 87460 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:2.0:1787827008.250179:0:88035:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 44 00000080:00200008:2.0:1787827008.250184:0:88035:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 100400 00000080:00200008:2.0:1787827008.250196:0:88035:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 100000 00000080:00200008:2.0:1787827008.250863:0:88035:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:2.0:1787827008.250868:0:88035:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:2.0:1787827008.250877:0:88035:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827008.250884:0:88035:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8e00/0xb3f83dad1b7ea387 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91d0dff expref: -99 pid: 87460 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:2.0:1787827008.250891:0:88035:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8e00/0xb3f83dad1b7ea387 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91d0dff expref: -99 pid: 87460 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:2.0:1787827008.250895:0:88035:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827008.250900:0:88035:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:2.0:1787827008.250902:0:88035:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ea387 00000080:00010000:2.0:1787827008.250905:0:88035:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ea387 from it ffffab3ac0dd3cb8 00010000:00010000:2.0:1787827008.250908:0:88035:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8e00/0xb3f83dad1b7ea387 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91d0dff expref: -99 pid: 87460 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:2.0:1787827008.250914:0:88035:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8e00/0xb3f83dad1b7ea387 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91d0dff expref: -99 pid: 87460 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00000080:00200008:2.0:1787827008.251135:0:88035:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:2.0:1787827008.251141:0:88035:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:2.0:1787827008.251149:0:88035:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827008.251155:0:88035:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8e00/0xb3f83dad1b7ea387 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91d0dff expref: -99 pid: 87460 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00010000:00010000:2.0:1787827008.251161:0:88035:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8e00/0xb3f83dad1b7ea387 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91d0dff expref: -99 pid: 87460 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00000080:00200000:2.0:1787827008.251169:0:88035:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827008.251173:0:88035:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:2.0:1787827008.251175:0:88035:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ea387 00000080:00010000:2.0:1787827008.251178:0:88035:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ea387 from it ffffab3ac0dd3cb8 00010000:00010000:2.0:1787827008.251180:0:88035:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8e00/0xb3f83dad1b7ea387 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91d0dff expref: -99 pid: 87460 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00010000:00010000:2.0:1787827008.251185:0:88035:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da8e00/0xb3f83dad1b7ea387 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91d0dff expref: -99 pid: 87460 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00000080:00200000:2.0:1787827008.251203:0:88035:0:(llite_lib.c:2903:ll_statfs()) VFS Op:sb=lustre (ffff985602bc9800) / proj=0 00000080:00000004:2.0:1787827008.251207:0:88035:0:(obd_class.h:1091:obd_statfs()) lustre-clilmv-ffff985602bc9800: age 2605, max_age 3158 00800000:00000004:2.0:1787827008.251215:0:88035:0:(obd_class.h:1091:obd_statfs()) lustre-MDT0000-mdc-ffff985602bc9800: age 3146, max_age 3158 00000100:00100000:2.0:1787827008.251246:0:88035:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db840 pname:cluuid:pid:xid:nid:opc:job df:7b4e8535-9db7-485d-abf6-bda966b63320:88035:1874669227362304:192.168.203.114@tcp:41:df.0 00000100:00100000:2.0:1787827008.251421:0:88035:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607ff0300 going to sleep for 11 seconds 00000100:00100000:3.0:1787827008.253016:0:88035:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 219043332506 gen 9 00000100:00100000:3.0:1787827008.253089:0:88035:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db840 pname:cluuid:pid:xid:nid:opc:job df:7b4e8535-9db7-485d-abf6-bda966b63320:88035:1874669227362304:192.168.203.114@tcp:41:df.0 00800000:00000004:3.0:1787827008.253118:0:88035:0:(obd_class.h:1117:obd_statfs()) lustre-MDT0000-mdc-ffff985602bc9800: update ffff985606a337f0 cache blocks 1802712/1916558 objects 523405/523980 00000080:00000004:3.0:1787827008.253124:0:88035:0:(obd_class.h:1117:obd_statfs()) lustre-clilmv-ffff985602bc9800: update ffff985606a31e20 cache blocks 1802712/1916558 objects 523405/523980 00000080:00000004:3.0:1787827008.253128:0:88035:0:(llite_lib.c:2755:ll_statfs_internal()) MDC blocks 1802712/1916558 objects 523405/523980 00010000:00010000:3.0:1787827008.308888:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200008:2.0:1787827014.295864:0:88245:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:2.0:1787827014.295874:0:88245:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:2.0:1787827014.295911:0:88245:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827014.295958:0:88245:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9c00/0xb3f83dad1b7ecf55 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 88245 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827014.295965:0:88245:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9c00/0xb3f83dad1b7ecf55 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 88245 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827014.295974:0:88245:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9c00/0xb3f83dad1b7ecf55 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 88245 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787827014.295998:0:88245:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:88245:1874669227874816:192.168.203.114@tcp:101:lfs.0 00000100:00100000:2.0:1787827014.296075:0:88245:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607ff0780 going to sleep for 11 seconds 00000100:00100000:2.0:1787827014.297516:0:88245:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 219043334106 gen 9 00000100:00100000:2.0:1787827014.299275:0:88245:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:88245:1874669227874816:192.168.203.114@tcp:101:lfs.0 00010000:00010000:2.0:1787827014.299296:0:88245:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9c00/0xb3f83dad1b7ecf55 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91da8df expref: -99 pid: 88245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827014.299306:0:88245:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9c00/0xb3f83dad1b7ecf55 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91da8df expref: -99 pid: 88245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827014.299318:0:88245:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9c00/0xb3f83dad1b7ecf55 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91da8df expref: -99 pid: 88245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827014.299328:0:88245:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9c00/0xb3f83dad1b7ecf55 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91da8df expref: -99 pid: 88245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827014.299339:0:88245:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9c00/0xb3f83dad1b7ecf55 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91da8df expref: -99 pid: 88245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827014.299347:0:88245:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9c00/0xb3f83dad1b7ecf55 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91da8df expref: -99 pid: 88245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827014.299354:0:88245:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9c00/0xb3f83dad1b7ecf55 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91da8df expref: -99 pid: 88245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1787827014.299366:0:88245:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9856140dbc00 x1874669227874816/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 576/688 e 0 to 0 dl 1787827030 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1787827014.299379:0:88245:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9c00/0xb3f83dad1b7ecf55 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91da8df expref: -99 pid: 88245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827014.299392:0:88245:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:2.0:1787827014.299398:0:88245:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:2.0:1787827014.299493:0:88245:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 69632 00000080:00010000:2.0:1787827014.299499:0:88245:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ecf55 00000080:00010000:2.0:1787827014.299505:0:88245:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:2.0:1787827014.299508:0:88245:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ecf55 from it ffffab3ac1cd7cb8 00010000:00010000:2.0:1787827014.299511:0:88245:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9c00/0xb3f83dad1b7ecf55 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91da8df expref: -99 pid: 88245 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827014.299520:0:88245:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9c00/0xb3f83dad1b7ecf55 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91da8df expref: -99 pid: 88245 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:2.0:1787827014.300137:0:88245:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 0, flags 0, foreign 0 00000080:00200000:2.0:1787827014.300143:0:88245:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:2.0:1787827014.300159:0:88245:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827014.300169:0:88245:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9c00/0xb3f83dad1b7ecf55 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91da8df expref: -99 pid: 88245 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787827014.300180:0:88245:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9c00/0xb3f83dad1b7ecf55 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91da8df expref: -99 pid: 88245 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787827014.300187:0:88245:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827014.300192:0:88245:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:2.0:1787827014.300195:0:88245:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ecf55 00000080:00010000:2.0:1787827014.300199:0:88245:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ecf55 from it ffffab3ac1cd7cb8 00010000:00010000:2.0:1787827014.300202:0:88245:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9c00/0xb3f83dad1b7ecf55 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91da8df expref: -99 pid: 88245 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787827014.300211:0:88245:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9c00/0xb3f83dad1b7ecf55 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91da8df expref: -99 pid: 88245 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1787827014.300300:0:88245:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:2.0:1787827014.300309:0:88245:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827014.300317:0:88245:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9c00/0xb3f83dad1b7ecf55 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91da8df expref: -99 pid: 88245 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1787827014.300325:0:88245:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9c00/0xb3f83dad1b7ecf55 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91da8df expref: -99 pid: 88245 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1787827014.300332:0:88245:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827014.300335:0:88245:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:2.0:1787827014.300338:0:88245:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ecf55 00000080:00010000:2.0:1787827014.300342:0:88245:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ecf55 from it ffffab3ac1cd7bb0 00010000:00010000:2.0:1787827014.300345:0:88245:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9c00/0xb3f83dad1b7ecf55 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91da8df expref: -99 pid: 88245 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1787827014.300352:0:88245:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9c00/0xb3f83dad1b7ecf55 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91da8df expref: -99 pid: 88245 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 1 00000080:00200000:2.0:1787827014.300360:0:88245:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 44 00000080:00200008:2.0:1787827014.300367:0:88245:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 100000 00000080:00200080:2.0:1787827014.300492:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00800000:00000080:2.0:1787827014.300507:0:88245:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00800000:00000004:2.0:1787827014.300513:0:88245:0:(obd_class.h:1091:obd_statfs()) lustre-MDT0000-mdc-ffff985602bc9800: age 3159, max_age 3164 00000100:00100000:2.0:1787827014.300544:0:88245:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:88245:1874669227874944:192.168.203.114@tcp:41:lfs.0 00000100:00100000:2.0:1787827014.300609:0:88245:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607ff0780 going to sleep for 11 seconds 00000100:00100000:2.0:1787827014.301783:0:88245:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:88245:1874669227874944:192.168.203.114@tcp:41:lfs.0 00800000:00000004:2.0:1787827014.301804:0:88245:0:(obd_class.h:1117:obd_statfs()) lustre-MDT0000-mdc-ffff985602bc9800: update ffff985606a337f0 cache blocks 317317/353529 objects 1022888/1024000 00000080:00200080:2.0:1787827014.301857:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00800000:00000080:2.0:1787827014.301869:0:88245:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00800000:00000004:2.0:1787827014.301898:0:88245:0:(obd_class.h:1091:obd_statfs()) lustre-MDT0001-mdc-ffff985602bc9800: age 3146, max_age 3164 00000100:00100000:2.0:1787827014.301923:0:88245:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:88245:1874669227875072:192.168.203.114@tcp:41:lfs.0 00000100:00100000:2.0:1787827014.301972:0:88245:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607ff0780 going to sleep for 11 seconds 00000100:00100000:2.0:1787827014.302817:0:88245:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:88245:1874669227875072:192.168.203.114@tcp:41:lfs.0 00800000:00000004:2.0:1787827014.302830:0:88245:0:(obd_class.h:1117:obd_statfs()) lustre-MDT0001-mdc-ffff985602bc9800: update ffff98560952b7f0 cache blocks 317401/353529 objects 1023737/1024000 00000080:00200080:2.0:1787827014.302865:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00800000:00000080:2.0:1787827014.302874:0:88245:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.302881:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00800000:00000080:2.0:1787827014.302898:0:88245:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.302905:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00800000:00000080:2.0:1787827014.302909:0:88245:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.302915:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00800000:00000080:2.0:1787827014.302919:0:88245:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.302924:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00800000:00000080:2.0:1787827014.302928:0:88245:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.302932:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00800000:00000080:2.0:1787827014.302936:0:88245:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.302940:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00800000:00000080:2.0:1787827014.302945:0:88245:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.302949:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00800000:00000080:2.0:1787827014.302953:0:88245:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.302957:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00800000:00000080:2.0:1787827014.302961:0:88245:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.302965:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00800000:00000080:2.0:1787827014.302968:0:88245:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.302973:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00800000:00000080:2.0:1787827014.302977:0:88245:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.302981:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00800000:00000080:2.0:1787827014.302984:0:88245:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.302988:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00800000:00000080:2.0:1787827014.302992:0:88245:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.302996:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00800000:00000080:2.0:1787827014.303000:0:88245:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.303004:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00800000:00000080:2.0:1787827014.303008:0:88245:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.303012:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00800000:00000080:2.0:1787827014.303016:0:88245:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.303020:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00800000:00000080:2.0:1787827014.303024:0:88245:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.303028:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00800000:00000080:2.0:1787827014.303031:0:88245:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.303035:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00800000:00000080:2.0:1787827014.303039:0:88245:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.303043:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00800000:00000080:2.0:1787827014.303047:0:88245:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.303051:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00800000:00000080:2.0:1787827014.303055:0:88245:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.303059:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00800000:00000080:2.0:1787827014.303063:0:88245:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.303067:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00800000:00000080:2.0:1787827014.303071:0:88245:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.303075:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00800000:00000080:2.0:1787827014.303078:0:88245:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.303082:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00800000:00000080:2.0:1787827014.303085:0:88245:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.303090:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00800000:00000080:2.0:1787827014.303103:0:88245:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.303116:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00800000:00000080:2.0:1787827014.303121:0:88245:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.303126:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00800000:00000080:2.0:1787827014.303130:0:88245:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.303134:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00800000:00000080:2.0:1787827014.303138:0:88245:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.303142:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00800000:00000080:2.0:1787827014.303146:0:88245:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.303151:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00800000:00000080:2.0:1787827014.303154:0:88245:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.303158:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00800000:00000080:2.0:1787827014.303162:0:88245:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.303166:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00800000:00000080:2.0:1787827014.303169:0:88245:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.303173:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00800000:00000080:2.0:1787827014.303176:0:88245:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.303179:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00800000:00000080:2.0:1787827014.303182:0:88245:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.303186:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00800000:00000080:2.0:1787827014.303189:0:88245:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.303193:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00800000:00000080:2.0:1787827014.303197:0:88245:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.303201:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00800000:00000080:2.0:1787827014.303204:0:88245:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.303207:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00800000:00000080:2.0:1787827014.303210:0:88245:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.303214:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00800000:00000080:2.0:1787827014.303217:0:88245:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.303221:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00800000:00000080:2.0:1787827014.303224:0:88245:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.303228:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00800000:00000080:2.0:1787827014.303231:0:88245:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.303234:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00800000:00000080:2.0:1787827014.303242:0:88245:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.303246:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00800000:00000080:2.0:1787827014.303249:0:88245:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.303254:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00800000:00000080:2.0:1787827014.303257:0:88245:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.303269:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00800000:00000080:2.0:1787827014.303278:0:88245:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.303282:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00800000:00000080:2.0:1787827014.303286:0:88245:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.303290:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00800000:00000080:2.0:1787827014.303294:0:88245:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.303297:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00800000:00000080:2.0:1787827014.303300:0:88245:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.303304:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00800000:00000080:2.0:1787827014.303308:0:88245:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.303312:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00800000:00000080:2.0:1787827014.303315:0:88245:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.303319:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00800000:00000080:2.0:1787827014.303322:0:88245:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.303326:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00800000:00000080:2.0:1787827014.303330:0:88245:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.303334:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00800000:00000080:2.0:1787827014.303338:0:88245:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.303341:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00800000:00000080:2.0:1787827014.303345:0:88245:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.303349:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00800000:00000080:2.0:1787827014.303352:0:88245:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.303370:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00800000:00000080:2.0:1787827014.303375:0:88245:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.303379:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00800000:00000080:2.0:1787827014.303383:0:88245:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.303387:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00800000:00000080:2.0:1787827014.303390:0:88245:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.303395:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00800000:00000080:2.0:1787827014.303398:0:88245:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.303402:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00800000:00000080:2.0:1787827014.303406:0:88245:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.303409:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00800000:00000080:2.0:1787827014.303413:0:88245:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.303417:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00800000:00000080:2.0:1787827014.303421:0:88245:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.303426:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00020000:00000080:2.0:1787827014.303433:0:88245:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00020000:00000004:2.0:1787827014.303440:0:88245:0:(obd_class.h:1091:obd_statfs()) lustre-OST0000-osc-ffff985602bc9800: age 3146, max_age 3164 00000100:00100000:2.0:1787827014.303469:0:88245:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:88245:1874669227875200:192.168.203.114@tcp:13:lfs.0 00000100:00100000:2.0:1787827014.303516:0:88245:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607ff0780 going to sleep for 11 seconds 00000100:00100000:2.0:1787827014.304471:0:88245:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:88245:1874669227875200:192.168.203.114@tcp:13:lfs.0 00020000:00000004:2.0:1787827014.304487:0:88245:0:(obd_class.h:1117:obd_statfs()) lustre-OST0000-osc-ffff985602bc9800: update ffff985609528450 cache blocks 901356/958279 objects 261344/262144 00000080:00200080:2.0:1787827014.304525:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00020000:00000080:2.0:1787827014.304535:0:88245:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00020000:00000004:2.0:1787827014.304541:0:88245:0:(obd_class.h:1091:obd_statfs()) lustre-OST0001-osc-ffff985602bc9800: age 3146, max_age 3164 00000100:00100000:2.0:1787827014.304560:0:88245:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:88245:1874669227875328:192.168.203.114@tcp:13:lfs.0 00000100:00100000:2.0:1787827014.304600:0:88245:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607ff0780 going to sleep for 11 seconds 00000100:00100000:2.0:1787827014.305382:0:88245:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:88245:1874669227875328:192.168.203.114@tcp:13:lfs.0 00020000:00000004:2.0:1787827014.305404:0:88245:0:(obd_class.h:1117:obd_statfs()) lustre-OST0001-osc-ffff985602bc9800: update ffff98560952d1c0 cache blocks 901363/958279 objects 261383/262144 00000080:00200080:2.0:1787827014.305433:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00020000:00000080:2.0:1787827014.305441:0:88245:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.305447:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00020000:00000080:2.0:1787827014.305452:0:88245:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.305457:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00020000:00000080:2.0:1787827014.305461:0:88245:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.305466:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00020000:00000080:2.0:1787827014.305470:0:88245:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.305474:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00020000:00000080:2.0:1787827014.305478:0:88245:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.305481:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00020000:00000080:2.0:1787827014.305484:0:88245:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.305487:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00020000:00000080:2.0:1787827014.305489:0:88245:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.305491:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00020000:00000080:2.0:1787827014.305493:0:88245:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.305494:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00020000:00000080:2.0:1787827014.305496:0:88245:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.305498:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00020000:00000080:2.0:1787827014.305500:0:88245:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.305502:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00020000:00000080:2.0:1787827014.305504:0:88245:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.305506:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00020000:00000080:2.0:1787827014.305508:0:88245:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.305510:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00020000:00000080:2.0:1787827014.305512:0:88245:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.305514:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00020000:00000080:2.0:1787827014.305516:0:88245:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.305518:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00020000:00000080:2.0:1787827014.305520:0:88245:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.305524:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00020000:00000080:2.0:1787827014.305532:0:88245:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.305536:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00020000:00000080:2.0:1787827014.305539:0:88245:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.305543:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00020000:00000080:2.0:1787827014.305547:0:88245:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.305550:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00020000:00000080:2.0:1787827014.305554:0:88245:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.305558:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00020000:00000080:2.0:1787827014.305561:0:88245:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.305564:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00020000:00000080:2.0:1787827014.305568:0:88245:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.305572:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00020000:00000080:2.0:1787827014.305575:0:88245:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.305584:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00020000:00000080:2.0:1787827014.305588:0:88245:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.305593:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00020000:00000080:2.0:1787827014.305597:0:88245:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.305601:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00020000:00000080:2.0:1787827014.305604:0:88245:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.305608:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00020000:00000080:2.0:1787827014.305612:0:88245:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.305616:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00020000:00000080:2.0:1787827014.305619:0:88245:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.305621:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00020000:00000080:2.0:1787827014.305623:0:88245:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.305625:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00020000:00000080:2.0:1787827014.305627:0:88245:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.305629:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00020000:00000080:2.0:1787827014.305631:0:88245:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.305633:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00020000:00000080:2.0:1787827014.305634:0:88245:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.305636:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00020000:00000080:2.0:1787827014.305638:0:88245:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.305640:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00020000:00000080:2.0:1787827014.305642:0:88245:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.305644:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00020000:00000080:2.0:1787827014.305645:0:88245:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.305647:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00020000:00000080:2.0:1787827014.305649:0:88245:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.305651:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00020000:00000080:2.0:1787827014.305653:0:88245:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.305655:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00020000:00000080:2.0:1787827014.305657:0:88245:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.305659:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00020000:00000080:2.0:1787827014.305660:0:88245:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.305662:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00020000:00000080:2.0:1787827014.305664:0:88245:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.305666:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00020000:00000080:2.0:1787827014.305667:0:88245:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.305669:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00020000:00000080:2.0:1787827014.305678:0:88245:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.305680:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00020000:00000080:2.0:1787827014.305682:0:88245:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.305684:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00020000:00000080:2.0:1787827014.305686:0:88245:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.305688:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00020000:00000080:2.0:1787827014.305711:0:88245:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.305718:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00020000:00000080:2.0:1787827014.305722:0:88245:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.305727:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00020000:00000080:2.0:1787827014.305730:0:88245:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.305734:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00020000:00000080:2.0:1787827014.305738:0:88245:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.305742:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00020000:00000080:2.0:1787827014.305746:0:88245:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.305750:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00020000:00000080:2.0:1787827014.305754:0:88245:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.305758:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00020000:00000080:2.0:1787827014.305761:0:88245:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.305765:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00020000:00000080:2.0:1787827014.305769:0:88245:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.305772:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00020000:00000080:2.0:1787827014.305776:0:88245:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.305780:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00020000:00000080:2.0:1787827014.305783:0:88245:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.305786:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00020000:00000080:2.0:1787827014.305789:0:88245:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.305793:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00020000:00000080:2.0:1787827014.305797:0:88245:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.305801:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00020000:00000080:2.0:1787827014.305805:0:88245:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.305809:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00020000:00000080:2.0:1787827014.305811:0:88245:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.305814:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00020000:00000080:2.0:1787827014.305816:0:88245:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.305818:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00020000:00000080:2.0:1787827014.305820:0:88245:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.305822:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00020000:00000080:2.0:1787827014.305823:0:88245:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.305825:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00020000:00000080:2.0:1787827014.305827:0:88245:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.305830:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00020000:00000080:2.0:1787827014.305832:0:88245:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200080:2.0:1787827014.305834:0:88245:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe939c4cb0 00020000:00000080:2.0:1787827014.305835:0:88245:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f0000 uarg=0000000000000000 00000080:00200008:2.0:1787827014.305845:0:88245:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 100000 00000080:00200000:1.0:1787827014.322904:0:88252:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787827014.322930:0:88252:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827014.322948:0:88252:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9c00/0xb3f83dad1b7ecf55 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91da8df expref: -99 pid: 88245 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 1 00010000:00010000:1.0:1787827014.322963:0:88252:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9c00/0xb3f83dad1b7ecf55 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91da8df expref: -99 pid: 88245 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 1 00000080:00200000:1.0:1787827014.322971:0:88252:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827014.322978:0:88252:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787827014.322981:0:88252:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ecf55 00000080:00010000:1.0:1787827014.322988:0:88252:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ecf55 from it ffffab3ac1cf7bd0 00010000:00010000:1.0:1787827014.322992:0:88252:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9c00/0xb3f83dad1b7ecf55 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91da8df expref: -99 pid: 88245 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 1 00010000:00010000:1.0:1787827014.323001:0:88252:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9c00/0xb3f83dad1b7ecf55 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91da8df expref: -99 pid: 88245 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00000080:00200000:1.0:1787827014.323016:0:88252:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827014.323028:0:88252:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=fsa-oleg314-client.virtnet, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), flags=1536 00000080:00200000:1.0:1787827014.323032:0:88252:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787827014.323038:0:88252:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827014.323043:0:88252:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9c00/0xb3f83dad1b7ecf55 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91da8df expref: -99 pid: 88245 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00010000:00010000:1.0:1787827014.323051:0:88252:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9c00/0xb3f83dad1b7ecf55 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91da8df expref: -99 pid: 88245 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00000080:00200000:1.0:1787827014.323057:0:88252:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827014.323061:0:88252:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787827014.323063:0:88252:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ecf55 00000080:00010000:1.0:1787827014.323067:0:88252:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ecf55 from it ffffab3ac1cf7cd0 00010000:00010000:1.0:1787827014.323070:0:88252:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9c00/0xb3f83dad1b7ecf55 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91da8df expref: -99 pid: 88245 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00010000:00010000:1.0:1787827014.323077:0:88252:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9c00/0xb3f83dad1b7ecf55 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91da8df expref: -99 pid: 88245 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00000080:00200000:1.0:1787827014.323085:0:88252:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827014.323089:0:88252:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787827014.323094:0:88252:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827014.323100:0:88252:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9c00/0xb3f83dad1b7ecf55 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91da8df expref: -99 pid: 88245 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00010000:00010000:1.0:1787827014.323108:0:88252:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9c00/0xb3f83dad1b7ecf55 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91da8df expref: -99 pid: 88245 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00000080:00200000:1.0:1787827014.323114:0:88252:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827014.323117:0:88252:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787827014.323119:0:88252:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ecf55 00000080:00010000:1.0:1787827014.323123:0:88252:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ecf55 from it ffffab3ac1cf7db0 00010000:00010000:1.0:1787827014.323126:0:88252:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9c00/0xb3f83dad1b7ecf55 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91da8df expref: -99 pid: 88245 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00010000:00010000:1.0:1787827014.323132:0:88252:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9c00/0xb3f83dad1b7ecf55 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91da8df expref: -99 pid: 88245 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00000080:00200000:1.0:1787827014.323139:0:88252:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827014.323144:0:88252:0:(namei.c:2129:ll_create_nd()) VFS Op:name=fsa-oleg314-client.virtnet, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), flags=33188, excl=1 00000080:00200000:1.0:1787827014.323148:0:88252:0:(namei.c:2084:ll_mknod()) VFS Op:name=fsa-oleg314-client.virtnet, dir=[0x200000007:0x1:0x0](ffff98561158e3c8) mode 100644 dev 0 00010000:00010000:1.0:1787827014.323174:0:88252:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9c00/0xb3f83dad1b7ecf55 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x5aa6d504d91da8df expref: -99 pid: 88245 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00010000:00010000:1.0:1787827014.323306:0:88252:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9c00/0xb3f83dad1b7ecf55 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91da8df expref: -99 pid: 88245 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00010000:00010000:1.0:1787827014.323314:0:88252:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787827014.323316:0:88252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9c00/0xb3f83dad1b7ecf55 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91da8df expref: -99 pid: 88245 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00000020:00100000:1.0:1787827014.323341:0:88252:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827014.323353:0:88252:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985608408f00 x1874669227875584/t0(0) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 528/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827014.323375:0:88252:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608408f00 pname:cluuid:pid:xid:nid:opc:job mcreate:7b4e8535-9db7-485d-abf6-bda966b63320:88252:1874669227875584:192.168.203.114@tcp:36:mcreate.0 00000100:00100000:1.0:1787827014.323455:0:88252:0:(client.c:2715:ptlrpc_set_wait()) set ffff9856054376c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827014.325206:0:88252:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608408f00 pname:cluuid:pid:xid:nid:opc:job mcreate:7b4e8535-9db7-485d-abf6-bda966b63320:88252:1874669227875584:192.168.203.114@tcp:36:mcreate.0 00000080:00200000:1.0:1787827014.325235:0:88252:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x2000182d1:0x965:0x0](ffff98561014db88) 00000080:00200000:1.0:1787827014.325262:0:88252:0:(namei.c:2137:ll_create_nd()) VFS Op:name=fsa-oleg314-client.virtnet, unhashed 1 00000080:00200000:1.0:1787827014.332331:0:88255:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787827014.332355:0:88255:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827014.332403:0:88255:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610cda800/0xb3f83dad1b7ecf5c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 88255 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827014.332410:0:88255:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610cda800/0xb3f83dad1b7ecf5c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 88255 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827014.332417:0:88255:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610cda800/0xb3f83dad1b7ecf5c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 88255 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787827014.332435:0:88255:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608409e00 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:88255:1874669227875712:192.168.203.114@tcp:101:rm.0 00000100:00100000:1.0:1787827014.332553:0:88255:0:(client.c:2715:ptlrpc_set_wait()) set ffff9856054376c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827014.333880:0:88255:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608409e00 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:88255:1874669227875712:192.168.203.114@tcp:101:rm.0 00010000:00010000:1.0:1787827014.333910:0:88255:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610cda800/0xb3f83dad1b7ecf5c lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91da8fb expref: -99 pid: 88255 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827014.333923:0:88255:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610cda800/0xb3f83dad1b7ecf5c lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91da8fb expref: -99 pid: 88255 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827014.333932:0:88255:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610cda800/0xb3f83dad1b7ecf5c lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91da8fb expref: -99 pid: 88255 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827014.333940:0:88255:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610cda800/0xb3f83dad1b7ecf5c lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91da8fb expref: -99 pid: 88255 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827014.333947:0:88255:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610cda800/0xb3f83dad1b7ecf5c lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91da8fb expref: -99 pid: 88255 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827014.333952:0:88255:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610cda800/0xb3f83dad1b7ecf5c lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91da8fb expref: -99 pid: 88255 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827014.333956:0:88255:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610cda800/0xb3f83dad1b7ecf5c lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91da8fb expref: -99 pid: 88255 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787827014.333963:0:88255:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff985608409e00 x1874669227875712/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 576/688 e 0 to 0 dl 1787827030 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787827014.333972:0:88255:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610cda800/0xb3f83dad1b7ecf5c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91da8fb expref: -99 pid: 88255 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827014.333978:0:88255:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:1.0:1787827014.333982:0:88255:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:1.0:1787827014.334000:0:88255:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 69632 00000080:00010000:1.0:1787827014.334005:0:88255:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ecf5c 00000080:00010000:1.0:1787827014.334008:0:88255:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787827014.334010:0:88255:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ecf5c from it ffffab3ac1cd7b58 00010000:00010000:1.0:1787827014.334012:0:88255:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610cda800/0xb3f83dad1b7ecf5c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91da8fb expref: -99 pid: 88255 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827014.334018:0:88255:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610cda800/0xb3f83dad1b7ecf5c lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91da8fb expref: -99 pid: 88255 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787827014.334037:0:88255:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827014.334048:0:88255:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=fsa-oleg314-client.virtnet, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), flags=8 00000080:00200000:1.0:1787827014.334051:0:88255:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=fsa-oleg314-client.virtnet, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=getattr 00000002:00010000:1.0:1787827014.334063:0:88255:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: fsa-oleg314-client.virtnet,[0x0:0x0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 01 00010000:00010000:1.0:1787827014.334081:0:88255:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610cdb800/0xb3f83dad1b7ecf63 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 88255 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827014.334086:0:88255:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610cdb800/0xb3f83dad1b7ecf63 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 88255 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827014.334090:0:88255:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610cdb800/0xb3f83dad1b7ecf63 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 88255 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787827014.334107:0:88255:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608409e00 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:88255:1874669227875840:192.168.203.114@tcp:101:rm.0 00000100:00100000:1.0:1787827014.334144:0:88255:0:(client.c:2715:ptlrpc_set_wait()) set ffff9856054376c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827014.335478:0:88255:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608409e00 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:88255:1874669227875840:192.168.203.114@tcp:101:rm.0 00010000:00010000:1.0:1787827014.335491:0:88255:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610cdb800/0xb3f83dad1b7ecf63 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91da917 expref: -99 pid: 88255 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827014.335512:0:88255:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610cdb800/0xb3f83dad1b7ecf63 lrc: 4/1,0 mode: --/PR res: [0x2000182d1:0x965:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91da917 expref: -99 pid: 88255 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827014.335519:0:88255:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610cdb800/0xb3f83dad1b7ecf63 lrc: 4/1,0 mode: --/PR res: [0x2000182d1:0x965:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91da917 expref: -99 pid: 88255 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827014.335527:0:88255:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610cdb800/0xb3f83dad1b7ecf63 lrc: 4/1,0 mode: PR/PR res: [0x2000182d1:0x965:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91da917 expref: -99 pid: 88255 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827014.335536:0:88255:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610cdb800/0xb3f83dad1b7ecf63 lrc: 4/1,0 mode: PR/PR res: [0x2000182d1:0x965:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91da917 expref: -99 pid: 88255 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827014.335545:0:88255:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610cdb800/0xb3f83dad1b7ecf63 lrc: 5/2,0 mode: PR/PR res: [0x2000182d1:0x965:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91da917 expref: -99 pid: 88255 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827014.335553:0:88255:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610cdb800/0xb3f83dad1b7ecf63 lrc: 5/2,0 mode: PR/PR res: [0x2000182d1:0x965:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91da917 expref: -99 pid: 88255 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827014.335560:0:88255:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610cdb800/0xb3f83dad1b7ecf63 lrc: 4/1,0 mode: PR/PR res: [0x2000182d1:0x965:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91da917 expref: -99 pid: 88255 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787827014.335581:0:88255:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff985608409e00 x1874669227875840/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 600/584 e 0 to 0 dl 1787827030 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787827014.335591:0:88255:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610cdb800/0xb3f83dad1b7ecf63 lrc: 3/1,0 mode: PR/PR res: [0x2000182d1:0x965:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91da917 expref: -99 pid: 88255 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827014.335604:0:88255:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8590033617/2405 (0 0) 00000080:00200000:1.0:1787827014.335608:0:88255:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:1.0:1787827014.335620:0:88255:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x2000182d1:0x965:0x0], updating i_size 0 00000080:00200000:1.0:1787827014.335623:0:88255:0:(namei.c:147:ll_iget()) got inode: [0x2000182d1:0x965:0x0](ffff98561014db88): rc = 0 00000080:00200000:1.0:1787827014.335644:0:88255:0:(file.c:6854:ll_layout_conf()) [0x2000182d1:0x965:0x0]: layout version change: 4294967294 -> 4294967295 00000080:00010000:1.0:1787827014.335648:0:88255:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x2000182d1:0x965:0x0](ffff98561014db88) for lock 0xb3f83dad1b7ecf63 00000080:00010000:1.0:1787827014.335656:0:88255:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x2000182d1:0x965:0x0](ffff98561014db88) 00000080:00010000:1.0:1787827014.335659:0:88255:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ecf63 from it ffffab3ac1cd7b20 00010000:00010000:1.0:1787827014.335662:0:88255:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610cdb800/0xb3f83dad1b7ecf63 lrc: 3/1,0 mode: PR/PR res: [0x2000182d1:0x965:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91da917 expref: -99 pid: 88255 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827014.335670:0:88255:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610cdb800/0xb3f83dad1b7ecf63 lrc: 2/0,0 mode: PR/PR res: [0x2000182d1:0x965:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91da917 expref: -99 pid: 88255 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:1.0:1787827014.335946:0:88255:0:(file.c:6313:ll_getattr_dentry()) START file fsa-oleg314-client.virtnet:[0x2000182d1:0x965:0x0](ffff98561014db88), request_mask 2047, flags 0, foreign 0 00000080:00200000:1.0:1787827014.335952:0:88255:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x2000182d1:0x965:0x0](ffff98561014db88),name=fsa-oleg314-client.virtnet 00000002:00010000:1.0:1787827014.335966:0:88255:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x2000182d1:0x965:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827014.335972:0:88255:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610cdb800/0xb3f83dad1b7ecf63 lrc: 2/1,0 mode: PR/PR res: [0x2000182d1:0x965:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91da917 expref: -99 pid: 88255 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827014.335980:0:88255:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610cdb800/0xb3f83dad1b7ecf63 lrc: 2/1,0 mode: PR/PR res: [0x2000182d1:0x965:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91da917 expref: -99 pid: 88255 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787827014.335985:0:88255:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827014.335988:0:88255:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x2000182d1:0x965:0x0](ffff98561014db88) 00000080:00010000:1.0:1787827014.335990:0:88255:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x2000182d1:0x965:0x0](ffff98561014db88) for lock 0xb3f83dad1b7ecf63 00000080:00010000:1.0:1787827014.335994:0:88255:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ecf63 from it ffffab3ac1cd7cb8 00010000:00010000:1.0:1787827014.335997:0:88255:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610cdb800/0xb3f83dad1b7ecf63 lrc: 3/1,0 mode: PR/PR res: [0x2000182d1:0x965:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91da917 expref: -99 pid: 88255 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827014.336002:0:88255:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610cdb800/0xb3f83dad1b7ecf63 lrc: 2/0,0 mode: PR/PR res: [0x2000182d1:0x965:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91da917 expref: -99 pid: 88255 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787827014.336037:0:88255:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787827014.336042:0:88255:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827014.336045:0:88255:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610cda800/0xb3f83dad1b7ecf5c lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91da8fb expref: -99 pid: 88255 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827014.336050:0:88255:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610cda800/0xb3f83dad1b7ecf5c lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91da8fb expref: -99 pid: 88255 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787827014.336054:0:88255:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827014.336057:0:88255:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787827014.336058:0:88255:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ecf5c 00000080:00010000:1.0:1787827014.336060:0:88255:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ecf5c from it ffffab3ac1cd7c10 00010000:00010000:1.0:1787827014.336062:0:88255:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610cda800/0xb3f83dad1b7ecf5c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91da8fb expref: -99 pid: 88255 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827014.336066:0:88255:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610cda800/0xb3f83dad1b7ecf5c lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91da8fb expref: -99 pid: 88255 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787827014.336070:0:88255:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827014.336076:0:88255:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=fsa-oleg314-client.virtnet, flags=0 00000080:00200000:1.0:1787827014.336079:0:88255:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787827014.336083:0:88255:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827014.336086:0:88255:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610cda800/0xb3f83dad1b7ecf5c lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91da8fb expref: -99 pid: 88255 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787827014.336090:0:88255:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610cda800/0xb3f83dad1b7ecf5c lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91da8fb expref: -99 pid: 88255 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787827014.336094:0:88255:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827014.336095:0:88255:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787827014.336096:0:88255:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ecf5c 00000080:00010000:1.0:1787827014.336098:0:88255:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ecf5c from it ffffab3ac1cd7d68 00010000:00010000:1.0:1787827014.336100:0:88255:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610cda800/0xb3f83dad1b7ecf5c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91da8fb expref: -99 pid: 88255 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787827014.336103:0:88255:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610cda800/0xb3f83dad1b7ecf5c lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91da8fb expref: -99 pid: 88255 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787827014.336107:0:88255:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827014.336110:0:88255:0:(namei.c:2418:ll_unlink()) VFS Op:name=fsa-oleg314-client.virtnet, dir=[0x200000007:0x1:0x0](ffff98561158e3c8) 00010000:00010000:1.0:1787827014.336123:0:88255:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610cda800/0xb3f83dad1b7ecf5c lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x5aa6d504d91da8fb expref: -99 pid: 88255 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827014.336224:0:88255:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610cdb800/0xb3f83dad1b7ecf63 lrc: 2/0,0 mode: PR/PR res: [0x2000182d1:0x965:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x5aa6d504d91da917 expref: -99 pid: 88255 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787827014.336231:0:88255:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x965:0x0]: losing layout lock 00010000:00010000:1.0:1787827014.336248:0:88255:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610cdb800/0xb3f83dad1b7ecf63 lrc: 1/0,0 mode: --/PR res: [0x2000182d1:0x965:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x5aa6d504d91da917 expref: -99 pid: 88255 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787827014.336256:0:88255:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610cda800/0xb3f83dad1b7ecf5c lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91da8fb expref: -99 pid: 88255 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827014.336259:0:88255:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 2 locks packed 00010000:00010000:1.0:1787827014.336261:0:88255:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610cdb800/0xb3f83dad1b7ecf63 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x965:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x5aa6d504d91da917 expref: -99 pid: 88255 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787827014.336273:0:88255:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610cda800/0xb3f83dad1b7ecf5c lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91da8fb expref: -99 pid: 88255 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000020:00100000:1.0:1787827014.336288:0:88255:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827014.336294:0:88255:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985608409e00 x1874669227876096/t0(0) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827014.336311:0:88255:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985608409e00 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:88255:1874669227876096:192.168.203.114@tcp:36:rm.0 00000100:00100000:1.0:1787827014.336355:0:88255:0:(client.c:2715:ptlrpc_set_wait()) set ffff9856054376c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827014.338692:0:88255:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985608409e00 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:88255:1874669227876096:192.168.203.114@tcp:36:rm.0 00000080:00200000:1.0:1787827014.338779:0:88255:0:(vvp_io.c:1885:vvp_io_init()) [0x2000182d1:0x965:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787827014.338790:0:88255:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x965:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787827014.338799:0:88255:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x965:0x0](ffff98561014db88) 00010000:00010000:1.0:1787827014.452869:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827015.476829:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827016.500854:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:1.0:1787827016.545130:0:88321:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: local REPLAY BARRIER on lustre-OST0000 00000100:00100000:1.0:1787827016.949814:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856086ac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:3c946890-15ca-497a-b838-fe1cba61f4dd:2384:1874669227876224:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787827016.949835:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856086af840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669227876352:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787827016.949899:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856086acb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669227876608:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787827016.949934:0:2383:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1787827016.949940:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856086afc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669227876480:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787827016.949961:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856086ae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669227876736:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787827016.951591:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856086ac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:3c946890-15ca-497a-b838-fe1cba61f4dd:2384:1874669227876224:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787827016.951632:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856086afc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669227876480:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787827016.951644:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856086ae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669227876736:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787827016.951654:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856086af840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669227876352:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787827016.951665:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856086acb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669227876608:192.168.203.114@tcp:400:kworker.0 00010000:00000001:0.0:1787827019.572804:0:74885:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827019.572810:0:74885:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827019.572816:0:74885:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827019.572817:0:74885:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827019.572820:0:74885:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827019.572822:0:74885:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827019.572825:0:74885:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827019.572826:0:74885:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1787827019.572890:0:33341:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1787827019.572893:0:33341:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1787827019.572895:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:0.0:1787827020.596779:0:74885:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827020.596791:0:74885:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:0.0:1787827020.596794:0:74885:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:0.0:1787827020.596797:0:74885:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827020.596802:0:74885:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:0.0:1787827020.596803:0:74885:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827020.596805:0:74885:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827020.596807:0:74885:0:(ldlm_pool.c:496:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827020.596810:0:74885:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827020.596811:0:74885:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827020.596814:0:74885:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827020.596815:0:74885:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827020.596818:0:74885:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827020.596819:0:74885:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1787827020.596841:0:15164:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1787827020.596853:0:15164:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1787827020.596855:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:0.0:1787827021.620806:0:74885:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827021.620812:0:74885:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827021.620816:0:74885:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827021.620817:0:74885:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827021.620825:0:74885:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827021.620826:0:74885:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827021.620830:0:74885:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827021.620831:0:74885:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1787827021.620862:0:33342:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1787827021.620865:0:33342:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1787827021.620868:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:2.0:1787827022.069048:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1787827022.069056:0:38:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1787827022.069065:0:38:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 904 at ffff9856140db840. 00000020:00000040:2.0:1787827022.069070:0:38:0:(genops.c:1210:class_import_get()) import ffff98561118e000 refcount=4 obd=MGC192.168.203.114@tcp 00000100:00000001:2.0:1787827022.069073:0:38:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1787827022.069075:0:38:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1787827022.069079:0:38:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1787827022.069085:0:38:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff985615779000. 00000100:00000001:2.0:1787827022.069091:0:38:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1787827022.069099:0:38:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS req@ffff9856140db840 x1874669228030464/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1787827022.069107:0:38:0:(jobid.c:939:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1787827022.069110:0:38:0:(jobid.c:979:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1787827022.069115:0:38:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000003944aa76] to pc [ptlrpcd_01_01+1] req@ffff9856140db840 x1874669228030464/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.069145:0:38:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1787827022.069148:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1787827022.069151:0:38:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1787827022.069156:0:38:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 904 at ffff9856140dad00. 00000020:00000040:2.0:1787827022.069159:0:38:0:(genops.c:1210:class_import_get()) import ffff985611182000 refcount=406 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000100:00000001:2.0:1787827022.069161:0:38:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1787827022.069162:0:38:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1787827022.069164:0:38:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1787827022.069168:0:38:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff985615779800. 00000100:00000001:2.0:1787827022.069172:0:38:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827022.069184:0:2386:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:2.0:1787827022.069189:0:38:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID req@ffff9856140dad00 x1874669228030592/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1787827022.069190:0:2386:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1787827022.069193:0:2386:0:(client.c:1813:ptlrpc_send_new_req()) Process entered 00000100:00000040:3.0:1787827022.069201:0:2386:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9856140db840 x1874669228030464/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.069205:0:38:0:(jobid.c:939:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1787827022.069209:0:38:0:(jobid.c:979:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827022.069214:0:2386:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000040:2.0:1787827022.069217:0:38:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000092783905] to pc [ptlrpcd_01_00+0] req@ffff9856140dad00 x1874669228030592/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827022.069217:0:2386:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1787827022.069222:0:2386:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1787827022.069225:0:2386:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1787827022.069228:0:38:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1787827022.069228:0:2386:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1787827022.069231:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1787827022.069234:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:3c946890-15ca-497a-b838-fe1cba61f4dd:2386:1874669228030464:192.168.203.114@tcp:400:kworker.0 00000100:00000001:2.0:1787827022.069236:0:38:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000001:3.0:1787827022.069238:0:2386:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:3.0:1787827022.069243:0:2386:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1787827022.069244:0:2386:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1787827022.069247:0:2386:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000010:2.0:1787827022.069250:0:38:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 904 at ffff9856140da1c0. 02000000:00000010:3.0:1787827022.069252:0:2386:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff98561288c600. 00000020:00000040:2.0:1787827022.069254:0:38:0:(genops.c:1210:class_import_get()) import ffff985611185800 refcount=4 obd=lustre-MDT0001-mdc-ffff985602bc9800 02000000:00000001:3.0:1787827022.069255:0:2386:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1787827022.069260:0:38:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1787827022.069262:0:38:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 00000400:00000200:3.0:1787827022.069264:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 02000000:00000001:2.0:1787827022.069266:0:38:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1787827022.069268:0:2386:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff985602ad93c8. 02000000:00000010:2.0:1787827022.069272:0:38:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff985615779500. 00000400:00000010:3.0:1787827022.069275:0:2386:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985609573088. 00000100:00000001:2.0:1787827022.069277:0:38:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:3.0:1787827022.069282:0:2386:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1874669228030464, portal 25 00000100:00000040:2.0:1787827022.069285:0:38:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID req@ffff9856140da1c0 x1874669228030720/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1787827022.069286:0:2386:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1787827022.069288:0:2386:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446630094203893824 : -113979505657792 : ffff9856140db840) 00000100:00000040:3.0:1787827022.069297:0:2386:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9856140db840 x1874669228030464/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827038 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.069303:0:38:0:(jobid.c:939:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1787827022.069308:0:38:0:(jobid.c:979:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827022.069308:0:2386:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1787827022.069312:0:2386:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.203.114@tcp 00000100:00000040:2.0:1787827022.069317:0:38:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000a51ce96b] to pc [ptlrpcd_01_01+1] req@ffff9856140da1c0 x1874669228030720/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000400:00000010:3.0:1787827022.069317:0:2386:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9856095733b8. 00000100:00000200:3.0:1787827022.069321:0:2386:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1874669228030464, offset 0 00000400:00000200:3.0:1787827022.069329:0:2386:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.203.114@tcp 00000100:00000001:2.0:1787827022.069338:0:38:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1787827022.069338:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:3.0:1787827022.069346:0:2386:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.203.114@tcp local destination 00000400:00000200:3.0:1787827022.069352:0:2386:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000040:2.0:1787827022.069353:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-OST0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000400:00000200:3.0:1787827022.069357:0:2386:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.203.14@tcp 00000100:00000001:2.0:1787827022.069358:0:38:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1787827022.069364:0:38:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 904 at ffff9856140db0c0. 00000400:00000200:3.0:1787827022.069364:0:2386:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.203.14@tcp lpni_is_preferred = 0 00000020:00000040:2.0:1787827022.069369:0:38:0:(genops.c:1210:class_import_get()) import ffff9856110b0000 refcount=3 obd=lustre-OST0000-osc-ffff985602bc9800 00000400:00000200:3.0:1787827022.069370:0:2386:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.203.114@tcp 00000100:00000001:2.0:1787827022.069374:0:38:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1787827022.069378:0:38:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 00000400:00000200:3.0:1787827022.069380:0:2386:0:(lib-move.c:1745:lnet_handle_send()) 192.168.203.14@tcp NI seq info: [21874:21874:256:4294967295] 192.168.203.114@tcp LPNI seq info [21874:21874:8:4294967295] 02000000:00000001:2.0:1787827022.069381:0:38:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1787827022.069387:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 02000000:00000010:2.0:1787827022.069400:0:38:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff985615779b00. 00000400:00000200:3.0:1787827022.069401:0:2386:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp:) -> 192.168.203.114@tcp(192.168.203.114@tcp:192.168.203.114@tcp) : PUT try# 0 00000100:00000001:2.0:1787827022.069405:0:38:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1787827022.069410:0:2386:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.203.114@tcp 00000800:00000010:3.0:1787827022.069415:0:2386:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff985609025200. 00000800:00000200:3.0:1787827022.069421:0:2386:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff985606751480] -> 12345-192.168.203.114@tcp (5) 00000100:00000040:2.0:1787827022.069427:0:38:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-OST0000_UUID req@ffff9856140db0c0 x1874669228030848/t0(0) o400->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000200:3.0:1787827022.069430:0:2386:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.203.114@tcp ip 192.168.203.114:988 00000800:00000200:3.0:1787827022.069436:0:2386:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff985609025200 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:2.0:1787827022.069443:0:38:0:(jobid.c:939:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1787827022.069445:0:38:0:(jobid.c:979:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827022.069449:0:2386:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1787827022.069451:0:38:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000874ef749] to pc [ptlrpcd_01_00+0] req@ffff9856140db0c0 x1874669228030848/t0(0) o400->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1787827022.069456:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000001:3.0:1787827022.069460:0:2386:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.203.114@tcp: PM QoS new deadline estimation for cpu 2 is 4297850108 00000100:00000001:2.0:1787827022.069467:0:38:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827022.069469:0:2386:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.203.114@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4297844989 00000100:00000040:2.0:1787827022.069470:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-OST0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1787827022.069475:0:2386:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.203.114@tcp: PM QoS new deadline estimation for cpu 3 is 4297850108 00000100:00000001:2.0:1787827022.069476:0:38:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1787827022.069481:0:38:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 904 at ffff9856140d8780. 00000100:00000001:3.0:1787827022.069481:0:2386:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.203.114@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4297844989 00000100:00000001:3.0:1787827022.069483:0:2386:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000020:00000040:2.0:1787827022.069486:0:38:0:(genops.c:1210:class_import_get()) import ffff9856110b1000 refcount=3 obd=lustre-OST0001-osc-ffff985602bc9800 00000100:00000001:3.0:1787827022.069487:0:2386:0:(client.c:1941:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1787827022.069491:0:38:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:3.0:1787827022.069491:0:2386:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827022.069493:0:2386:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:2.0:1787827022.069494:0:38:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1787827022.069498:0:38:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827022.069500:0:2386:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 02000000:00000010:2.0:1787827022.069502:0:38:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff985615779a00. 00000100:00000001:3.0:1787827022.069503:0:2386:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1787827022.069507:0:2386:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1787827022.069508:0:38:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827022.069509:0:2386:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1787827022.069511:0:2386:0:(client.c:1813:ptlrpc_send_new_req()) Process entered 00000100:00000040:2.0:1787827022.069520:0:38:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-OST0001_UUID req@ffff9856140d8780 x1874669228030976/t0(0) o400->lustre-OST0001-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000040:3.0:1787827022.069521:0:2386:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9856140da1c0 x1874669228030720/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827022.069540:0:2386:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1787827022.069542:0:2386:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1787827022.069544:0:38:0:(jobid.c:939:lustre_get_jobid()) Process entered 02000000:00000001:3.0:1787827022.069545:0:2386:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1787827022.069548:0:2386:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1787827022.069549:0:38:0:(jobid.c:979:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1787827022.069550:0:2386:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1787827022.069553:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228030720:192.168.203.114@tcp:400:kworker.0 00000100:00000040:2.0:1787827022.069556:0:38:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000000aa0081b] to pc [ptlrpcd_01_01+1] req@ffff9856140d8780 x1874669228030976/t0(0) o400->lustre-OST0001-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827022.069557:0:2386:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:3.0:1787827022.069560:0:2386:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1787827022.069561:0:2386:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1787827022.069562:0:2386:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:3.0:1787827022.069564:0:2386:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff98561288d200. 02000000:00000001:3.0:1787827022.069565:0:2386:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1787827022.069567:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000010:3.0:1787827022.069569:0:2386:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff985602ad9790. 00000400:00000010:3.0:1787827022.069574:0:2386:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985609573bb0. 00000100:00000200:3.0:1787827022.069577:0:2386:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1874669228030720, portal 10 00000100:00000001:3.0:1787827022.069581:0:2386:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1787827022.069582:0:2386:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446630094203888064 : -113979505663552 : ffff9856140da1c0) 00000100:00000001:2.0:1787827022.069590:0:38:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1787827022.069591:0:2386:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9856140da1c0 x1874669228030720/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827038 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827022.069598:0:2386:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:2.0:1787827022.069599:0:38:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (3177) 00000100:00000040:3.0:1787827022.069600:0:2386:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.203.114@tcp 00000400:00000010:3.0:1787827022.069604:0:2386:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985609573770. 00000100:00000200:3.0:1787827022.069606:0:2386:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1874669228030720, offset 0 00000400:00000200:3.0:1787827022.069609:0:2386:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.203.114@tcp 00000400:00000200:3.0:1787827022.069612:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:3.0:1787827022.069614:0:2386:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.203.114@tcp local destination 00000400:00000200:3.0:1787827022.069624:0:2386:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:3.0:1787827022.069627:0:2386:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.203.14@tcp 00000400:00000200:3.0:1787827022.069628:0:2386:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.203.14@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1787827022.069630:0:2386:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.203.114@tcp 00000400:00000200:3.0:1787827022.069632:0:2386:0:(lib-move.c:1745:lnet_handle_send()) 192.168.203.14@tcp NI seq info: [21875:21875:255:4294967295] 192.168.203.114@tcp LPNI seq info [21875:21875:7:4294967295] 00000400:00000200:3.0:1787827022.069635:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000001:2.0:1787827022.069637:0:2385:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1787827022.069641:0:2385:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000400:00000200:3.0:1787827022.069642:0:2386:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp:) -> 192.168.203.114@tcp(192.168.203.114@tcp:192.168.203.114@tcp) : PUT try# 0 00000100:00000001:2.0:1787827022.069645:0:2385:0:(client.c:1813:ptlrpc_send_new_req()) Process entered 00000800:00000200:3.0:1787827022.069647:0:2386:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.203.114@tcp 00000800:00000010:3.0:1787827022.069651:0:2386:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff985609025f00. 00000100:00000040:2.0:1787827022.069654:0:2385:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9856140dad00 x1874669228030592/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000800:00000200:3.0:1787827022.069657:0:2386:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff985606751480] -> 12345-192.168.203.114@tcp (5) 00000800:00000200:3.0:1787827022.069666:0:2386:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.203.114@tcp ip 192.168.203.114:988 00000100:00000001:2.0:1787827022.069672:0:2385:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000800:00000200:3.0:1787827022.069672:0:2386:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff985609025f00 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1787827022.069676:0:2386:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1787827022.069679:0:2386:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff985602bc9800: using a timeout of 11981 usecs (12 jiffies) 00000400:00000200:3.0:1787827022.069683:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000001:3.0:1787827022.069684:0:2386:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff985602bc9800: PM QoS new deadline estimation for cpu 2 is 4297840140 00000100:00000001:2.0:1787827022.069687:0:2385:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827022.069714:0:2386:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff985602bc9800: PM QoS request active for cpu 2, keep current deadline 4297850108 00000100:00000001:3.0:1787827022.069718:0:2386:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff985602bc9800: PM QoS new deadline estimation for cpu 3 is 4297840141 00000100:00000001:3.0:1787827022.069720:0:2386:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff985602bc9800: PM QoS request active for cpu 3, keep current deadline 4297850108 00000100:00000001:3.0:1787827022.069723:0:2386:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1787827022.069727:0:2385:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:3.0:1787827022.069727:0:2386:0:(client.c:1941:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1787827022.069731:0:2385:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1787827022.069732:0:2386:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1787827022.069738:0:2385:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1787827022.069742:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669228030592:192.168.203.114@tcp:400:kworker.0 00000100:00000001:2.0:1787827022.069746:0:2385:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1787827022.069749:0:2385:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1787827022.069751:0:2385:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1787827022.069754:0:2385:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:3.0:1787827022.069754:0:2386:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 02000000:00000010:2.0:1787827022.069758:0:2385:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff985611861800. 00000100:00000001:3.0:1787827022.069760:0:2386:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 02000000:00000001:2.0:1787827022.069763:0:2385:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827022.069763:0:2386:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1787827022.069768:0:2386:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:2.0:1787827022.069771:0:2385:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000001:3.0:1787827022.069772:0:2386:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000400:00000010:2.0:1787827022.069776:0:2385:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff98561080ee70. 00000100:00000001:3.0:1787827022.069776:0:2386:0:(client.c:1813:ptlrpc_send_new_req()) Process entered 00000400:00000010:2.0:1787827022.069782:0:2385:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985610afa220. 00000100:00000040:3.0:1787827022.069782:0:2386:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9856140d8780 x1874669228030976/t0(0) o400->lustre-OST0001-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1787827022.069788:0:2385:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1874669228030592, portal 10 00000100:00000001:2.0:1787827022.069792:0:2385:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1787827022.069794:0:2385:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446630094203890944 : -113979505660672 : ffff9856140dad00) 00000100:00000001:3.0:1787827022.069796:0:2386:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1787827022.069799:0:2386:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1787827022.069801:0:2386:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1787827022.069804:0:2386:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000040:2.0:1787827022.069805:0:2385:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9856140dad00 x1874669228030592/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827038 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1787827022.069806:0:2386:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1787827022.069812:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228030976:192.168.203.114@tcp:400:kworker.0 00000100:00000001:3.0:1787827022.069818:0:2386:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:2.0:1787827022.069821:0:2385:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000001:3.0:1787827022.069823:0:2386:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 00000100:00000040:2.0:1787827022.069827:0:2385:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.203.114@tcp 02000000:00000001:3.0:1787827022.069827:0:2386:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1787827022.069841:0:2385:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985610afa990. 02000000:00000001:3.0:1787827022.069843:0:2386:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000200:2.0:1787827022.069845:0:2385:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1874669228030592, offset 0 02000000:00000010:3.0:1787827022.069849:0:2386:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff98561288d600. 02000000:00000001:3.0:1787827022.069853:0:2386:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1787827022.069854:0:2385:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.203.114@tcp 00000400:00000200:3.0:1787827022.069861:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827022.069863:0:2385:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000010:3.0:1787827022.069867:0:2386:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff985602ad9c08. 00000400:00000200:2.0:1787827022.069870:0:2385:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.203.114@tcp local destination 00000400:00000010:3.0:1787827022.069873:0:2386:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985609573ee0. 00000400:00000200:2.0:1787827022.069875:0:2385:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000200:3.0:1787827022.069878:0:2386:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1874669228030976, portal 4 00000400:00000200:2.0:1787827022.069881:0:2385:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.203.14@tcp 00000100:00000001:3.0:1787827022.069883:0:2386:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000400:00000200:2.0:1787827022.069886:0:2385:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.203.14@tcp lpni_is_preferred = 0 00000100:00000001:3.0:1787827022.069887:0:2386:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446630094203881344 : -113979505670272 : ffff9856140d8780) 00000400:00000200:2.0:1787827022.069892:0:2385:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.203.114@tcp 00000100:00000040:3.0:1787827022.069897:0:2386:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9856140d8780 x1874669228030976/t0(0) o400->lustre-OST0001-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827038 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1787827022.069898:0:2385:0:(lib-move.c:1745:lnet_handle_send()) 192.168.203.14@tcp NI seq info: [21876:21876:254:4294967295] 192.168.203.114@tcp LPNI seq info [21876:21876:6:4294967295] 00000400:00000200:2.0:1787827022.069910:0:2385:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827022.069919:0:2385:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp:) -> 192.168.203.114@tcp(192.168.203.114@tcp:192.168.203.114@tcp) : PUT try# 0 00000100:00000001:3.0:1787827022.069924:0:2386:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000800:00000200:2.0:1787827022.069925:0:2385:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.203.114@tcp 00000100:00000040:3.0:1787827022.069930:0:2386:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.203.114@tcp 00000800:00000010:2.0:1787827022.069932:0:2385:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff985615779d00. 00000400:00000010:3.0:1787827022.069934:0:2386:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985609573aa0. 00000800:00000200:2.0:1787827022.069940:0:2385:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff985606751480] -> 12345-192.168.203.114@tcp (5) 00000100:00000200:3.0:1787827022.069940:0:2386:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 28, xid 1874669228030976, offset 0 00000800:00000200:2.0:1787827022.069948:0:2385:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.203.114@tcp ip 192.168.203.114:988 00000400:00000200:3.0:1787827022.069951:0:2386:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.203.114@tcp 00000800:00000200:2.0:1787827022.069953:0:2385:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff985615779d00 type 1, nob 320 niov 1 nkiov 1 00000400:00000200:3.0:1787827022.069958:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000001:2.0:1787827022.069959:0:2385:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1787827022.069965:0:2385:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff985602bc9800: using a timeout of 38055 usecs (39 jiffies) 00000400:00000200:3.0:1787827022.069966:0:2386:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.203.114@tcp local destination 00000400:00000200:2.0:1787827022.069972:0:2385:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:3.0:1787827022.069973:0:2386:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:2.0:1787827022.069977:0:2385:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff985602bc9800: PM QoS new deadline estimation for cpu 2 is 4297840168 00000400:00000200:3.0:1787827022.069979:0:2386:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.203.14@tcp 00000100:00000001:2.0:1787827022.069983:0:2385:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff985602bc9800: PM QoS request active for cpu 2, keep current deadline 4297850108 00000400:00000200:3.0:1787827022.069985:0:2386:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.203.14@tcp lpni_is_preferred = 0 00000100:00000001:2.0:1787827022.069988:0:2385:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff985602bc9800: PM QoS new deadline estimation for cpu 3 is 4297840168 00000400:00000200:3.0:1787827022.069991:0:2386:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.203.114@tcp 00000100:00000001:2.0:1787827022.069994:0:2385:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff985602bc9800: PM QoS request active for cpu 3, keep current deadline 4297850108 00000400:00000200:3.0:1787827022.069997:0:2386:0:(lib-move.c:1745:lnet_handle_send()) 192.168.203.14@tcp NI seq info: [21877:21877:253:4294967295] 192.168.203.114@tcp LPNI seq info [21877:21877:5:4294967295] 00000100:00000001:2.0:1787827022.069999:0:2385:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1787827022.070004:0:2385:0:(client.c:1941:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1787827022.070008:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000001:2.0:1787827022.070009:0:2385:0:(client.c:1813:ptlrpc_send_new_req()) Process entered 00000100:00000040:2.0:1787827022.070017:0:2385:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9856140db0c0 x1874669228030848/t0(0) o400->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1787827022.070021:0:2386:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp:) -> 192.168.203.114@tcp(192.168.203.114@tcp:192.168.203.114@tcp) : PUT try# 0 00000800:00000200:3.0:1787827022.070033:0:2386:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.203.114@tcp 00000100:00000001:2.0:1787827022.070036:0:2385:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1787827022.070039:0:2385:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:3.0:1787827022.070039:0:2386:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff985609025a00. 02000000:00000001:2.0:1787827022.070045:0:2385:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 00000800:00000200:3.0:1787827022.070046:0:2386:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff985606751480] -> 12345-192.168.203.114@tcp (5) 02000000:00000001:2.0:1787827022.070048:0:2385:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1787827022.070053:0:2385:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1787827022.070056:0:2386:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.203.114@tcp ip 192.168.203.114:988 00000100:00100000:2.0:1787827022.070060:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669228030848:192.168.203.114@tcp:400:kworker.0 00000800:00000200:3.0:1787827022.070061:0:2386:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff985609025a00 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:2.0:1787827022.070068:0:2385:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:3.0:1787827022.070070:0:2386:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1787827022.070073:0:2385:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 00000100:00000040:3.0:1787827022.070074:0:2386:0:(niobuf.c:635:kick_cpu_latency()) lustre-OST0001-osc-ffff985602bc9800: using a timeout of 13856 usecs (14 jiffies) 02000000:00000001:2.0:1787827022.070075:0:2385:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1787827022.070079:0:2385:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000200:3.0:1787827022.070082:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 02000000:00000010:2.0:1787827022.070083:0:2385:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff985611860000. 02000000:00000001:2.0:1787827022.070087:0:2385:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827022.070088:0:2386:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0001-osc-ffff985602bc9800: PM QoS new deadline estimation for cpu 2 is 4297840143 00000400:00000200:2.0:1787827022.070091:0:2385:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000001:3.0:1787827022.070094:0:2386:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0001-osc-ffff985602bc9800: PM QoS request active for cpu 2, keep current deadline 4297850108 00000400:00000010:2.0:1787827022.070095:0:2385:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff98561080ef78. 00000100:00000001:3.0:1787827022.070101:0:2386:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0001-osc-ffff985602bc9800: PM QoS new deadline estimation for cpu 3 is 4297840143 00000400:00000010:2.0:1787827022.070102:0:2385:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985610afa880. 00000100:00000001:3.0:1787827022.070105:0:2386:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0001-osc-ffff985602bc9800: PM QoS request active for cpu 3, keep current deadline 4297850108 00000100:00000200:2.0:1787827022.070108:0:2385:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1874669228030848, portal 4 00000100:00000001:3.0:1787827022.070109:0:2386:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1787827022.070111:0:2385:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1787827022.070113:0:2386:0:(client.c:1941:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1787827022.070115:0:2385:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446630094203891904 : -113979505659712 : ffff9856140db0c0) 00000100:00000001:3.0:1787827022.070118:0:2386:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827022.070121:0:2386:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.070123:0:2385:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9856140db0c0 x1874669228030848/t0(0) o400->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827038 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827022.070126:0:2386:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1787827022.070128:0:2386:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1787827022.070133:0:2386:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1787827022.070135:0:2386:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1787827022.070137:0:2385:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:3.0:1787827022.070139:0:2386:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1787827022.070143:0:2385:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.203.114@tcp 00000100:00000001:3.0:1787827022.070145:0:2386:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1787827022.070148:0:2385:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985610afa440. 00000100:00000200:2.0:1787827022.070152:0:2385:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 28, xid 1874669228030848, offset 0 00000400:00000200:2.0:1787827022.070157:0:2385:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.203.114@tcp 00000400:00000200:2.0:1787827022.070160:0:2385:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827022.070163:0:2385:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.203.114@tcp local destination 00000400:00000200:2.0:1787827022.070165:0:2385:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:2.0:1787827022.070167:0:2385:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.203.14@tcp 00000400:00000200:2.0:1787827022.070169:0:2385:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.203.14@tcp lpni_is_preferred = 0 00000400:00000200:2.0:1787827022.070170:0:2385:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.203.114@tcp 00000400:00000200:2.0:1787827022.070172:0:2385:0:(lib-move.c:1745:lnet_handle_send()) 192.168.203.14@tcp NI seq info: [21878:21878:252:4294967295] 192.168.203.114@tcp LPNI seq info [21878:21878:4:4294967295] 00000400:00000200:2.0:1787827022.070175:0:2385:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827022.070179:0:2385:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp:) -> 192.168.203.114@tcp(192.168.203.114@tcp:192.168.203.114@tcp) : PUT try# 0 00000800:00000200:2.0:1787827022.070181:0:2385:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.203.114@tcp 00000800:00000010:2.0:1787827022.070184:0:2385:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff985615779100. 00000800:00000200:2.0:1787827022.070187:0:2385:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff985606751480] -> 12345-192.168.203.114@tcp (5) 00000800:00000200:2.0:1787827022.070191:0:2385:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.203.114@tcp ip 192.168.203.114:988 00000800:00000200:2.0:1787827022.070193:0:2385:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff985615779100 type 1, nob 320 niov 1 nkiov 1 00000800:00000200:3.0:1787827022.070195:0:2378:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:2.0:1787827022.070197:0:2385:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1787827022.070202:0:2385:0:(niobuf.c:635:kick_cpu_latency()) lustre-OST0000-osc-ffff985602bc9800: using a timeout of 14613 usecs (15 jiffies) 00000800:00000010:3.0:1787827022.070202:0:2378:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff985609025200. 00000400:00000200:2.0:1787827022.070206:0:2385:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000001:2.0:1787827022.070209:0:2385:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff985602bc9800: PM QoS new deadline estimation for cpu 2 is 4297840144 00000100:00000001:2.0:1787827022.070210:0:2385:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff985602bc9800: PM QoS request active for cpu 2, keep current deadline 4297850108 00000100:00000001:2.0:1787827022.070212:0:2385:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff985602bc9800: PM QoS new deadline estimation for cpu 3 is 4297840144 00000100:00000001:2.0:1787827022.070213:0:2385:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff985602bc9800: PM QoS request active for cpu 3, keep current deadline 4297850108 00000400:00000200:3.0:1787827022.070213:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:2.0:1787827022.070218:0:2385:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1787827022.070220:0:2385:0:(client.c:1941:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1787827022.070222:0:2385:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1787827022.070224:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000100:00000001:2.0:1787827022.070225:0:2385:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1787827022.070229:0:2385:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1787827022.070232:0:2385:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000400:00000200:3.0:1787827022.070232:0:2378:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9856095733b8 00000100:00000001:2.0:1787827022.070234:0:2385:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1787827022.070237:0:2385:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000400:00000010:3.0:1787827022.070237:0:2378:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9856095733b8. 00000100:00000001:2.0:1787827022.070241:0:2385:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1787827022.070244:0:2385:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827022.070244:0:2378:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1787827022.070251:0:2378:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff9856140db840 x1874669228030464/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827038 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827022.070261:0:2378:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1787827022.070267:0:2378:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9856140db840 x1874669228030464/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827038 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827022.070273:0:2378:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827022.070275:0:2378:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1787827022.070287:0:2378:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1787827022.070289:0:2378:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff985609025f00. 00000400:00000200:3.0:1787827022.070292:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1787827022.070294:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000400:00000200:3.0:1787827022.070297:0:2378:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985609573770 00000400:00000010:3.0:1787827022.070298:0:2378:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985609573770. 00000100:00000001:3.0:1787827022.070301:0:2378:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1787827022.070304:0:2378:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff9856140da1c0 x1874669228030720/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827038 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827022.070310:0:2378:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1787827022.070312:0:2378:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9856140da1c0 x1874669228030720/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827038 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827022.070317:0:2378:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827022.070319:0:2378:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1787827022.070324:0:2378:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1787827022.070326:0:2378:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff985615779d00. 00000400:00000200:3.0:1787827022.070330:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1787827022.070333:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000400:00000200:3.0:1787827022.070335:0:2378:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985610afa990 00000400:00000010:3.0:1787827022.070336:0:2378:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985610afa990. 00000100:00000001:3.0:1787827022.070338:0:2378:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1787827022.070342:0:2378:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff9856140dad00 x1874669228030592/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827038 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827022.070347:0:2378:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1787827022.070349:0:2378:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9856140dad00 x1874669228030592/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827038 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827022.070354:0:2378:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827022.070355:0:2378:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1787827022.070362:0:2378:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1787827022.070363:0:2378:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff985609025a00. 00000400:00000200:3.0:1787827022.070365:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1787827022.070368:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000400:00000200:3.0:1787827022.070370:0:2378:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985609573aa0 00000400:00000010:3.0:1787827022.070372:0:2378:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985609573aa0. 00000100:00000001:3.0:1787827022.070373:0:2378:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1787827022.070377:0:2378:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff9856140d8780 x1874669228030976/t0(0) o400->lustre-OST0001-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827038 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827022.070383:0:2378:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1787827022.070386:0:2378:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9856140d8780 x1874669228030976/t0(0) o400->lustre-OST0001-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827038 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827022.070391:0:2378:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827022.070392:0:2378:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1787827022.070451:0:2378:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1787827022.070454:0:2378:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff985615779100. 00000400:00000200:3.0:1787827022.070457:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1787827022.070460:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000400:00000200:3.0:1787827022.070462:0:2378:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985610afa440 00000400:00000010:3.0:1787827022.070464:0:2378:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985610afa440. 00000100:00000001:3.0:1787827022.070466:0:2378:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1787827022.070469:0:2378:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff9856140db0c0 x1874669228030848/t0(0) o400->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827038 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827022.070475:0:2378:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1787827022.070478:0:2378:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9856140db0c0 x1874669228030848/t0(0) o400->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827038 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827022.070484:0:2378:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827022.070485:0:2378:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1787827022.074788:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827022.074807:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1787827022.074811:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827022.074814:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827022.074823:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827022.074830:0:2377:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp) <- 192.168.203.114@tcp : PUT - for me 00000400:00000200:2.0:1787827022.074853:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827022.074857:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827022.074862:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827022.074866:0:2377:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.203.114@tcp of length 224 into portal 25 MB=0x6a90071687a00 00000400:00000200:2.0:1787827022.074871:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827022.074877:0:2377:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.203.114@tcp of length 224/224 into md 0x32235 [1] + 224 00000800:00000001:2.0:1787827022.074884:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827022.074889:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827022.074892:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1787827022.074896:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000100:00000001:2.0:1787827022.074903:0:2377:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1787827022.074911:0:2377:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff9856140db840 x1874669228030464/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827038 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1787827022.074922:0:2377:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9856140db840 x1874669228030464/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827038 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.074939:0:2377:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1787827022.074949:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827022.074953:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1787827022.074955:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827022.074957:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827022.074960:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827022.074963:0:2377:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp) <- 192.168.203.114@tcp : PUT - for me 00000400:00000200:2.0:1787827022.074968:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827022.074971:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827022.074975:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827022.074979:0:2377:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.203.114@tcp of length 224 into portal 10 MB=0x6a90071687a80 00000400:00000200:2.0:1787827022.074982:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827022.074990:0:2377:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.203.114@tcp of length 224/224 into md 0x32255 [1] + 224 00000800:00000001:2.0:1787827022.074994:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827022.074998:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827022.075001:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1787827022.075005:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000100:00000001:2.0:1787827022.075008:0:2377:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1787827022.075013:0:2377:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff9856140dad00 x1874669228030592/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827038 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1787827022.075022:0:2377:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9856140dad00 x1874669228030592/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827038 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827022.075029:0:2386:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1787827022.075040:0:2377:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1787827022.075044:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:3.0:1787827022.075052:0:2386:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000800:00000001:2.0:1787827022.075053:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1787827022.075061:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827022.075066:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:3.0:1787827022.075070:0:2386:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000200:2.0:1787827022.075071:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000001:3.0:1787827022.075077:0:2386:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000200:2.0:1787827022.075086:0:2377:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp) <- 192.168.203.114@tcp : PUT - for me 00000400:00000010:3.0:1787827022.075087:0:2386:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff985602ad93c8. 00000400:00000200:2.0:1787827022.075099:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:3.0:1787827022.075099:0:2386:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985609573088 00000400:00000010:3.0:1787827022.075106:0:2386:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985609573088. 00000400:00000200:2.0:1787827022.075112:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000001:3.0:1787827022.075115:0:2386:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000200:2.0:1787827022.075120:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827022.075135:0:2377:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.203.114@tcp of length 224 into portal 10 MB=0x6a90071687b00 00000100:00000200:3.0:1787827022.075135:0:2386:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff9856140db840 x1874669228030464/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827038 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1787827022.075145:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827022.075149:0:2377:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.203.114@tcp of length 224/224 into md 0x32245 [1] + 224 00000800:00000001:2.0:1787827022.075157:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827022.075165:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827022.075171:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000200:3.0:1787827022.075172:0:2386:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff9856140db840 x1874669228030464/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827038 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1787827022.075186:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000100:00000001:3.0:1787827022.075191:0:2386:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1787827022.075195:0:2386:0:(client.c:3001:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1787827022.075198:0:2386:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:3.0:1787827022.075206:0:2386:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1787827022.075207:0:2377:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:3.0:1787827022.075211:0:2386:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000200:2.0:1787827022.075219:0:2377:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff9856140da1c0 x1874669228030720/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827038 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827022.075219:0:2386:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1787827022.075227:0:2386:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1787827022.075241:0:2386:0:(import.c:2149:obd_at_measure()) add 5 to ffff98561118e460 time=22 v=5 (5 5 5 5) 00000100:00001000:3.0:1787827022.075247:0:2386:0:(import.c:2149:obd_at_measure()) add 1 to ffff98561118e418 time=37 v=5 (1 0 0 0) 00000100:00000001:3.0:1787827022.075250:0:2386:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1787827022.075252:0:2386:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1787827022.075255:0:2386:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1787827022.075258:0:2386:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827022.075261:0:2386:0:(client.c:1797:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1787827022.075269:0:2386:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9856140db840 x1874669228030464/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827038 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827022.075277:0:2386:0:(client.c:2354:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1787827022.075281:0:2386:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1787827022.075283:0:2386:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1787827022.075287:0:2386:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9856140db840 x1874669228030464/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827038 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827022.075295:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:3c946890-15ca-497a-b838-fe1cba61f4dd:2386:1874669228030464:192.168.203.114@tcp:400:kworker.0 00000100:00000001:3.0:1787827022.075301:0:2386:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827022.075304:0:2386:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1787827022.075307:0:2386:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9856140db840 x1874669228030464/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827038 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827022.075314:0:2386:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1787827022.075317:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1787827022.075321:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff98561288c600. 02000000:00000001:3.0:1787827022.075324:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1787827022.075328:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:3.0:1787827022.075333:0:2386:0:(genops.c:1223:class_import_put()) import ffff98561118e000 refcount=3 obd=MGC192.168.203.114@tcp 00000020:00000001:3.0:1787827022.075337:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 00000100:00000040:2.0:1787827022.075342:0:2377:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9856140da1c0 x1874669228030720/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827038 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 02000000:00000010:3.0:1787827022.075343:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff985615779000. 02000000:00000001:3.0:1787827022.075353:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1787827022.075358:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000001:2.0:1787827022.075360:0:2377:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000010:3.0:1787827022.075363:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff9856140db840. 00000800:00000001:2.0:1787827022.075370:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:3.0:1787827022.075370:0:2386:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1787827022.075372:0:2386:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1787827022.075378:0:2386:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1787827022.075379:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1787827022.075381:0:2386:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000001:2.0:1787827022.075385:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:3.0:1787827022.075385:0:2386:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000400:00000001:3.0:1787827022.075393:0:2386:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000800:00000001:2.0:1787827022.075394:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:3.0:1787827022.075397:0:2386:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1787827022.075401:0:2386:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff985602ad9790. 00000400:00000200:2.0:1787827022.075403:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:3.0:1787827022.075408:0:2386:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985609573bb0 00000400:00000200:2.0:1787827022.075413:0:2377:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp) <- 192.168.203.114@tcp : PUT - for me 00000400:00000200:2.0:1787827022.075418:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827022.075422:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000010:3.0:1787827022.075427:0:2386:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985609573bb0. 00000400:00000200:2.0:1787827022.075434:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000001:3.0:1787827022.075436:0:2386:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000200:2.0:1787827022.075442:0:2377:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.203.114@tcp of length 224 into portal 4 MB=0x6a90071687c00 00000100:00000200:3.0:1787827022.075446:0:2386:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff9856140da1c0 x1874669228030720/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827038 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1787827022.075455:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827022.075474:0:2377:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 4 from 12345-192.168.203.114@tcp of length 224/224 into md 0x32265 [1] + 224 00000100:00000200:3.0:1787827022.075477:0:2386:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff9856140da1c0 x1874669228030720/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827038 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1787827022.075488:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:3.0:1787827022.075492:0:2386:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1787827022.075496:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1787827022.075496:0:2386:0:(client.c:3001:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1787827022.075503:0:2386:0:(client.c:1593:after_reply()) Process entered 00000400:00000200:2.0:1787827022.075504:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1787827022.075510:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000100:00000001:2.0:1787827022.075514:0:2377:0:(events.c:78:reply_in_callback()) Process entered 02000000:00000001:3.0:1787827022.075517:0:2386:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000200:2.0:1787827022.075522:0:2377:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff9856140d8780 x1874669228030976/t0(0) o400->lustre-OST0001-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827038 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827022.075522:0:2386:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1787827022.075526:0:2386:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1787827022.075531:0:2386:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1787827022.075545:0:2386:0:(import.c:2149:obd_at_measure()) add 5 to ffff985611185c60 time=22 v=5 (5 5 5 5) 00000100:00000040:2.0:1787827022.075547:0:2377:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9856140d8780 x1874669228030976/t0(0) o400->lustre-OST0001-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827038 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00001000:3.0:1787827022.075558:0:2386:0:(import.c:2149:obd_at_measure()) add 1 to ffff985611185c18 time=22 v=5 (1 1 1 1) 00000100:00000001:3.0:1787827022.075574:0:2386:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1787827022.075576:0:2377:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1787827022.075578:0:2386:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1787827022.075583:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00010000:00000001:3.0:1787827022.075584:0:2386:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1787827022.075595:0:2386:0:(ldlm_request.c:1703:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1787827022.075596:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1787827022.075605:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:3.0:1787827022.075607:0:2386:0:(client.c:3109:ptlrpc_free_committed()) Process entered 00000800:00000001:2.0:1787827022.075610:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1787827022.075613:0:2386:0:(client.c:3121:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff985602bc9800: skip recheck: last_committed 0 00000100:00000001:3.0:1787827022.075618:0:2386:0:(client.c:3122:ptlrpc_free_committed()) Process leaving 00000400:00000200:2.0:1787827022.075619:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000001:3.0:1787827022.075623:0:2386:0:(client.c:1797:after_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1787827022.075632:0:2377:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp) <- 192.168.203.114@tcp : PUT - for me 00000100:00000040:3.0:1787827022.075636:0:2386:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9856140da1c0 x1874669228030720/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827038 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1787827022.075647:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827022.075660:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827022.075673:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000001:3.0:1787827022.075685:0:2386:0:(client.c:2354:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000400:00000200:2.0:1787827022.075686:0:2377:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.203.114@tcp of length 224 into portal 4 MB=0x6a90071687b80 00000100:00000001:3.0:1787827022.075733:0:2386:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1787827022.075735:0:2386:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1787827022.075743:0:2386:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9856140da1c0 x1874669228030720/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827038 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1787827022.075754:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827022.075769:0:2377:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 4 from 12345-192.168.203.114@tcp of length 224/224 into md 0x32275 [1] + 224 00000100:00100000:3.0:1787827022.075772:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228030720:192.168.203.114@tcp:400:kworker.0 00000800:00000001:2.0:1787827022.075779:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:3.0:1787827022.075782:0:2386:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827022.075787:0:2386:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1787827022.075792:0:2386:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9856140da1c0 x1874669228030720/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827038 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1787827022.075795:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827022.075806:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:3.0:1787827022.075812:0:2386:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1787827022.075816:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 00000400:00000200:2.0:1787827022.075821:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 02000000:00000010:3.0:1787827022.075821:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff98561288d200. 00000100:00000001:2.0:1787827022.075830:0:2377:0:(events.c:78:reply_in_callback()) Process entered 02000000:00000001:3.0:1787827022.075845:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000200:2.0:1787827022.075848:0:2377:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff9856140db0c0 x1874669228030848/t0(0) o400->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827038 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000020:00000001:3.0:1787827022.075850:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:3.0:1787827022.075852:0:2386:0:(genops.c:1223:class_import_put()) import ffff985611185800 refcount=3 obd=lustre-MDT0001-mdc-ffff985602bc9800 00000020:00000001:3.0:1787827022.075858:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:3.0:1787827022.075862:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff985615779500. 02000000:00000001:3.0:1787827022.075871:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1787827022.075875:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1787827022.075878:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff9856140da1c0. 00000100:00000040:2.0:1787827022.075880:0:2377:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9856140db0c0 x1874669228030848/t0(0) o400->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827038 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827022.075885:0:2386:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1787827022.075889:0:2386:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1787827022.075895:0:2386:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1787827022.075899:0:2377:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1787827022.075901:0:2386:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1787827022.075905:0:2386:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000800:00000001:2.0:1787827022.075911:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000400:00000001:3.0:1787827022.075912:0:2386:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1787827022.075917:0:2386:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000800:00000001:2.0:1787827022.075919:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:3.0:1787827022.075920:0:2386:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff985602ad9c08. 00000400:00000200:3.0:1787827022.075927:0:2386:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985609573ee0 00000400:00000010:3.0:1787827022.075929:0:2386:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985609573ee0. 00000100:00000001:3.0:1787827022.075932:0:2386:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1787827022.075939:0:2386:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff9856140d8780 x1874669228030976/t0(0) o400->lustre-OST0001-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827038 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.075942:0:2385:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1787827022.075947:0:2385:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000200:3.0:1787827022.075956:0:2386:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff9856140d8780 x1874669228030976/t0(0) o400->lustre-OST0001-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827038 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000400:00000001:2.0:1787827022.075957:0:2385:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1787827022.075960:0:2385:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1787827022.075965:0:2385:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff98561080ee70. 00000100:00000001:3.0:1787827022.075976:0:2386:0:(events.c:163:reply_in_callback()) Process leaving 00000400:00000200:2.0:1787827022.075977:0:2385:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985610afa220 00000100:00000001:3.0:1787827022.075980:0:2386:0:(client.c:3001:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:2.0:1787827022.075983:0:2385:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985610afa220. 00000100:00000001:3.0:1787827022.075986:0:2386:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:3.0:1787827022.075990:0:2386:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1787827022.075991:0:2385:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:3.0:1787827022.075993:0:2386:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1787827022.075996:0:2386:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:2.0:1787827022.076001:0:2385:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff9856140dad00 x1874669228030592/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827038 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1787827022.076002:0:2386:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1787827022.076012:0:2386:0:(import.c:2149:obd_at_measure()) add 5 to ffff9856110b1460 time=100 v=5 (5 5 5 5) 00000100:00001000:3.0:1787827022.076025:0:2386:0:(import.c:2149:obd_at_measure()) add 1 to ffff9856110b1418 time=151 v=5 (1 1 1 1) 00000100:00000200:2.0:1787827022.076032:0:2385:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff9856140dad00 x1874669228030592/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827038 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827022.076033:0:2386:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1787827022.076035:0:2386:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1787827022.076042:0:2386:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1787827022.076049:0:2386:0:(ldlm_request.c:1723:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1787827022.076051:0:2385:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1787827022.076055:0:2386:0:(client.c:3109:ptlrpc_free_committed()) Process entered 00000100:00000001:2.0:1787827022.076057:0:2385:0:(client.c:3001:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1787827022.076059:0:2386:0:(client.c:3121:ptlrpc_free_committed()) lustre-OST0001-osc-ffff985602bc9800: skip recheck: last_committed 4294967636 00000100:00000001:2.0:1787827022.076062:0:2385:0:(client.c:1593:after_reply()) Process entered 00000100:00000001:3.0:1787827022.076064:0:2386:0:(client.c:3122:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1787827022.076069:0:2386:0:(client.c:1797:after_reply()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1787827022.076070:0:2385:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1787827022.076076:0:2385:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1787827022.076082:0:2385:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1787827022.076083:0:2386:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9856140d8780 x1874669228030976/t0(0) o400->lustre-OST0001-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827038 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1787827022.076089:0:2385:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827022.076100:0:2386:0:(client.c:2354:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1787827022.076103:0:2386:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00001000:2.0:1787827022.076104:0:2385:0:(import.c:2149:obd_at_measure()) add 5 to ffff985611182460 time=100 v=5 (5 5 5 5) 00000100:00000001:3.0:1787827022.076106:0:2386:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:2.0:1787827022.076116:0:2385:0:(import.c:2149:obd_at_measure()) add 1 to ffff985611182418 time=36 v=5 (1 0 0 0) 00000100:00000040:3.0:1787827022.076119:0:2386:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9856140d8780 x1874669228030976/t0(0) o400->lustre-OST0001-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827038 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.076125:0:2385:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1787827022.076132:0:2385:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1787827022.076139:0:2385:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00100000:3.0:1787827022.076141:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228030976:192.168.203.114@tcp:400:kworker.0 00010000:00000001:2.0:1787827022.076146:0:2385:0:(ldlm_request.c:1703:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827022.076149:0:2386:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1787827022.076153:0:2385:0:(client.c:3109:ptlrpc_free_committed()) Process entered 00000100:00000001:3.0:1787827022.076154:0:2386:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00100000:2.0:1787827022.076157:0:2385:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 219043334508 gen 9 00000100:00000040:3.0:1787827022.076163:0:2386:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9856140d8780 x1874669228030976/t0(0) o400->lustre-OST0001-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827038 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1787827022.076171:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985608408f00 x1874669227875584/t219043334107(219043334107) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 528/448 e 0 to 0 dl 1787827030 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827022.076176:0:2386:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1787827022.076180:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1787827022.076184:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff98561288d600. 02000000:00000001:3.0:1787827022.076189:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1787827022.076191:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:3.0:1787827022.076192:0:2386:0:(genops.c:1223:class_import_put()) import ffff9856110b1000 refcount=2 obd=lustre-OST0001-osc-ffff985602bc9800 00000100:00000001:2.0:1787827022.076195:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000020:00000001:3.0:1787827022.076197:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:3.0:1787827022.076201:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff985615779a00. 00000100:00000040:2.0:1787827022.076204:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985608408f00 x1874669227875584/t219043334107(219043334107) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 528/448 e 0 to 0 dl 1787827030 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1787827022.076207:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1787827022.076210:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1787827022.076213:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff9856140d8780. 00000100:00000001:3.0:1787827022.076223:0:2386:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1787827022.076225:0:2386:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1787827022.076232:0:2386:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1787827022.076245:0:2386:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1787827022.076248:0:2386:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827022.076258:0:2386:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1787827022.076260:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 00000100:00000001:3.0:1787827022.076262:0:2386:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1787827022.076265:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.076270:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985607cde400. 02000000:00000001:2.0:1787827022.076277:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.076280:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.076281:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=405 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.076285:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.076287:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 1024 at ffff985607cdc400. 02000000:00000001:2.0:1787827022.076291:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.076295:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.076297:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985608408f00. 00000100:00000001:2.0:1787827022.076300:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.076302:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.076308:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985608409e00 x1874669227876096/t219043334108(219043334108) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827030 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.076315:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.076318:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985608409e00 x1874669227876096/t219043334108(219043334108) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827030 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.076324:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.076325:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.076327:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985607cde800. 02000000:00000001:2.0:1787827022.076329:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.076330:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.076331:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=404 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.076333:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.076335:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607c24e00. 02000000:00000001:2.0:1787827022.076337:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.076338:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.076339:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985608409e00. 00000100:00000001:2.0:1787827022.076342:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.076343:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.076347:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9856086af840 x1874669227877120/t219043334109(219043334109) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.076360:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.076362:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9856086af840 x1874669227877120/t219043334109(219043334109) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.076368:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.076369:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.076370:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985608655c00. 02000000:00000001:2.0:1787827022.076373:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.076374:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.076375:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=403 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.076377:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.076378:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985610d72e00. 02000000:00000001:2.0:1787827022.076380:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.076381:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.076382:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff9856086af840. 00000100:00000001:2.0:1787827022.076384:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.076385:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.076389:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9856086ae940 x1874669227877504/t219043334110(219043334110) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.076395:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.076398:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9856086ae940 x1874669227877504/t219043334110(219043334110) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.076404:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.076405:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.076406:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985608655800. 02000000:00000001:2.0:1787827022.076408:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.076409:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.076410:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=402 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.076412:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.076413:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985610d73e00. 02000000:00000001:2.0:1787827022.076415:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.076416:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.076418:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff9856086ae940. 00000100:00000001:2.0:1787827022.076420:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.076421:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.076425:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9856086afc00 x1874669227877888/t219043334111(219043334111) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.076432:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.076435:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9856086afc00 x1874669227877888/t219043334111(219043334111) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.076442:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.076443:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.076444:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985608656c00. 02000000:00000001:2.0:1787827022.076447:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.076448:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.076449:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=401 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.076451:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.076453:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985610d72400. 02000000:00000001:2.0:1787827022.076455:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.076456:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.076457:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff9856086afc00. 00000100:00000001:2.0:1787827022.076459:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.076461:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.076466:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985608409680 x1874669227878272/t219043334112(219043334112) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.076472:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.076475:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985608409680 x1874669227878272/t219043334112(219043334112) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.076481:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.076482:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.076483:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985607cdf400. 02000000:00000001:2.0:1787827022.076486:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.076487:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.076488:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=400 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.076490:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.076492:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985612a4c000. 02000000:00000001:2.0:1787827022.076494:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.076495:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.076497:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985608409680. 00000100:00000001:2.0:1787827022.076499:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.076500:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.076504:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9856084083c0 x1874669227878656/t219043334113(219043334113) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.076510:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.076512:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9856084083c0 x1874669227878656/t219043334113(219043334113) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.076525:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.076526:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.076528:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985607cdc000. 02000000:00000001:2.0:1787827022.076531:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.076532:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.076533:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=399 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.076535:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.076537:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607ca8e00. 02000000:00000001:2.0:1787827022.076539:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.076542:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.076544:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff9856084083c0. 00000100:00000001:2.0:1787827022.076549:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.076551:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.076556:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9856086ac000 x1874669227879040/t219043334114(219043334114) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.076563:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.076566:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9856086ac000 x1874669227879040/t219043334114(219043334114) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.076572:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.076573:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.076574:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985608654000. 02000000:00000001:2.0:1787827022.076576:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.076577:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.076578:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=398 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.076580:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.076582:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985612073800. 02000000:00000001:2.0:1787827022.076584:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.076585:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.076586:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff9856086ac000. 00000100:00000001:2.0:1787827022.076588:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.076590:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.076594:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff98560840b480 x1874669227879424/t219043334115(219043334115) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.076600:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.076603:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff98560840b480 x1874669227879424/t219043334115(219043334115) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.076609:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.076611:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.076612:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985607cdc800. 02000000:00000001:2.0:1787827022.076614:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.076615:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.076616:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=397 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.076618:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.076619:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607ca8c00. 02000000:00000001:2.0:1787827022.076621:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.076622:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.076623:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff98560840b480. 00000100:00000001:2.0:1787827022.076625:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.076626:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.076630:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985608409a40 x1874669227879808/t219043334116(219043334116) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.076637:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.076639:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985608409a40 x1874669227879808/t219043334116(219043334116) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.076645:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.076646:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.076647:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985607cdf000. 02000000:00000001:2.0:1787827022.076649:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.076651:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.076652:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=396 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.076654:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.076655:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607ca9000. 02000000:00000001:2.0:1787827022.076658:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.076659:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.076660:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985608409a40. 00000100:00000001:2.0:1787827022.076662:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.076664:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.076667:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff98560840b0c0 x1874669227880192/t219043334117(219043334117) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.076674:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.076677:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff98560840b0c0 x1874669227880192/t219043334117(219043334117) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.076683:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.076684:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.076685:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985607cdcc00. 02000000:00000001:2.0:1787827022.076687:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.076689:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.076714:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=395 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.076717:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.076718:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607ca9400. 02000000:00000001:2.0:1787827022.076721:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.076722:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.076723:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff98560840b0c0. 00000100:00000001:2.0:1787827022.076726:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.076727:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.076732:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985608408780 x1874669227880576/t219043334118(219043334118) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.076738:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.076741:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985608408780 x1874669227880576/t219043334118(219043334118) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.076747:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.076748:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.076750:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985607cdf800. 02000000:00000001:2.0:1787827022.076752:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.076753:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.076754:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=394 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.076756:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.076758:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607ca9600. 02000000:00000001:2.0:1787827022.076760:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.076761:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.076762:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985608408780. 00000100:00000001:2.0:1787827022.076764:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.076765:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.076769:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff98560840ad00 x1874669227880960/t219043334119(219043334119) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.076775:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.076778:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff98560840ad00 x1874669227880960/t219043334119(219043334119) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.076784:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.076785:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.076787:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985607cdd400. 02000000:00000001:2.0:1787827022.076788:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.076790:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.076791:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=393 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.076793:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.076794:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607ca8800. 02000000:00000001:2.0:1787827022.076796:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.076797:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.076798:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff98560840ad00. 00000100:00000001:2.0:1787827022.076800:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.076802:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.076806:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9856084092c0 x1874669227881344/t219043334120(219043334120) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.076818:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.076821:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9856084092c0 x1874669227881344/t219043334120(219043334120) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.076827:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.076828:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.076830:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985607cdfc00. 02000000:00000001:2.0:1787827022.076844:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.076846:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.076847:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=392 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.076849:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.076851:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607ca8200. 02000000:00000001:2.0:1787827022.076853:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.076854:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.076856:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff9856084092c0. 00000100:00000001:2.0:1787827022.076858:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.076859:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.076863:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff98560840a1c0 x1874669227881728/t219043334121(219043334121) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.076869:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.076872:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff98560840a1c0 x1874669227881728/t219043334121(219043334121) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.076878:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.076879:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.076881:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985607cde000. 02000000:00000001:2.0:1787827022.076883:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.076884:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.076885:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=391 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.076887:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.076889:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607ca8a00. 02000000:00000001:2.0:1787827022.076891:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.076892:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.076893:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff98560840a1c0. 00000100:00000001:2.0:1787827022.076895:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.076896:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.076900:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff98560840a940 x1874669227882112/t219043334122(219043334122) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.076906:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.076909:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff98560840a940 x1874669227882112/t219043334122(219043334122) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.076915:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.076916:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.076917:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985607cdec00. 02000000:00000001:2.0:1787827022.076919:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.076921:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.076922:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=390 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.076923:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.076925:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607ca8000. 02000000:00000001:2.0:1787827022.076926:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.076927:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.076929:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff98560840a940. 00000100:00000001:2.0:1787827022.076931:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.076932:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.076936:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985608408b40 x1874669227882496/t219043334123(219043334123) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.076942:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.076945:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985608408b40 x1874669227882496/t219043334123(219043334123) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.076950:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.076951:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.076953:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985607cdd800. 02000000:00000001:2.0:1787827022.076955:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.076956:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.076957:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=389 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.076959:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.076961:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607ca9a00. 02000000:00000001:2.0:1787827022.076963:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.076964:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.076966:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985608408b40. 00000100:00000001:2.0:1787827022.076968:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.076969:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.076973:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9856086acb40 x1874669227882880/t219043334124(219043334124) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.076980:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.076982:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9856086acb40 x1874669227882880/t219043334124(219043334124) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.076989:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.076990:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.076992:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985608654800. 02000000:00000001:2.0:1787827022.076994:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.076995:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.076996:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=388 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.076998:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.076999:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985610db8400. 02000000:00000001:2.0:1787827022.077001:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.077002:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.077003:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff9856086acb40. 00000100:00000001:2.0:1787827022.077005:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.077006:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.077010:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9856086ac780 x1874669227883264/t219043334125(219043334125) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.077016:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.077019:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9856086ac780 x1874669227883264/t219043334125(219043334125) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.077025:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.077026:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.077027:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985608657000. 02000000:00000001:2.0:1787827022.077029:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.077030:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.077031:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=387 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.077033:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.077034:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985610db9800. 02000000:00000001:2.0:1787827022.077037:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.077038:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.077039:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff9856086ac780. 00000100:00000001:2.0:1787827022.077040:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.077042:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.077045:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9856086af0c0 x1874669227883648/t219043334126(219043334126) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.077051:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.077053:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9856086af0c0 x1874669227883648/t219043334126(219043334126) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.077058:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.077059:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.077061:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985608654c00. 02000000:00000001:2.0:1787827022.077062:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.077063:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.077064:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=386 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.077066:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.077067:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985610db8000. 02000000:00000001:2.0:1787827022.077069:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.077070:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.077071:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff9856086af0c0. 00000100:00000001:2.0:1787827022.077073:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.077074:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.077077:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9856086ada40 x1874669227884032/t219043334127(219043334127) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.077083:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.077086:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9856086ada40 x1874669227884032/t219043334127(219043334127) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.077092:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.077094:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.077095:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985608657800. 02000000:00000001:2.0:1787827022.077097:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.077099:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.077100:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=385 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.077102:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.077103:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985610db8e00. 02000000:00000001:2.0:1787827022.077105:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.077106:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.077108:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff9856086ada40. 00000100:00000001:2.0:1787827022.077110:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.077111:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.077116:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9856086af480 x1874669227884416/t219043334128(219043334128) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.077123:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.077125:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9856086af480 x1874669227884416/t219043334128(219043334128) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.077131:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.077132:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.077134:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985608657400. 02000000:00000001:2.0:1787827022.077136:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.077137:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.077138:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=384 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.077140:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.077141:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985610db9e00. 02000000:00000001:2.0:1787827022.077143:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.077144:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.077145:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff9856086af480. 00000100:00000001:2.0:1787827022.077147:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.077148:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.077152:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9856086ac3c0 x1874669227884800/t219043334129(219043334129) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.077158:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.077161:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9856086ac3c0 x1874669227884800/t219043334129(219043334129) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.077167:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.077168:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.077170:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985608655000. 02000000:00000001:2.0:1787827022.077172:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.077173:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.077174:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=383 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.077175:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.077176:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985610db8800. 02000000:00000001:2.0:1787827022.077178:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.077179:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.077180:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff9856086ac3c0. 00000100:00000001:2.0:1787827022.077182:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.077183:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.077187:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9856086ad680 x1874669227885184/t219043334130(219043334130) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.077193:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.077196:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9856086ad680 x1874669227885184/t219043334130(219043334130) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.077202:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.077203:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.077204:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985608656800. 02000000:00000001:2.0:1787827022.077206:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.077207:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.077208:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=382 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.077210:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.077211:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985610db9a00. 02000000:00000001:2.0:1787827022.077213:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.077214:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.077215:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff9856086ad680. 00000100:00000001:2.0:1787827022.077217:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.077217:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.077221:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff98560840b840 x1874669227885568/t219043334131(219043334131) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.077233:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.077236:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff98560840b840 x1874669227885568/t219043334131(219043334131) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.077242:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.077244:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.077245:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985607cdd000. 02000000:00000001:2.0:1787827022.077247:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.077248:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.077249:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=381 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.077251:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.077252:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607ca9c00. 02000000:00000001:2.0:1787827022.077254:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.077255:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.077257:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff98560840b840. 00000100:00000001:2.0:1787827022.077259:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.077260:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.077264:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985608408000 x1874669227885952/t219043334132(219043334132) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.077270:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.077273:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985608408000 x1874669227885952/t219043334132(219043334132) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.077279:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.077280:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.077281:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985607cddc00. 02000000:00000001:2.0:1787827022.077283:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.077283:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.077284:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=380 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.077286:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.077287:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607ca9800. 02000000:00000001:2.0:1787827022.077289:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.077289:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.077291:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985608408000. 00000100:00000001:2.0:1787827022.077293:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.077294:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.077298:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff98560840a580 x1874669227886336/t219043334133(219043334133) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.077304:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.077307:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff98560840a580 x1874669227886336/t219043334133(219043334133) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.077313:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.077314:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.077316:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985607c28c00. 02000000:00000001:2.0:1787827022.077318:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.077319:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.077321:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=379 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.077323:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.077324:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607ca9200. 02000000:00000001:2.0:1787827022.077326:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.077327:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.077328:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff98560840a580. 00000100:00000001:2.0:1787827022.077330:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.077331:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.077336:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff98560840bc00 x1874669227886720/t219043334134(219043334134) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.077343:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.077346:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff98560840bc00 x1874669227886720/t219043334134(219043334134) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.077352:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.077353:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.077354:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985607c2b000. 02000000:00000001:2.0:1787827022.077356:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.077358:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.077359:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=378 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.077361:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.077362:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff98561119c000. 02000000:00000001:2.0:1787827022.077365:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.077366:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.077367:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff98560840bc00. 00000100:00000001:2.0:1787827022.077369:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.077370:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.077374:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607db8b40 x1874669227887104/t219043334135(219043334135) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.077381:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.077384:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607db8b40 x1874669227887104/t219043334135(219043334135) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.077390:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.077391:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.077392:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985607c28800. 02000000:00000001:2.0:1787827022.077395:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.077396:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.077397:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=377 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.077399:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.077400:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff98561119ca00. 02000000:00000001:2.0:1787827022.077402:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.077403:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.077405:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607db8b40. 00000100:00000001:2.0:1787827022.077407:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.077408:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.077412:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607dbbc00 x1874669227887488/t219043334136(219043334136) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.077418:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.077421:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607dbbc00 x1874669227887488/t219043334136(219043334136) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.077427:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.077428:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.077429:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985607c28000. 02000000:00000001:2.0:1787827022.077432:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.077434:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.077435:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=376 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.077437:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.077438:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff98561119d400. 02000000:00000001:2.0:1787827022.077440:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.077441:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.077442:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607dbbc00. 00000100:00000001:2.0:1787827022.077445:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.077446:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.077450:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607dbb840 x1874669227887872/t219043334137(219043334137) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.077456:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.077459:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607dbb840 x1874669227887872/t219043334137(219043334137) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.077464:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.077465:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.077467:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985607c2b400. 02000000:00000001:2.0:1787827022.077469:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.077470:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.077471:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=375 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.077474:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.077475:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff98561119c200. 02000000:00000001:2.0:1787827022.077476:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.077478:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.077479:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607dbb840. 00000100:00000001:2.0:1787827022.077481:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.077482:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.077486:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607db8000 x1874669227888256/t219043334138(219043334138) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.077493:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.077495:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607db8000 x1874669227888256/t219043334138(219043334138) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.077501:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.077503:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.077504:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985607c2ac00. 02000000:00000001:2.0:1787827022.077506:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.077507:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.077509:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=374 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.077510:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.077512:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff98561119c600. 02000000:00000001:2.0:1787827022.077513:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.077514:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.077516:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607db8000. 00000100:00000001:2.0:1787827022.077518:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.077519:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.077523:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607db8780 x1874669227888640/t219043334139(219043334139) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.077530:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.077533:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607db8780 x1874669227888640/t219043334139(219043334139) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.077538:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.077540:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.077541:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985607c29800. 02000000:00000001:2.0:1787827022.077543:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.077545:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.077546:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=373 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.077548:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.077549:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff98561119de00. 02000000:00000001:2.0:1787827022.077551:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.077552:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.077553:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607db8780. 00000100:00000001:2.0:1787827022.077555:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.077557:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.077561:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607dbb0c0 x1874669227889024/t219043334140(219043334140) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.077567:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.077569:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607dbb0c0 x1874669227889024/t219043334140(219043334140) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.077576:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.077577:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.077579:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985607c29c00. 02000000:00000001:2.0:1787827022.077580:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.077581:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.077582:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=372 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.077584:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.077585:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff98561119d600. 02000000:00000001:2.0:1787827022.077586:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.077587:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.077588:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607dbb0c0. 00000100:00000001:2.0:1787827022.077590:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.077591:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.077595:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607db9a40 x1874669227889408/t219043334141(219043334141) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.077601:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.077603:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607db9a40 x1874669227889408/t219043334141(219043334141) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.077608:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.077609:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.077610:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985607c29000. 02000000:00000001:2.0:1787827022.077612:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.077613:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.077614:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=371 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.077615:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.077616:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607ca9e00. 02000000:00000001:2.0:1787827022.077618:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.077619:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.077620:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607db9a40. 00000100:00000001:2.0:1787827022.077622:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.077623:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.077627:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607dbb480 x1874669227889792/t219043334142(219043334142) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.077637:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.077641:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607dbb480 x1874669227889792/t219043334142(219043334142) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.077646:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.077647:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.077649:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985607c2a800. 02000000:00000001:2.0:1787827022.077651:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.077652:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.077653:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=370 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.077655:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.077656:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9856039b0e00. 02000000:00000001:2.0:1787827022.077658:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.077659:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.077661:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607dbb480. 00000100:00000001:2.0:1787827022.077663:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.077664:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.077668:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607db83c0 x1874669227890176/t219043334143(219043334143) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.077674:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.077677:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607db83c0 x1874669227890176/t219043334143(219043334143) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.077683:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.077684:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.077686:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985607c2a400. 02000000:00000001:2.0:1787827022.077688:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.077688:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.077689:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=369 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.077714:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.077716:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607f8ca00. 02000000:00000001:2.0:1787827022.077718:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.077720:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.077721:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607db83c0. 00000100:00000001:2.0:1787827022.077724:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.077725:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.077730:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607db9680 x1874669227890560/t219043334144(219043334144) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.077736:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.077739:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607db9680 x1874669227890560/t219043334144(219043334144) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.077745:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.077746:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.077747:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985607c28400. 02000000:00000001:2.0:1787827022.077750:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.077751:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.077752:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=368 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.077754:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.077755:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607f8d600. 02000000:00000001:2.0:1787827022.077757:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.077758:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.077759:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607db9680. 00000100:00000001:2.0:1787827022.077761:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.077762:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.077766:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607dbad00 x1874669227890944/t219043334145(219043334145) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.077773:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.077775:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607dbad00 x1874669227890944/t219043334145(219043334145) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.077781:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.077783:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.077784:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985607c2bc00. 02000000:00000001:2.0:1787827022.077786:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.077787:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.077788:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=367 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.077789:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.077791:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607f8d800. 02000000:00000001:2.0:1787827022.077793:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.077794:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.077795:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607dbad00. 00000100:00000001:2.0:1787827022.077797:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.077799:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.077802:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607db92c0 x1874669227891328/t219043334146(219043334146) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.077807:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.077810:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607db92c0 x1874669227891328/t219043334146(219043334146) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.077816:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.077817:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.077819:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985607c2a000. 02000000:00000001:2.0:1787827022.077821:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.077822:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.077823:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=366 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.077825:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.077826:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607f8dc00. 02000000:00000001:2.0:1787827022.077829:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.077830:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.077831:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607db92c0. 00000100:00000001:2.0:1787827022.077845:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.077846:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.077851:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607db9e00 x1874669227891712/t219043334147(219043334147) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.077858:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.077861:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607db9e00 x1874669227891712/t219043334147(219043334147) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.077867:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.077868:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.077869:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985607c29400. 02000000:00000001:2.0:1787827022.077871:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.077873:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.077874:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=365 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.077876:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.077877:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607f8c400. 02000000:00000001:2.0:1787827022.077879:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.077880:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.077881:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607db9e00. 00000100:00000001:2.0:1787827022.077883:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.077884:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.077888:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607db8f00 x1874669227892096/t219043334148(219043334148) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.077894:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.077896:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607db8f00 x1874669227892096/t219043334148(219043334148) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.077902:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.077903:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.077905:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985607c2b800. 02000000:00000001:2.0:1787827022.077907:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.077908:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.077908:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=364 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.077910:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.077911:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607f8c000. 02000000:00000001:2.0:1787827022.077913:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.077914:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.077915:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607db8f00. 00000100:00000001:2.0:1787827022.077917:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.077919:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.077923:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607dba580 x1874669227892480/t219043334149(219043334149) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.077929:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.077932:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607dba580 x1874669227892480/t219043334149(219043334149) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.077938:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.077939:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.077941:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98560870b000. 02000000:00000001:2.0:1787827022.077966:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.077969:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.077970:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=363 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.077972:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.077974:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607f8c600. 02000000:00000001:2.0:1787827022.077976:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.077977:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.077979:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607dba580. 00000100:00000001:2.0:1787827022.077981:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.077982:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.077988:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607dba1c0 x1874669227892864/t219043334150(219043334150) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.077994:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.077997:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607dba1c0 x1874669227892864/t219043334150(219043334150) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.078003:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.078004:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.078006:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985608708800. 02000000:00000001:2.0:1787827022.078009:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.078010:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.078011:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=362 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.078013:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.078014:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607f8d200. 02000000:00000001:2.0:1787827022.078016:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.078017:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.078019:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607dba1c0. 00000100:00000001:2.0:1787827022.078021:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.078022:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.078026:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607dba940 x1874669227893248/t219043334151(219043334151) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.078032:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.078035:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607dba940 x1874669227893248/t219043334151(219043334151) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.078042:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.078043:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.078044:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985608708000. 02000000:00000001:2.0:1787827022.078046:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.078047:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.078048:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=361 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.078050:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.078052:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607f8c200. 02000000:00000001:2.0:1787827022.078054:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.078055:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.078056:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607dba940. 00000100:00000001:2.0:1787827022.078058:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.078059:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.078064:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985614176580 x1874669227893632/t219043334152(219043334152) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.078071:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.078073:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985614176580 x1874669227893632/t219043334152(219043334152) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.078079:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.078081:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.078082:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98560870b400. 02000000:00000001:2.0:1787827022.078084:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.078086:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.078087:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=360 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.078089:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.078090:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607f8cc00. 02000000:00000001:2.0:1787827022.078092:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.078093:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.078094:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985614176580. 00000100:00000001:2.0:1787827022.078097:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.078098:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.078102:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985614175a40 x1874669227894016/t219043334153(219043334153) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.078113:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.078116:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985614175a40 x1874669227894016/t219043334153(219043334153) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.078122:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.078123:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.078124:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98560870ac00. 02000000:00000001:2.0:1787827022.078127:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.078128:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.078129:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=359 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.078131:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.078132:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607f8de00. 02000000:00000001:2.0:1787827022.078134:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.078135:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.078137:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985614175a40. 00000100:00000001:2.0:1787827022.078141:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.078142:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.078147:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985614174b40 x1874669227894400/t219043334154(219043334154) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.078153:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.078156:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985614174b40 x1874669227894400/t219043334154(219043334154) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.078162:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.078163:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.078165:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985608709800. 02000000:00000001:2.0:1787827022.078167:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.078168:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.078169:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=358 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.078171:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.078173:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607f8c800. 02000000:00000001:2.0:1787827022.078174:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.078175:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.078177:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985614174b40. 00000100:00000001:2.0:1787827022.078179:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.078180:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.078184:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985614177480 x1874669227894784/t219043334155(219043334155) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.078190:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.078193:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985614177480 x1874669227894784/t219043334155(219043334155) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.078199:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.078200:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.078201:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985608709c00. 02000000:00000001:2.0:1787827022.078203:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.078204:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.078205:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=357 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.078207:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.078208:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607f8ce00. 02000000:00000001:2.0:1787827022.078210:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.078211:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.078212:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985614177480. 00000100:00000001:2.0:1787827022.078215:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.078216:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.078220:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985614176d00 x1874669227895168/t219043334156(219043334156) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.078226:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.078229:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985614176d00 x1874669227895168/t219043334156(219043334156) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.078234:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.078235:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.078236:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985608709000. 02000000:00000001:2.0:1787827022.078238:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.078239:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.078240:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=356 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.078242:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.078243:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607f8d400. 02000000:00000001:2.0:1787827022.078245:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.078246:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.078247:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985614176d00. 00000100:00000001:2.0:1787827022.078249:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.078250:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.078254:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985614174000 x1874669227895552/t219043334157(219043334157) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.078259:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.078262:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985614174000 x1874669227895552/t219043334157(219043334157) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.078268:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.078269:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.078271:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98560870a800. 02000000:00000001:2.0:1787827022.078273:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.078274:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.078275:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=355 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.078277:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.078278:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607f8da00. 02000000:00000001:2.0:1787827022.078280:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.078281:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.078282:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985614174000. 00000100:00000001:2.0:1787827022.078285:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.078286:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.078291:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9856141770c0 x1874669227895936/t219043334158(219043334158) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.078298:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.078301:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9856141770c0 x1874669227895936/t219043334158(219043334158) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.078307:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.078308:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.078309:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98560870a400. 02000000:00000001:2.0:1787827022.078311:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.078313:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.078314:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=354 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.078316:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.078317:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff98561119c400. 02000000:00000001:2.0:1787827022.078319:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.078320:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.078321:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff9856141770c0. 00000100:00000001:2.0:1787827022.078323:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.078324:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.078328:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985614176940 x1874669227896320/t219043334159(219043334159) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.078335:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.078338:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985614176940 x1874669227896320/t219043334159(219043334159) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.078344:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.078345:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.078346:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985608708400. 02000000:00000001:2.0:1787827022.078348:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.078349:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.078350:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=353 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.078352:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.078353:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985604c15600. 02000000:00000001:2.0:1787827022.078356:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.078357:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.078358:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985614176940. 00000100:00000001:2.0:1787827022.078360:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.078361:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.078365:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985614174780 x1874669227896704/t219043334160(219043334160) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.078370:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.078373:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985614174780 x1874669227896704/t219043334160(219043334160) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.078378:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.078380:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.078381:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98560870bc00. 02000000:00000001:2.0:1787827022.078383:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.078384:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.078385:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=352 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.078387:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.078388:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985610f2b600. 02000000:00000001:2.0:1787827022.078391:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.078392:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.078393:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985614174780. 00000100:00000001:2.0:1787827022.078394:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.078396:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.078399:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985614177c00 x1874669227897088/t219043334161(219043334161) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.078406:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.078409:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985614177c00 x1874669227897088/t219043334161(219043334161) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.078415:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.078416:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.078418:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98560870a000. 02000000:00000001:2.0:1787827022.078420:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.078421:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.078423:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=351 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.078424:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.078426:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985610f2a000. 02000000:00000001:2.0:1787827022.078428:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.078429:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.078431:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985614177c00. 00000100:00000001:2.0:1787827022.078433:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.078434:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.078439:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9856141743c0 x1874669227897472/t219043334162(219043334162) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.078445:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.078448:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9856141743c0 x1874669227897472/t219043334162(219043334162) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.078454:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.078455:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.078456:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985608709400. 02000000:00000001:2.0:1787827022.078458:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.078459:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.078460:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=350 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.078462:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.078463:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985610f2a400. 02000000:00000001:2.0:1787827022.078464:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.078465:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.078466:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff9856141743c0. 00000100:00000001:2.0:1787827022.078468:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.078469:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.078473:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9856141761c0 x1874669227897856/t219043334163(219043334163) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.078480:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.078483:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9856141761c0 x1874669227897856/t219043334163(219043334163) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.078488:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.078490:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.078492:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98560870b800. 02000000:00000001:2.0:1787827022.078494:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.078495:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.078496:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=349 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.078498:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.078499:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985610f2ae00. 02000000:00000001:2.0:1787827022.078501:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.078502:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.078503:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff9856141761c0. 00000100:00000001:2.0:1787827022.078505:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.078506:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.078510:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985613163480 x1874669227898240/t219043334164(219043334164) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.078522:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.078525:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985613163480 x1874669227898240/t219043334164(219043334164) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.078531:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.078532:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.078533:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985608708c00. 02000000:00000001:2.0:1787827022.078535:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.078536:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.078537:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=348 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.078539:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.078541:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985610f2be00. 02000000:00000001:2.0:1787827022.078543:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.078544:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.078545:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985613163480. 00000100:00000001:2.0:1787827022.078584:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.078585:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.078591:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985613161a40 x1874669227898624/t219043334165(219043334165) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.078598:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.078601:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985613161a40 x1874669227898624/t219043334165(219043334165) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.078607:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.078608:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.078610:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9856085a0000. 02000000:00000001:2.0:1787827022.078613:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.078614:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.078615:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=347 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.078617:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.078619:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985610f2a200. 02000000:00000001:2.0:1787827022.078621:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.078622:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.078623:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985613161a40. 00000100:00000001:2.0:1787827022.078626:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.078627:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.078631:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff98560993b840 x1874669227899008/t219043334166(219043334166) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.078637:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.078640:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff98560993b840 x1874669227899008/t219043334166(219043334166) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.078646:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.078647:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.078648:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9856085a0800. 02000000:00000001:2.0:1787827022.078650:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.078651:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.078653:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=346 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.078655:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.078656:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607f8d000. 02000000:00000001:2.0:1787827022.078658:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.078659:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.078660:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff98560993b840. 00000100:00000001:2.0:1787827022.078662:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.078663:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.078668:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff98560993a580 x1874669227899392/t219043334167(219043334167) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.078674:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.078677:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff98560993a580 x1874669227899392/t219043334167(219043334167) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.078683:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.078684:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.078686:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9856085a3000. 02000000:00000001:2.0:1787827022.078688:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.078689:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.078707:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=345 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.078710:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.078712:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9856057e3400. 02000000:00000001:2.0:1787827022.078717:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.078718:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.078720:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff98560993a580. 00000100:00000001:2.0:1787827022.078723:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.078724:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.078728:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9856099383c0 x1874669227899776/t219043334168(219043334168) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.078735:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.078738:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9856099383c0 x1874669227899776/t219043334168(219043334168) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.078743:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.078745:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.078746:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9856085a0c00. 02000000:00000001:2.0:1787827022.078748:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.078749:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.078750:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=344 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.078752:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.078754:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9856057e2600. 02000000:00000001:2.0:1787827022.078756:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.078757:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.078758:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff9856099383c0. 00000100:00000001:2.0:1787827022.078760:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.078761:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.078765:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985609938b40 x1874669227900160/t219043334169(219043334169) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.078771:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.078774:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985609938b40 x1874669227900160/t219043334169(219043334169) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.078779:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.078780:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.078782:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9856085a3800. 02000000:00000001:2.0:1787827022.078783:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.078785:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.078786:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=343 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.078788:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.078789:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9856057e3a00. 02000000:00000001:2.0:1787827022.078791:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.078792:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.078793:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985609938b40. 00000100:00000001:2.0:1787827022.078795:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.078796:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.078800:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985609939e00 x1874669227900544/t219043334170(219043334170) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.078806:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.078809:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985609939e00 x1874669227900544/t219043334170(219043334170) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.078815:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.078816:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.078817:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9856085a1400. 02000000:00000001:2.0:1787827022.078819:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.078820:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.078821:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=342 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.078823:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.078824:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9856057e3c00. 02000000:00000001:2.0:1787827022.078825:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.078826:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.078827:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985609939e00. 00000100:00000001:2.0:1787827022.078830:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.078831:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.078847:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff98560993b0c0 x1874669227900928/t219043334171(219043334171) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.078852:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.078855:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff98560993b0c0 x1874669227900928/t219043334171(219043334171) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.078860:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.078861:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.078862:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9856085a2000. 02000000:00000001:2.0:1787827022.078865:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.078867:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.078868:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=341 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.078869:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.078871:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9856057e2800. 02000000:00000001:2.0:1787827022.078872:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.078874:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.078875:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff98560993b0c0. 00000100:00000001:2.0:1787827022.078877:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.078879:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.078883:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff98560993a940 x1874669227901312/t219043334172(219043334172) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.078889:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.078892:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff98560993a940 x1874669227901312/t219043334172(219043334172) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.078898:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.078899:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.078900:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9856085a3c00. 02000000:00000001:2.0:1787827022.078902:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.078903:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.078905:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=340 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.078906:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.078908:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9856057e2a00. 02000000:00000001:2.0:1787827022.078909:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.078910:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.078912:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff98560993a940. 00000100:00000001:2.0:1787827022.078914:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.078915:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.078919:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9856099392c0 x1874669227901696/t219043334173(219043334173) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.078926:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.078929:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9856099392c0 x1874669227901696/t219043334173(219043334173) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.078935:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.078936:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.078937:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9856085a0400. 02000000:00000001:2.0:1787827022.078940:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.078941:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.078941:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=339 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.078943:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.078944:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9856057e3e00. 02000000:00000001:2.0:1787827022.078947:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.078948:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.078949:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff9856099392c0. 00000100:00000001:2.0:1787827022.078951:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.078953:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.078957:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985609938f00 x1874669227902080/t219043334174(219043334174) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.078963:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.078966:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985609938f00 x1874669227902080/t219043334174(219043334174) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.078972:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.078973:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.078975:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9856085a2400. 02000000:00000001:2.0:1787827022.078977:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.078978:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.078979:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=338 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.078981:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.078982:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9856057e3200. 02000000:00000001:2.0:1787827022.078984:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.078985:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.078986:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985609938f00. 00000100:00000001:2.0:1787827022.078988:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.078989:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.078993:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985609938780 x1874669227902464/t219043334175(219043334175) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.079004:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.079007:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985609938780 x1874669227902464/t219043334175(219043334175) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.079013:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.079014:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.079015:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9856085a2800. 02000000:00000001:2.0:1787827022.079016:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.079017:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.079018:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=337 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.079020:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.079021:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9856057e3000. 02000000:00000001:2.0:1787827022.079023:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.079024:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.079025:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985609938780. 00000100:00000001:2.0:1787827022.079027:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.079029:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.079032:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff98560993b480 x1874669227902848/t219043334176(219043334176) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.079038:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.079040:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff98560993b480 x1874669227902848/t219043334176(219043334176) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.079046:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.079048:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.079049:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9856085a1000. 02000000:00000001:2.0:1787827022.079051:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.079053:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.079054:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=336 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.079055:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.079057:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9856057e2400. 02000000:00000001:2.0:1787827022.079059:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.079060:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.079062:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff98560993b480. 00000100:00000001:2.0:1787827022.079064:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.079065:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.079069:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985609939a40 x1874669227903232/t219043334177(219043334177) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.079075:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.079078:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985609939a40 x1874669227903232/t219043334177(219043334177) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.079084:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.079085:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.079087:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9856085a1c00. 02000000:00000001:2.0:1787827022.079089:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.079090:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.079091:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=335 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.079093:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.079094:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985610f2a800. 02000000:00000001:2.0:1787827022.079096:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.079097:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.079098:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985609939a40. 00000100:00000001:2.0:1787827022.079100:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.079101:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.079105:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff98560993ad00 x1874669227903616/t219043334178(219043334178) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.079111:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.079114:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff98560993ad00 x1874669227903616/t219043334178(219043334178) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.079120:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.079121:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.079122:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9856085a1800. 02000000:00000001:2.0:1787827022.079124:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.079125:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.079127:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=334 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.079128:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.079130:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985610e99c00. 02000000:00000001:2.0:1787827022.079132:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.079133:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.079135:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff98560993ad00. 00000100:00000001:2.0:1787827022.079137:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.079138:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.079142:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985609939680 x1874669227904000/t219043334179(219043334179) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.079148:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.079151:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985609939680 x1874669227904000/t219043334179(219043334179) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.079157:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.079158:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.079159:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9856085a2c00. 02000000:00000001:2.0:1787827022.079162:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.079163:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.079164:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=333 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.079166:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.079167:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985610e99e00. 02000000:00000001:2.0:1787827022.079169:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.079170:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.079172:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985609939680. 00000100:00000001:2.0:1787827022.079174:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.079175:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.079179:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985609938000 x1874669227904384/t219043334180(219043334180) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.079185:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.079188:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985609938000 x1874669227904384/t219043334180(219043334180) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.079194:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.079195:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.079197:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9856085a3400. 02000000:00000001:2.0:1787827022.079199:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.079201:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.079202:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=332 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.079203:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.079205:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985610e98000. 02000000:00000001:2.0:1787827022.079207:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.079208:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.079209:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985609938000. 00000100:00000001:2.0:1787827022.079211:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.079212:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.079217:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff98560993bc00 x1874669227904768/t219043334181(219043334181) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.079223:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.079226:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff98560993bc00 x1874669227904768/t219043334181(219043334181) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.079232:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.079233:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.079234:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985610e50c00. 02000000:00000001:2.0:1787827022.079237:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.079238:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.079239:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=331 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.079241:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.079243:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985610e98200. 02000000:00000001:2.0:1787827022.079244:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.079246:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.079247:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff98560993bc00. 00000100:00000001:2.0:1787827022.079249:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.079250:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.079254:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985610e4c3c0 x1874669227905152/t219043334182(219043334182) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.079260:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.079263:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610e4c3c0 x1874669227905152/t219043334182(219043334182) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.079269:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.079271:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.079272:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985610e51c00. 02000000:00000001:2.0:1787827022.079275:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.079276:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.079277:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=330 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.079279:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.079280:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985610e99a00. 02000000:00000001:2.0:1787827022.079282:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.079283:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.079285:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610e4c3c0. 00000100:00000001:2.0:1787827022.079287:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.079288:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.079292:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985610e4da40 x1874669227905536/t219043334183(219043334183) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.079298:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.079302:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610e4da40 x1874669227905536/t219043334183(219043334183) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.079308:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.079309:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.079310:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985610e50800. 02000000:00000001:2.0:1787827022.079312:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.079313:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.079315:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=329 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.079316:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.079318:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985610e98a00. 02000000:00000001:2.0:1787827022.079319:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.079320:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.079322:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610e4da40. 00000100:00000001:2.0:1787827022.079324:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.079325:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.079329:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985610e4d680 x1874669227905920/t219043334184(219043334184) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.079335:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.079337:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610e4d680 x1874669227905920/t219043334184(219043334184) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.079344:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.079345:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.079346:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985610e53800. 02000000:00000001:2.0:1787827022.079349:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.079350:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.079351:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=328 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.079353:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.079354:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985610e99200. 02000000:00000001:2.0:1787827022.079356:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.079357:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.079358:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610e4d680. 00000100:00000001:2.0:1787827022.079361:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.079362:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.079366:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985610e4d2c0 x1874669227906304/t219043334185(219043334185) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.079373:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.079376:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610e4d2c0 x1874669227906304/t219043334185(219043334185) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.079383:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.079385:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.079386:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985610e52000. 02000000:00000001:2.0:1787827022.079389:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.079390:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.079391:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=327 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.079392:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.079394:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985610e99600. 02000000:00000001:2.0:1787827022.079396:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.079397:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.079398:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610e4d2c0. 00000100:00000001:2.0:1787827022.079400:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.079401:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.079405:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985610e4de00 x1874669227906688/t219043334186(219043334186) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.079417:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.079420:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610e4de00 x1874669227906688/t219043334186(219043334186) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.079427:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.079428:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.079430:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985610e52c00. 02000000:00000001:2.0:1787827022.079432:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.079433:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.079435:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=326 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.079437:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.079438:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9856057e2200. 02000000:00000001:2.0:1787827022.079440:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.079441:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.079443:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610e4de00. 00000100:00000001:2.0:1787827022.079446:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.079448:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.079452:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985610e4fc00 x1874669227907072/t219043334187(219043334187) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.079459:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.079462:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610e4fc00 x1874669227907072/t219043334187(219043334187) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.079469:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.079470:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.079472:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985610e50000. 02000000:00000001:2.0:1787827022.079474:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.079475:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.079476:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=325 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.079477:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.079479:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985608540400. 02000000:00000001:2.0:1787827022.079481:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.079482:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.079484:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610e4fc00. 00000100:00000001:2.0:1787827022.079486:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.079488:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.079492:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985610e4e940 x1874669227907456/t219043334188(219043334188) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.079499:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.079502:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610e4e940 x1874669227907456/t219043334188(219043334188) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.079509:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.079510:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.079511:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985610e53400. 02000000:00000001:2.0:1787827022.079513:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.079515:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.079516:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=324 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.079518:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.079519:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985608540000. 02000000:00000001:2.0:1787827022.079521:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.079522:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.079523:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610e4e940. 00000100:00000001:2.0:1787827022.079525:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.079527:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.079531:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985610e4e1c0 x1874669227907840/t219043334189(219043334189) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.079538:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.079541:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610e4e1c0 x1874669227907840/t219043334189(219043334189) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.079547:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.079548:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.079550:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985610e53c00. 02000000:00000001:2.0:1787827022.079552:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.079553:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.079554:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=323 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.079556:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.079558:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985608541200. 02000000:00000001:2.0:1787827022.079560:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.079561:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.079562:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610e4e1c0. 00000100:00000001:2.0:1787827022.079564:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.079565:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.079570:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985610e4c780 x1874669227908224/t219043334190(219043334190) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.079577:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.079581:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610e4c780 x1874669227908224/t219043334190(219043334190) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.079587:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.079589:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.079590:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985610e50400. 02000000:00000001:2.0:1787827022.079592:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.079594:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.079595:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=322 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.079597:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.079599:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985608540c00. 02000000:00000001:2.0:1787827022.079600:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.079602:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.079603:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610e4c780. 00000100:00000001:2.0:1787827022.079606:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.079607:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.079611:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985610e4c000 x1874669227908608/t219043334191(219043334191) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.079619:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.079622:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610e4c000 x1874669227908608/t219043334191(219043334191) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.079629:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.079630:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.079632:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985610e52800. 02000000:00000001:2.0:1787827022.079634:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.079635:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.079637:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=321 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.079639:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.079640:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985608541800. 02000000:00000001:2.0:1787827022.079642:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.079643:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.079644:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610e4c000. 00000100:00000001:2.0:1787827022.079647:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.079648:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.079652:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985610e4cf00 x1874669227908992/t219043334192(219043334192) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.079659:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.079662:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610e4cf00 x1874669227908992/t219043334192(219043334192) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.079668:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.079669:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.079670:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985610e51400. 02000000:00000001:2.0:1787827022.079673:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.079674:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.079676:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=320 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.079678:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.079679:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985608541400. 02000000:00000001:2.0:1787827022.079681:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.079682:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.079684:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610e4cf00. 00000100:00000001:2.0:1787827022.079686:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.079687:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.079713:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985610e4e580 x1874669227909376/t219043334193(219043334193) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.079720:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.079724:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610e4e580 x1874669227909376/t219043334193(219043334193) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.079730:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.079732:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.079733:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985610e51800. 02000000:00000001:2.0:1787827022.079736:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.079737:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.079738:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=319 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.079740:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.079741:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985608541e00. 02000000:00000001:2.0:1787827022.079743:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.079744:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.079746:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610e4e580. 00000100:00000001:2.0:1787827022.079748:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.079749:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.079753:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985610e4f0c0 x1874669227909760/t219043334194(219043334194) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.079760:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.079763:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610e4f0c0 x1874669227909760/t219043334194(219043334194) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.079770:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.079771:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.079772:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985610e52400. 02000000:00000001:2.0:1787827022.079775:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.079776:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.079777:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=318 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.079779:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.079780:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985608541000. 02000000:00000001:2.0:1787827022.079782:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.079783:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.079785:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610e4f0c0. 00000100:00000001:2.0:1787827022.079787:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.079788:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.079792:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff98561408cf00 x1874669227910144/t219043334195(219043334195) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.079799:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.079802:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff98561408cf00 x1874669227910144/t219043334195(219043334195) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.079810:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.079811:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.079813:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985610e53000. 02000000:00000001:2.0:1787827022.079815:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.079816:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.079817:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=317 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.079819:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.079820:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985608540800. 02000000:00000001:2.0:1787827022.079822:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.079823:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.079825:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff98561408cf00. 00000100:00000001:2.0:1787827022.079831:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.079845:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.079850:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff98561408e580 x1874669227910528/t219043334196(219043334196) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.079857:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.079860:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff98561408e580 x1874669227910528/t219043334196(219043334196) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.079867:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.079868:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.079870:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98560854fc00. 02000000:00000001:2.0:1787827022.079872:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.079874:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.079875:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=316 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.079877:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.079879:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985608541c00. 02000000:00000001:2.0:1787827022.079881:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.079882:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.079884:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff98561408e580. 00000100:00000001:2.0:1787827022.079887:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.079889:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.079893:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff98561408da40 x1874669227910912/t219043334197(219043334197) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.079905:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.079908:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff98561408da40 x1874669227910912/t219043334197(219043334197) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.079914:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.079915:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.079917:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98560854e000. 02000000:00000001:2.0:1787827022.079919:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.079920:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.079921:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=315 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.079923:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.079924:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985608540e00. 02000000:00000001:2.0:1787827022.079926:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.079928:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.079929:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff98561408da40. 00000100:00000001:2.0:1787827022.079932:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.079933:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.079938:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff98561408c3c0 x1874669227911296/t219043334198(219043334198) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.079945:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.079948:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff98561408c3c0 x1874669227911296/t219043334198(219043334198) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.079955:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.079956:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.079957:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98560854c400. 02000000:00000001:2.0:1787827022.079960:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.079961:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.079962:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=314 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.079964:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.079966:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985608541600. 02000000:00000001:2.0:1787827022.079968:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.079969:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.079970:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff98561408c3c0. 00000100:00000001:2.0:1787827022.079972:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.079974:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.079977:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff98561408f0c0 x1874669227911680/t219043334199(219043334199) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.079983:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.079986:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff98561408f0c0 x1874669227911680/t219043334199(219043334199) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.079992:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.079994:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.079995:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98560854e400. 02000000:00000001:2.0:1787827022.079997:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.079998:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.079999:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=313 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.080001:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.080003:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985608540600. 02000000:00000001:2.0:1787827022.080004:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.080005:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.080006:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff98561408f0c0. 00000100:00000001:2.0:1787827022.080009:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.080010:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.080015:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff98561408de00 x1874669227912064/t219043334200(219043334200) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.080022:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.080024:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff98561408de00 x1874669227912064/t219043334200(219043334200) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.080030:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.080031:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.080032:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98560854d400. 02000000:00000001:2.0:1787827022.080035:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.080036:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.080038:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=312 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.080040:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.080041:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985608541a00. 02000000:00000001:2.0:1787827022.080043:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.080044:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.080045:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff98561408de00. 00000100:00000001:2.0:1787827022.080048:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.080049:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.080054:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff98561408e1c0 x1874669227912448/t219043334201(219043334201) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.080061:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.080063:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff98561408e1c0 x1874669227912448/t219043334201(219043334201) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.080069:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.080070:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.080072:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98560854f800. 02000000:00000001:2.0:1787827022.080074:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.080075:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.080076:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=311 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.080078:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.080079:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985610e99400. 02000000:00000001:2.0:1787827022.080081:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.080082:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.080083:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff98561408e1c0. 00000100:00000001:2.0:1787827022.080085:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.080086:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.080090:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff98561408f480 x1874669227912832/t219043334202(219043334202) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.080096:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.080099:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff98561408f480 x1874669227912832/t219043334202(219043334202) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.080104:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.080106:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.080107:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98560854c800. 02000000:00000001:2.0:1787827022.080109:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.080110:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.080111:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=310 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.080113:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.080114:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985602b1a200. 02000000:00000001:2.0:1787827022.080116:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.080118:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.080119:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff98561408f480. 00000100:00000001:2.0:1787827022.080120:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.080121:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.080125:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff98561408c000 x1874669227913216/t219043334203(219043334203) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.080131:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.080133:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff98561408c000 x1874669227913216/t219043334203(219043334203) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.080139:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.080140:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.080142:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98560854cc00. 02000000:00000001:2.0:1787827022.080143:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.080144:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.080145:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=309 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.080147:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.080148:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9856204b2200. 02000000:00000001:2.0:1787827022.080150:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.080150:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.080152:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff98561408c000. 00000100:00000001:2.0:1787827022.080154:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.080154:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.080159:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff98561408f840 x1874669227913600/t219043334204(219043334204) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.080164:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.080166:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff98561408f840 x1874669227913600/t219043334204(219043334204) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.080172:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.080173:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.080175:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98560854dc00. 02000000:00000001:2.0:1787827022.080177:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.080178:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.080179:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=308 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.080180:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.080182:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9856098bfa00. 02000000:00000001:2.0:1787827022.080184:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.080185:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.080186:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff98561408f840. 00000100:00000001:2.0:1787827022.080188:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.080189:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.080194:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff98561408c780 x1874669227913984/t219043334205(219043334205) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.080200:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.080203:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff98561408c780 x1874669227913984/t219043334205(219043334205) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.080209:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.080210:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.080211:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98560854ec00. 02000000:00000001:2.0:1787827022.080213:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.080214:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.080215:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=307 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.080217:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.080218:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9856098be400. 02000000:00000001:2.0:1787827022.080220:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.080221:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.080223:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff98561408c780. 00000100:00000001:2.0:1787827022.080225:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.080226:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.080229:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985602ee03c0 x1874669227914368/t219043334206(219043334206) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.080235:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.080238:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985602ee03c0 x1874669227914368/t219043334206(219043334206) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.080244:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.080245:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.080247:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98560854e800. 02000000:00000001:2.0:1787827022.080248:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.080250:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.080251:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=306 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.080252:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.080253:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9856098be800. 02000000:00000001:2.0:1787827022.080255:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.080256:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.080257:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985602ee03c0. 00000100:00000001:2.0:1787827022.080260:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.080261:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.080265:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985602ee1e00 x1874669227914752/t219043334207(219043334207) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.080271:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.080274:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985602ee1e00 x1874669227914752/t219043334207(219043334207) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.080280:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.080281:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.080282:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98560854f000. 02000000:00000001:2.0:1787827022.080284:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.080285:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.080287:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=305 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.080288:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.080289:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985608540200. 02000000:00000001:2.0:1787827022.080291:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.080291:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.080293:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985602ee1e00. 00000100:00000001:2.0:1787827022.080295:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.080296:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.080300:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985602ee3c00 x1874669227915136/t219043334208(219043334208) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.080311:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.080313:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985602ee3c00 x1874669227915136/t219043334208(219043334208) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.080319:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.080320:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.080321:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98560854d800. 02000000:00000001:2.0:1787827022.080323:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.080324:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.080325:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=304 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.080327:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.080329:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff98561185e800. 02000000:00000001:2.0:1787827022.080331:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.080332:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.080334:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985602ee3c00. 00000100:00000001:2.0:1787827022.080336:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.080337:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.080341:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985602ee21c0 x1874669227915520/t219043334209(219043334209) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.080347:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.080350:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985602ee21c0 x1874669227915520/t219043334209(219043334209) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.080356:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.080357:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.080358:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98560854f400. 02000000:00000001:2.0:1787827022.080360:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.080361:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.080362:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=303 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.080364:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.080365:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9856098bea00. 02000000:00000001:2.0:1787827022.080367:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.080368:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.080370:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985602ee21c0. 00000100:00000001:2.0:1787827022.080372:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.080373:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.080376:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985602ee0f00 x1874669227915904/t219043334210(219043334210) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.080382:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.080384:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985602ee0f00 x1874669227915904/t219043334210(219043334210) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.080390:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.080391:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.080392:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98560854d000. 02000000:00000001:2.0:1787827022.080394:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.080395:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.080396:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=302 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.080397:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.080398:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985602f08400. 02000000:00000001:2.0:1787827022.080402:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.080402:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.080403:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985602ee0f00. 00000100:00000001:2.0:1787827022.080405:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.080406:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.080410:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985602ee12c0 x1874669227916288/t219043334211(219043334211) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.080417:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.080419:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985602ee12c0 x1874669227916288/t219043334211(219043334211) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.080426:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.080427:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.080428:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98560854c000. 02000000:00000001:2.0:1787827022.080431:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.080432:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.080433:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=301 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.080435:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.080437:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff98560578e800. 02000000:00000001:2.0:1787827022.080452:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.080454:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.080455:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985602ee12c0. 00000100:00000001:2.0:1787827022.080458:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.080459:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.080465:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985602ee30c0 x1874669227916672/t219043334212(219043334212) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.080471:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.080474:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985602ee30c0 x1874669227916672/t219043334212(219043334212) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.080480:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.080481:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.080483:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98560543e400. 02000000:00000001:2.0:1787827022.080486:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.080487:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.080488:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=300 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.080490:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.080492:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff98560578ea00. 02000000:00000001:2.0:1787827022.080494:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.080494:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.080496:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985602ee30c0. 00000100:00000001:2.0:1787827022.080498:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.080499:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.080503:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985602ee0780 x1874669227917056/t219043334213(219043334213) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.080509:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.080512:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985602ee0780 x1874669227917056/t219043334213(219043334213) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.080517:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.080519:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.080520:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98560543c000. 02000000:00000001:2.0:1787827022.080523:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.080524:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.080525:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=299 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.080528:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.080529:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff98560578e400. 02000000:00000001:2.0:1787827022.080531:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.080532:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.080533:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985602ee0780. 00000100:00000001:2.0:1787827022.080535:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.080536:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.080540:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985602ee1680 x1874669227917440/t219043334214(219043334214) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.080547:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.080550:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985602ee1680 x1874669227917440/t219043334214(219043334214) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.080556:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.080557:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.080558:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98560543cc00. 02000000:00000001:2.0:1787827022.080560:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.080561:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.080562:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=298 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.080564:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.080565:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff98560578e600. 02000000:00000001:2.0:1787827022.080567:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.080568:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.080569:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985602ee1680. 00000100:00000001:2.0:1787827022.080572:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.080573:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.080577:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985602ee1a40 x1874669227917824/t219043334215(219043334215) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.080583:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.080586:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985602ee1a40 x1874669227917824/t219043334215(219043334215) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.080591:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.080593:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.080595:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98560543e800. 02000000:00000001:2.0:1787827022.080597:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.080598:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.080599:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=297 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.080601:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.080602:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff98560578ec00. 02000000:00000001:2.0:1787827022.080604:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.080605:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.080606:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985602ee1a40. 00000100:00000001:2.0:1787827022.080608:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.080609:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.080613:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985602ee2940 x1874669227918208/t219043334216(219043334216) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.080619:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.080622:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985602ee2940 x1874669227918208/t219043334216(219043334216) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.080628:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.080629:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.080631:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98560543ec00. 02000000:00000001:2.0:1787827022.080633:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.080634:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.080635:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=296 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.080637:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.080638:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff98560578e000. 02000000:00000001:2.0:1787827022.080640:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.080641:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.080642:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985602ee2940. 00000100:00000001:2.0:1787827022.080645:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.080646:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.080650:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985602ee0000 x1874669227918592/t219043334217(219043334217) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.080657:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.080660:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985602ee0000 x1874669227918592/t219043334217(219043334217) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.080666:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.080667:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.080668:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98560543fc00. 02000000:00000001:2.0:1787827022.080670:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.080671:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.080672:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=295 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.080674:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.080676:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff98560578f000. 02000000:00000001:2.0:1787827022.080678:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.080680:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.080681:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985602ee0000. 00000100:00000001:2.0:1787827022.080683:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.080684:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.080688:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985602ee2580 x1874669227918976/t219043334218(219043334218) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.080715:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.080718:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985602ee2580 x1874669227918976/t219043334218(219043334218) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.080724:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.080725:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.080727:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98560543d800. 02000000:00000001:2.0:1787827022.080729:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.080730:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.080731:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=294 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.080733:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.080735:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff98561185f000. 02000000:00000001:2.0:1787827022.080737:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.080738:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.080739:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985602ee2580. 00000100:00000001:2.0:1787827022.080741:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.080742:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.080746:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985602ee3840 x1874669227919360/t219043334219(219043334219) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.080757:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.080760:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985602ee3840 x1874669227919360/t219043334219(219043334219) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.080765:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.080766:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.080768:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98560543f400. 02000000:00000001:2.0:1787827022.080770:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.080771:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.080772:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=293 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.080774:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.080775:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607f01400. 02000000:00000001:2.0:1787827022.080777:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.080778:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.080780:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985602ee3840. 00000100:00000001:2.0:1787827022.080782:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.080783:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.080787:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985602ee2d00 x1874669227919744/t219043334220(219043334220) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.080793:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.080796:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985602ee2d00 x1874669227919744/t219043334220(219043334220) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.080802:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.080803:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.080804:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98560543d000. 02000000:00000001:2.0:1787827022.080806:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.080807:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.080808:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=292 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.080810:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.080812:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607f01000. 02000000:00000001:2.0:1787827022.080814:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.080815:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.080816:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985602ee2d00. 00000100:00000001:2.0:1787827022.080818:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.080819:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.080823:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985610e812c0 x1874669227920128/t219043334221(219043334221) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.080829:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.080844:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610e812c0 x1874669227920128/t219043334221(219043334221) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.080851:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.080852:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.080853:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98560543f800. 02000000:00000001:2.0:1787827022.080856:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.080857:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.080858:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=291 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.080860:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.080861:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607f01a00. 02000000:00000001:2.0:1787827022.080863:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.080864:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.080865:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610e812c0. 00000100:00000001:2.0:1787827022.080868:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.080869:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.080873:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985610f830c0 x1874669227920512/t219043334222(219043334222) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.080879:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.080882:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610f830c0 x1874669227920512/t219043334222(219043334222) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.080887:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.080888:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.080890:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985608491000. 02000000:00000001:2.0:1787827022.080892:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.080893:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.080894:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=290 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.080896:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.080898:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607c25000. 02000000:00000001:2.0:1787827022.080900:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.080901:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.080902:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610f830c0. 00000100:00000001:2.0:1787827022.080904:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.080905:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.080909:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985610f83480 x1874669227920896/t219043334223(219043334223) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.080916:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.080919:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610f83480 x1874669227920896/t219043334223(219043334223) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.080925:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.080926:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.080927:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985608492800. 02000000:00000001:2.0:1787827022.080929:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.080930:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.080932:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=289 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.080933:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.080935:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607c25600. 02000000:00000001:2.0:1787827022.080937:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.080938:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.080939:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610f83480. 00000100:00000001:2.0:1787827022.080941:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.080942:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.080946:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985610f81e00 x1874669227921280/t219043334224(219043334224) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.080953:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.080956:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610f81e00 x1874669227921280/t219043334224(219043334224) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.080961:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.080963:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.080964:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985608490400. 02000000:00000001:2.0:1787827022.080967:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.080969:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.080970:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=288 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.080972:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.080973:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607c24600. 02000000:00000001:2.0:1787827022.080975:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.080976:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.080977:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610f81e00. 00000100:00000001:2.0:1787827022.080979:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.080980:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.080983:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985610f82d00 x1874669227921664/t219043334225(219043334225) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.080989:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.080992:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610f82d00 x1874669227921664/t219043334225(219043334225) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.080997:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.080999:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.081000:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985608493c00. 02000000:00000001:2.0:1787827022.081003:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.081004:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.081005:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=287 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.081006:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.081008:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607c25800. 02000000:00000001:2.0:1787827022.081009:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.081010:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.081011:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610f82d00. 00000100:00000001:2.0:1787827022.081014:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.081015:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.081019:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985610f80f00 x1874669227922048/t219043334226(219043334226) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.081024:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.081027:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610f80f00 x1874669227922048/t219043334226(219043334226) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.081032:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.081034:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.081035:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985608490000. 02000000:00000001:2.0:1787827022.081037:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.081038:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.081039:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=286 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.081041:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.081042:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607c24200. 02000000:00000001:2.0:1787827022.081044:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.081045:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.081046:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610f80f00. 00000100:00000001:2.0:1787827022.081049:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.081050:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.081054:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985610f812c0 x1874669227922432/t219043334227(219043334227) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.081060:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.081062:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610f812c0 x1874669227922432/t219043334227(219043334227) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.081069:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.081070:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.081071:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985608490800. 02000000:00000001:2.0:1787827022.081073:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.081074:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.081075:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=285 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.081077:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.081078:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607c25e00. 02000000:00000001:2.0:1787827022.081080:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.081081:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.081083:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610f812c0. 00000100:00000001:2.0:1787827022.081085:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.081086:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.081090:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985610f80b40 x1874669227922816/t219043334228(219043334228) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.081096:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.081098:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610f80b40 x1874669227922816/t219043334228(219043334228) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.081105:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.081106:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.081108:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985608493000. 02000000:00000001:2.0:1787827022.081111:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.081112:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.081113:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=284 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.081115:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.081116:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607c24c00. 02000000:00000001:2.0:1787827022.081118:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.081119:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.081121:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610f80b40. 00000100:00000001:2.0:1787827022.081123:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.081124:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.081128:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985610e83c00 x1874669227923200/t219043334229(219043334229) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.081134:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.081137:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610e83c00 x1874669227923200/t219043334229(219043334229) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.081142:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.081143:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.081145:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98560543f000. 02000000:00000001:2.0:1787827022.081147:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.081147:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.081149:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=283 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.081151:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.081152:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607f00a00. 02000000:00000001:2.0:1787827022.081154:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.081155:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.081156:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610e83c00. 00000100:00000001:2.0:1787827022.081158:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.081159:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.081164:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985610e83840 x1874669227923584/t219043334230(219043334230) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.081174:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.081177:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610e83840 x1874669227923584/t219043334230(219043334230) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.081183:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.081185:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.081186:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98560543d400. 02000000:00000001:2.0:1787827022.081188:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.081189:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.081190:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=282 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.081192:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.081193:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607f00400. 02000000:00000001:2.0:1787827022.081195:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.081196:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.081198:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610e83840. 00000100:00000001:2.0:1787827022.081200:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.081201:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.081205:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985610e81a40 x1874669227923968/t219043334231(219043334231) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.081211:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.081214:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610e81a40 x1874669227923968/t219043334231(219043334231) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.081220:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.081221:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.081222:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98560543e000. 02000000:00000001:2.0:1787827022.081224:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.081225:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.081226:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=281 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.081228:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.081229:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607f00e00. 02000000:00000001:2.0:1787827022.081231:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.081232:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.081234:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610e81a40. 00000100:00000001:2.0:1787827022.081236:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.081237:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.081241:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985610e81680 x1874669227924352/t219043334232(219043334232) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.081247:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.081250:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610e81680 x1874669227924352/t219043334232(219043334232) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.081256:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.081257:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.081259:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985608894800. 02000000:00000001:2.0:1787827022.081261:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.081262:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.081263:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=280 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.081265:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.081267:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607f01e00. 02000000:00000001:2.0:1787827022.081268:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.081269:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.081271:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610e81680. 00000100:00000001:2.0:1787827022.081273:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.081274:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.081278:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985610e82940 x1874669227924736/t219043334233(219043334233) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.081284:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.081287:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610e82940 x1874669227924736/t219043334233(219043334233) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.081293:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.081294:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.081296:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985610ce0000. 02000000:00000001:2.0:1787827022.081298:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.081299:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.081300:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=279 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.081302:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.081303:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607f01800. 02000000:00000001:2.0:1787827022.081305:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.081306:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.081307:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610e82940. 00000100:00000001:2.0:1787827022.081310:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.081311:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.081315:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985610e83480 x1874669227925120/t219043334234(219043334234) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.081322:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.081324:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610e83480 x1874669227925120/t219043334234(219043334234) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.081330:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.081331:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.081333:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985610ce2800. 02000000:00000001:2.0:1787827022.081335:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.081336:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.081337:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=278 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.081338:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.081340:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607f00600. 02000000:00000001:2.0:1787827022.081341:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.081343:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.081344:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610e83480. 00000100:00000001:2.0:1787827022.081346:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.081347:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.081351:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985610f80000 x1874669227925504/t219043334235(219043334235) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.081356:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.081359:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610f80000 x1874669227925504/t219043334235(219043334235) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.081364:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.081365:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.081367:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985608493800. 02000000:00000001:2.0:1787827022.081369:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.081370:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.081372:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=277 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.081373:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.081375:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607c25200. 02000000:00000001:2.0:1787827022.081377:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.081378:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.081379:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610f80000. 00000100:00000001:2.0:1787827022.081381:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.081382:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.081386:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985610f83840 x1874669227925888/t219043334236(219043334236) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.081392:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.081395:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610f83840 x1874669227925888/t219043334236(219043334236) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.081401:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.081402:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.081403:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985608491400. 02000000:00000001:2.0:1787827022.081406:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.081407:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.081408:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=276 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.081410:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.081411:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607c24000. 02000000:00000001:2.0:1787827022.081413:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.081414:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.081416:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610f83840. 00000100:00000001:2.0:1787827022.081418:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.081419:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.081424:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985610e81e00 x1874669227926272/t219043334237(219043334237) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.081430:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.081433:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610e81e00 x1874669227926272/t219043334237(219043334237) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.081439:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.081441:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.081442:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985610ce1800. 02000000:00000001:2.0:1787827022.081444:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.081445:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.081447:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=275 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.081448:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.081450:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607f00c00. 02000000:00000001:2.0:1787827022.081451:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.081452:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.081454:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610e81e00. 00000100:00000001:2.0:1787827022.081456:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.081457:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.081461:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985610e803c0 x1874669227926656/t219043334238(219043334238) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.081468:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.081470:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610e803c0 x1874669227926656/t219043334238(219043334238) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.081475:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.081476:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.081478:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985610ce0800. 02000000:00000001:2.0:1787827022.081480:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.081481:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.081482:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=274 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.081483:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.081484:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607f00000. 02000000:00000001:2.0:1787827022.081486:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.081487:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.081488:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610e803c0. 00000100:00000001:2.0:1787827022.081490:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.081491:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.081495:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985610e82580 x1874669227927040/t219043334239(219043334239) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.081502:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.081505:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610e82580 x1874669227927040/t219043334239(219043334239) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.081511:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.081512:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.081513:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985610ce0c00. 02000000:00000001:2.0:1787827022.081516:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.081517:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.081518:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=273 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.081520:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.081521:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607f01600. 02000000:00000001:2.0:1787827022.081523:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.081525:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.081526:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610e82580. 00000100:00000001:2.0:1787827022.081528:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.081529:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.081533:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985610e80b40 x1874669227927424/t219043334240(219043334240) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.081539:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.081542:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610e80b40 x1874669227927424/t219043334240(219043334240) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.081549:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.081550:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.081552:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985610ce0400. 02000000:00000001:2.0:1787827022.081554:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.081555:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.081556:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=272 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.081558:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.081559:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff98560578fe00. 02000000:00000001:2.0:1787827022.081561:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.081562:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.081563:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610e80b40. 00000100:00000001:2.0:1787827022.081565:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.081566:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.081570:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985610e80f00 x1874669227927808/t219043334241(219043334241) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.081581:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.081584:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610e80f00 x1874669227927808/t219043334241(219043334241) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.081589:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.081590:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.081592:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985610ce2400. 02000000:00000001:2.0:1787827022.081594:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.081595:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.081596:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=271 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.081598:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.081599:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985602e52600. 02000000:00000001:2.0:1787827022.081601:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.081602:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.081603:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610e80f00. 00000100:00000001:2.0:1787827022.081606:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.081607:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.081611:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985610e82d00 x1874669227928192/t219043334242(219043334242) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.081617:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.081620:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610e82d00 x1874669227928192/t219043334242(219043334242) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.081626:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.081627:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.081629:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985610ce2c00. 02000000:00000001:2.0:1787827022.081630:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.081631:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.081632:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=270 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.081634:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.081636:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607f00800. 02000000:00000001:2.0:1787827022.081637:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.081638:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.081640:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610e82d00. 00000100:00000001:2.0:1787827022.081642:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.081643:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.081648:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985610e80780 x1874669227928576/t219043334243(219043334243) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.081654:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.081656:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610e80780 x1874669227928576/t219043334243(219043334243) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.081662:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.081664:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.081665:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985610ce1c00. 02000000:00000001:2.0:1787827022.081667:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.081668:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.081669:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=269 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.081671:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.081673:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985614db4c00. 02000000:00000001:2.0:1787827022.081675:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.081676:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.081677:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610e80780. 00000100:00000001:2.0:1787827022.081680:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.081681:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.081685:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985610f81a40 x1874669227928960/t219043334244(219043334244) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.081711:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.081714:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610f81a40 x1874669227928960/t219043334244(219043334244) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.081720:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.081722:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.081723:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985608491800. 02000000:00000001:2.0:1787827022.081726:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.081727:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.081728:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=268 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.081730:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.081732:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607c25a00. 02000000:00000001:2.0:1787827022.081734:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.081735:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.081737:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610f81a40. 00000100:00000001:2.0:1787827022.081739:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.081741:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.081745:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985610f80780 x1874669227929344/t219043334245(219043334245) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.081751:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.081755:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610f80780 x1874669227929344/t219043334245(219043334245) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.081761:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.081762:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.081764:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9856110f2800. 02000000:00000001:2.0:1787827022.081767:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.081768:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.081769:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=267 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.081771:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.081772:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607c24800. 02000000:00000001:2.0:1787827022.081774:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.081775:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.081776:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610f80780. 00000100:00000001:2.0:1787827022.081778:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.081780:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.081784:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985610e830c0 x1874669227929728/t219043334246(219043334246) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.081790:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.081793:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610e830c0 x1874669227929728/t219043334246(219043334246) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.081799:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.081800:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.081802:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985610ce2000. 02000000:00000001:2.0:1787827022.081804:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.081806:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.081807:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=266 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.081809:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.081810:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985614db4a00. 02000000:00000001:2.0:1787827022.081812:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.081813:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.081814:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610e830c0. 00000100:00000001:2.0:1787827022.081816:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.081817:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.081821:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985610e821c0 x1874669227930112/t219043334247(219043334247) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.081828:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.081831:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610e821c0 x1874669227930112/t219043334247(219043334247) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.081849:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.081850:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.081852:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985610ce3800. 02000000:00000001:2.0:1787827022.081854:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.081855:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.081856:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=265 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.081859:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.081860:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985614db4600. 02000000:00000001:2.0:1787827022.081862:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.081863:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.081864:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610e821c0. 00000100:00000001:2.0:1787827022.081866:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.081867:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.081872:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607e392c0 x1874669227930496/t219043334248(219043334248) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.081878:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.081881:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607e392c0 x1874669227930496/t219043334248(219043334248) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.081888:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.081889:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.081890:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985610ce3c00. 02000000:00000001:2.0:1787827022.081892:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.081894:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.081895:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=264 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.081897:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.081898:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985614db5a00. 02000000:00000001:2.0:1787827022.081900:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.081901:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.081903:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607e392c0. 00000100:00000001:2.0:1787827022.081907:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.081908:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.081912:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607e3ad00 x1874669227930880/t219043334249(219043334249) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.081918:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.081921:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607e3ad00 x1874669227930880/t219043334249(219043334249) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.081927:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.081928:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.081929:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985610ce3400. 02000000:00000001:2.0:1787827022.081931:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.081932:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.081933:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=263 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.081935:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.081936:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985614db5800. 02000000:00000001:2.0:1787827022.081937:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.081939:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.081940:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607e3ad00. 00000100:00000001:2.0:1787827022.081943:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.081944:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.081948:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607e39680 x1874669227931264/t219043334250(219043334250) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.081954:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.081957:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607e39680 x1874669227931264/t219043334250(219043334250) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.081963:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.081964:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.081966:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985610ce1400. 02000000:00000001:2.0:1787827022.081968:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.081969:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.081970:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=262 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.081971:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.081973:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9856012e7e00. 02000000:00000001:2.0:1787827022.081975:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.081976:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.081978:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607e39680. 00000100:00000001:2.0:1787827022.081980:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.081981:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.081985:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607e383c0 x1874669227931648/t219043334251(219043334251) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.081992:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.081994:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607e383c0 x1874669227931648/t219043334251(219043334251) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.082000:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.082001:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.082003:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985604fbfc00. 02000000:00000001:2.0:1787827022.082024:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.082027:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.082028:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=261 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.082030:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.082032:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9856073aba00. 02000000:00000001:2.0:1787827022.082034:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.082036:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.082037:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607e383c0. 00000100:00000001:2.0:1787827022.082039:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.082041:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.082046:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607e3b480 x1874669227932032/t219043334252(219043334252) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.082057:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.082060:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607e3b480 x1874669227932032/t219043334252(219043334252) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.082066:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.082067:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.082068:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985604fbd400. 02000000:00000001:2.0:1787827022.082070:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.082072:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.082073:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=260 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.082074:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.082076:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9856073ab600. 02000000:00000001:2.0:1787827022.082077:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.082078:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.082080:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607e3b480. 00000100:00000001:2.0:1787827022.082082:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.082083:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.082087:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607e39a40 x1874669227932416/t219043334253(219043334253) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.082093:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.082095:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607e39a40 x1874669227932416/t219043334253(219043334253) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.082101:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.082102:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.082103:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985604fbec00. 02000000:00000001:2.0:1787827022.082105:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.082106:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.082107:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=259 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.082109:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.082110:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9856073aa000. 02000000:00000001:2.0:1787827022.082112:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.082113:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.082114:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607e39a40. 00000100:00000001:2.0:1787827022.082117:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.082118:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.082122:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607e3b0c0 x1874669227932800/t219043334254(219043334254) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.082128:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.082130:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607e3b0c0 x1874669227932800/t219043334254(219043334254) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.082136:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.082137:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.082139:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985604fbc000. 02000000:00000001:2.0:1787827022.082141:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.082142:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.082143:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=258 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.082145:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.082146:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9856073aac00. 02000000:00000001:2.0:1787827022.082148:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.082149:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.082150:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607e3b0c0. 00000100:00000001:2.0:1787827022.082153:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.082154:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.082158:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607e38780 x1874669227933184/t219043334255(219043334255) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.082164:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.082167:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607e38780 x1874669227933184/t219043334255(219043334255) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.082173:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.082174:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.082175:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985604fbf400. 02000000:00000001:2.0:1787827022.082177:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.082178:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.082179:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=257 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.082181:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.082183:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985614db4e00. 02000000:00000001:2.0:1787827022.082185:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.082186:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.082187:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607e38780. 00000100:00000001:2.0:1787827022.082189:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.082190:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.082194:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607e38000 x1874669227933568/t219043334256(219043334256) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.082200:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.082203:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607e38000 x1874669227933568/t219043334256(219043334256) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.082209:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.082210:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.082211:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985604fbe000. 02000000:00000001:2.0:1787827022.082213:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.082215:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.082216:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=256 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.082218:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.082219:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9856157f1400. 02000000:00000001:2.0:1787827022.082223:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.082224:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.082225:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607e38000. 00000100:00000001:2.0:1787827022.082227:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.082229:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.082233:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607e3b840 x1874669227933952/t219043334257(219043334257) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.082240:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.082243:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607e3b840 x1874669227933952/t219043334257(219043334257) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.082249:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.082250:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.082252:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985604fbcc00. 02000000:00000001:2.0:1787827022.082254:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.082255:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.082256:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=255 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.082258:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.082260:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff98560120be00. 02000000:00000001:2.0:1787827022.082262:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.082263:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.082264:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607e3b840. 00000100:00000001:2.0:1787827022.082266:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.082268:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.082272:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607e3bc00 x1874669227934336/t219043334258(219043334258) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.082279:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.082282:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607e3bc00 x1874669227934336/t219043334258(219043334258) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.082287:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.082289:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.082290:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985604fbd800. 02000000:00000001:2.0:1787827022.082292:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.082293:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.082295:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=254 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.082297:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.082298:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9856073aa400. 02000000:00000001:2.0:1787827022.082300:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.082301:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.082302:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607e3bc00. 00000100:00000001:2.0:1787827022.082304:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.082305:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.082310:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607e38b40 x1874669227934720/t219043334259(219043334259) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.082316:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.082319:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607e38b40 x1874669227934720/t219043334259(219043334259) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.082325:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.082326:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.082328:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985604fbe400. 02000000:00000001:2.0:1787827022.082330:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.082331:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.082332:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=253 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.082334:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.082336:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9856039eb200. 02000000:00000001:2.0:1787827022.082338:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.082339:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.082340:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607e38b40. 00000100:00000001:2.0:1787827022.082342:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.082343:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.082348:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607e3a940 x1874669227935104/t219043334260(219043334260) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.082355:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.082357:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607e3a940 x1874669227935104/t219043334260(219043334260) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.082364:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.082365:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.082366:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985604fbf000. 02000000:00000001:2.0:1787827022.082368:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.082369:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.082370:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=252 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.082372:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.082374:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985602eeb000. 02000000:00000001:2.0:1787827022.082376:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.082377:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.082378:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607e3a940. 00000100:00000001:2.0:1787827022.082381:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.082382:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.082386:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607e3a1c0 x1874669227935488/t219043334261(219043334261) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.082393:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.082396:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607e3a1c0 x1874669227935488/t219043334261(219043334261) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.082403:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.082404:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.082406:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985604fbdc00. 02000000:00000001:2.0:1787827022.082408:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.082409:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.082410:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=251 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.082412:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.082413:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985602eebc00. 02000000:00000001:2.0:1787827022.082415:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.082416:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.082417:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607e3a1c0. 00000100:00000001:2.0:1787827022.082419:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.082420:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.082424:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607e3a580 x1874669227935872/t219043334262(219043334262) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.082431:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.082433:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607e3a580 x1874669227935872/t219043334262(219043334262) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.082439:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.082440:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.082441:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985604fbd000. 02000000:00000001:2.0:1787827022.082443:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.082444:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.082445:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=250 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.082447:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.082448:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985602eebe00. 02000000:00000001:2.0:1787827022.082450:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.082451:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.082452:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607e3a580. 00000100:00000001:2.0:1787827022.082454:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.082455:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.082459:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607e38f00 x1874669227936256/t219043334263(219043334263) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.082470:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.082473:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607e38f00 x1874669227936256/t219043334263(219043334263) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.082479:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.082480:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.082481:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985604fbc800. 02000000:00000001:2.0:1787827022.082483:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.082484:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.082485:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=249 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.082486:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.082487:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985602eeb800. 02000000:00000001:2.0:1787827022.082489:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.082490:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.082491:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607e38f00. 00000100:00000001:2.0:1787827022.082493:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.082494:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.082498:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607e39e00 x1874669227936640/t219043334264(219043334264) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.082503:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.082506:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607e39e00 x1874669227936640/t219043334264(219043334264) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.082511:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.082512:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.082514:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985604fbf800. 02000000:00000001:2.0:1787827022.082515:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.082516:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.082517:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=248 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.082519:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.082520:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff98560120b000. 02000000:00000001:2.0:1787827022.082522:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.082523:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.082524:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607e39e00. 00000100:00000001:2.0:1787827022.082527:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.082528:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.082532:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985610e33480 x1874669227937024/t219043334265(219043334265) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.082538:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.082541:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610e33480 x1874669227937024/t219043334265(219043334265) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.082547:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.082548:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.082549:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9856088c2800. 02000000:00000001:2.0:1787827022.082552:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.082554:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.082555:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=247 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.082557:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.082558:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985602ffe600. 02000000:00000001:2.0:1787827022.082560:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.082562:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.082563:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610e33480. 00000100:00000001:2.0:1787827022.082565:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.082566:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.082571:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985610e31a40 x1874669227937408/t219043334266(219043334266) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.082577:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.082579:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610e31a40 x1874669227937408/t219043334266(219043334266) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.082585:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.082586:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.082587:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9856088c1000. 02000000:00000001:2.0:1787827022.082589:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.082590:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.082591:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=246 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.082593:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.082595:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985602ffe200. 02000000:00000001:2.0:1787827022.082597:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.082598:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.082599:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610e31a40. 00000100:00000001:2.0:1787827022.082601:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.082602:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.082607:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985610e330c0 x1874669227937792/t219043334267(219043334267) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.082613:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.082616:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610e330c0 x1874669227937792/t219043334267(219043334267) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.082622:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.082623:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.082625:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9856088c1400. 02000000:00000001:2.0:1787827022.082627:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.082628:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.082629:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=245 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.082630:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.082632:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985602fffe00. 02000000:00000001:2.0:1787827022.082633:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.082634:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.082636:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610e330c0. 00000100:00000001:2.0:1787827022.082637:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.082639:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.082642:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985610e30780 x1874669227938176/t219043334268(219043334268) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.082648:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.082651:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610e30780 x1874669227938176/t219043334268(219043334268) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.082656:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.082657:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.082658:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9856088c2000. 02000000:00000001:2.0:1787827022.082661:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.082662:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.082663:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=244 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.082665:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.082666:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985602eeb600. 02000000:00000001:2.0:1787827022.082668:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.082669:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.082671:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610e30780. 00000100:00000001:2.0:1787827022.082673:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.082674:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.082678:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985610e30000 x1874669227938560/t219043334269(219043334269) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.082685:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.082688:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610e30000 x1874669227938560/t219043334269(219043334269) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.082714:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.082716:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.082717:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9856066b3000. 02000000:00000001:2.0:1787827022.082720:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.082721:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.082722:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=243 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.082724:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.082725:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985610f8f200. 02000000:00000001:2.0:1787827022.082728:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.082729:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.082730:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610e30000. 00000100:00000001:2.0:1787827022.082732:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.082733:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.082737:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985610e33840 x1874669227938944/t219043334270(219043334270) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.082743:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.082746:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610e33840 x1874669227938944/t219043334270(219043334270) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.082752:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.082754:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.082755:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9856066b1800. 02000000:00000001:2.0:1787827022.082758:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.082759:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.082760:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=242 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.082762:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.082764:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985610f8f800. 02000000:00000001:2.0:1787827022.082765:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.082766:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.082768:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610e33840. 00000100:00000001:2.0:1787827022.082770:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.082771:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.082776:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985610e33c00 x1874669227939328/t219043334271(219043334271) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.082782:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.082785:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610e33c00 x1874669227939328/t219043334271(219043334271) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.082791:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.082792:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.082793:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9856066b2c00. 02000000:00000001:2.0:1787827022.082796:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.082797:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.082798:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=241 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.082800:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.082801:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985610f8f600. 02000000:00000001:2.0:1787827022.082803:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.082804:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.082806:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610e33c00. 00000100:00000001:2.0:1787827022.082808:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.082809:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.082813:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985610e30b40 x1874669227939712/t219043334272(219043334272) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.082818:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.082821:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610e30b40 x1874669227939712/t219043334272(219043334272) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.082827:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.082828:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.082830:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9856012a8800. 02000000:00000001:2.0:1787827022.082843:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.082845:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.082846:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=240 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.082848:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.082849:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985610f8e000. 02000000:00000001:2.0:1787827022.082852:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.082853:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.082854:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610e30b40. 00000100:00000001:2.0:1787827022.082856:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.082858:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.082862:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985610e32940 x1874669227940096/t219043334273(219043334273) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.082869:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.082871:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610e32940 x1874669227940096/t219043334273(219043334273) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.082877:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.082879:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.082880:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9856012a9c00. 02000000:00000001:2.0:1787827022.082882:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.082883:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.082884:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=239 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.082886:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.082887:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985610f8f000. 02000000:00000001:2.0:1787827022.082889:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.082890:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.082891:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610e32940. 00000100:00000001:2.0:1787827022.082894:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.082895:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.082898:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985610e321c0 x1874669227940480/t219043334274(219043334274) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.082910:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.082913:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610e321c0 x1874669227940480/t219043334274(219043334274) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.082919:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.082921:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.082922:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9856012a9800. 02000000:00000001:2.0:1787827022.082924:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.082925:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.082926:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=238 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.082928:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.082929:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985610f8ec00. 02000000:00000001:2.0:1787827022.082932:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.082933:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.082934:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610e321c0. 00000100:00000001:2.0:1787827022.082936:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.082937:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.082941:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985610e32580 x1874669227940864/t219043334275(219043334275) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.082948:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.082951:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610e32580 x1874669227940864/t219043334275(219043334275) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.082957:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.082958:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.082959:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9856012a8000. 02000000:00000001:2.0:1787827022.082961:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.082963:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.082964:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=237 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.082966:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.082967:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985610f8e600. 02000000:00000001:2.0:1787827022.082969:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.082970:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.082972:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610e32580. 00000100:00000001:2.0:1787827022.082974:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.082975:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.082979:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985610e30f00 x1874669227941248/t219043334276(219043334276) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.082986:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.082988:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610e30f00 x1874669227941248/t219043334276(219043334276) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.082994:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.082996:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.082997:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9856012ab800. 02000000:00000001:2.0:1787827022.082999:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.083001:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.083002:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=236 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.083004:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.083005:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985610f8ea00. 02000000:00000001:2.0:1787827022.083007:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.083008:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.083009:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610e30f00. 00000100:00000001:2.0:1787827022.083011:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.083013:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.083016:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985610e31e00 x1874669227941632/t219043334277(219043334277) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.083023:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.083026:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610e31e00 x1874669227941632/t219043334277(219043334277) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.083032:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.083033:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.083034:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9856012aa000. 02000000:00000001:2.0:1787827022.083036:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.083038:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.083039:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=235 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.083041:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.083042:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985602fff200. 02000000:00000001:2.0:1787827022.083044:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.083045:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.083046:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610e31e00. 00000100:00000001:2.0:1787827022.083048:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.083049:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.083054:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985610e312c0 x1874669227942016/t219043334278(219043334278) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.083060:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.083062:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610e312c0 x1874669227942016/t219043334278(219043334278) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.083068:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.083069:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.083070:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9856012ab000. 02000000:00000001:2.0:1787827022.083072:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.083074:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.083075:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=234 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.083076:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.083078:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985611f2fa00. 02000000:00000001:2.0:1787827022.083081:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.083082:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.083083:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610e312c0. 00000100:00000001:2.0:1787827022.083085:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.083086:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.083090:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985610e32d00 x1874669227942400/t219043334279(219043334279) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.083096:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.083099:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610e32d00 x1874669227942400/t219043334279(219043334279) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.083104:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.083105:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.083106:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9856012a9000. 02000000:00000001:2.0:1787827022.083108:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.083109:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.083110:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=233 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.083112:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.083114:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985610f8e400. 02000000:00000001:2.0:1787827022.083116:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.083117:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.083118:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610e32d00. 00000100:00000001:2.0:1787827022.083120:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.083121:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.083125:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9856086ad2c0 x1874669227942784/t219043334280(219043334280) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.083130:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.083133:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9856086ad2c0 x1874669227942784/t219043334280(219043334280) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.083139:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.083140:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.083141:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985608655400. 02000000:00000001:2.0:1787827022.083144:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.083145:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.083146:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=232 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.083148:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.083149:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985610db9200. 02000000:00000001:2.0:1787827022.083151:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.083152:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.083153:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff9856086ad2c0. 00000100:00000001:2.0:1787827022.083155:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.083156:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.083160:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9856086ade00 x1874669227943168/t219043334281(219043334281) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.083166:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.083168:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9856086ade00 x1874669227943168/t219043334281(219043334281) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.083174:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.083176:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.083177:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985608656000. 02000000:00000001:2.0:1787827022.083179:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.083180:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.083181:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=231 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.083183:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.083184:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985610db9c00. 02000000:00000001:2.0:1787827022.083186:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.083187:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.083188:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff9856086ade00. 00000100:00000001:2.0:1787827022.083191:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.083192:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.083196:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9856086acf00 x1874669227943552/t219043334282(219043334282) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.083202:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.083205:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9856086acf00 x1874669227943552/t219043334282(219043334282) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.083210:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.083211:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.083213:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985608657c00. 02000000:00000001:2.0:1787827022.083215:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.083216:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.083217:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=230 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.083219:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.083220:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985610db9000. 02000000:00000001:2.0:1787827022.083222:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.083223:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.083224:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff9856086acf00. 00000100:00000001:2.0:1787827022.083227:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.083228:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.083231:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9856086ae580 x1874669227943936/t219043334283(219043334283) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.083237:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.083240:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9856086ae580 x1874669227943936/t219043334283(219043334283) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.083246:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.083247:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.083249:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985608654400. 02000000:00000001:2.0:1787827022.083251:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.083252:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.083254:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=229 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.083255:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.083257:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985610db8600. 02000000:00000001:2.0:1787827022.083259:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.083260:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.083261:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff9856086ae580. 00000100:00000001:2.0:1787827022.083263:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.083264:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.083268:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9856086ae1c0 x1874669227944320/t219043334284(219043334284) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.083275:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.083277:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9856086ae1c0 x1874669227944320/t219043334284(219043334284) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.083282:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.083283:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.083285:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985608656400. 02000000:00000001:2.0:1787827022.083297:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.083299:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.083300:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=228 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.083303:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.083304:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985610db9400. 02000000:00000001:2.0:1787827022.083307:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.083308:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.083309:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff9856086ae1c0. 00000100:00000001:2.0:1787827022.083311:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.083313:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.083317:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9856086aed00 x1874669227944704/t219043334285(219043334285) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.083328:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.083331:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9856086aed00 x1874669227944704/t219043334285(219043334285) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.083337:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.083338:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.083339:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985602fd4400. 02000000:00000001:2.0:1787827022.083342:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.083343:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.083344:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=227 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.083346:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.083347:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985610db8c00. 02000000:00000001:2.0:1787827022.083349:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.083350:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.083351:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff9856086aed00. 00000100:00000001:2.0:1787827022.083357:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.083358:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.083363:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985608653c00 x1874669227945088/t219043334286(219043334286) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.083369:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.083371:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985608653c00 x1874669227945088/t219043334286(219043334286) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.083377:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.083378:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.083380:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985602fd6000. 02000000:00000001:2.0:1787827022.083382:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.083383:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.083384:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=226 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.083386:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.083388:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985612073200. 02000000:00000001:2.0:1787827022.083391:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.083392:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.083393:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985608653c00. 00000100:00000001:2.0:1787827022.083396:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.083397:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.083402:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985608650b40 x1874669227945472/t219043334287(219043334287) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.083409:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.083412:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985608650b40 x1874669227945472/t219043334287(219043334287) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.083418:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.083419:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.083421:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9856067c2c00. 02000000:00000001:2.0:1787827022.083424:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.083425:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.083426:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=225 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.083428:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.083429:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985601343a00. 02000000:00000001:2.0:1787827022.083431:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.083432:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.083434:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985608650b40. 00000100:00000001:2.0:1787827022.083436:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.083437:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.083442:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985608652940 x1874669227945856/t219043334288(219043334288) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.083448:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.083451:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985608652940 x1874669227945856/t219043334288(219043334288) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.083457:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.083458:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.083460:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9856067c3000. 02000000:00000001:2.0:1787827022.083463:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.083464:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.083465:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=224 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.083467:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.083469:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985610ded000. 02000000:00000001:2.0:1787827022.083472:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.083473:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.083474:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985608652940. 00000100:00000001:2.0:1787827022.083477:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.083478:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.083483:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9856086521c0 x1874669227946240/t219043334289(219043334289) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.083489:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.083492:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9856086521c0 x1874669227946240/t219043334289(219043334289) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.083499:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.083500:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.083501:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9856067c2400. 02000000:00000001:2.0:1787827022.083503:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.083505:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.083506:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=223 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.083508:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.083509:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985610dec400. 02000000:00000001:2.0:1787827022.083511:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.083512:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.083514:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff9856086521c0. 00000100:00000001:2.0:1787827022.083516:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.083517:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.083521:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985610e31680 x1874669227946624/t219043334290(219043334290) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.083528:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.083531:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610e31680 x1874669227946624/t219043334290(219043334290) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.083537:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.083539:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.083540:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9856012aa400. 02000000:00000001:2.0:1787827022.083542:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.083544:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.083545:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=222 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.083547:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.083548:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9856095b5800. 02000000:00000001:2.0:1787827022.083551:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.083552:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.083553:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610e31680. 00000100:00000001:2.0:1787827022.083555:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.083556:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.083561:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985610e303c0 x1874669227947008/t219043334291(219043334291) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.083567:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.083570:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610e303c0 x1874669227947008/t219043334291(219043334291) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.083576:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.083577:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.083578:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9856012a8c00. 02000000:00000001:2.0:1787827022.083580:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.083582:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.083583:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=221 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.083585:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.083586:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9856095b5c00. 02000000:00000001:2.0:1787827022.083588:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.083589:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.083590:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610e303c0. 00000100:00000001:2.0:1787827022.083592:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.083593:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.083597:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607cd5a40 x1874669227947392/t219043334292(219043334292) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.083603:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.083606:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607cd5a40 x1874669227947392/t219043334292(219043334292) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.083612:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.083614:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.083615:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9856012a8400. 02000000:00000001:2.0:1787827022.083617:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.083618:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.083619:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=220 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.083621:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.083623:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9856095b4400. 02000000:00000001:2.0:1787827022.083624:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.083625:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.083627:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607cd5a40. 00000100:00000001:2.0:1787827022.083629:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.083631:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.083634:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607cd70c0 x1874669227947776/t219043334293(219043334293) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.083640:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.083643:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607cd70c0 x1874669227947776/t219043334293(219043334293) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.083649:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.083650:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.083652:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9856012aac00. 02000000:00000001:2.0:1787827022.083654:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.083655:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.083656:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=219 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.083658:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.083659:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9856095b5000. 02000000:00000001:2.0:1787827022.083661:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.083662:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.083663:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607cd70c0. 00000100:00000001:2.0:1787827022.083665:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.083666:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.083669:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607cd4780 x1874669227948160/t219043334294(219043334294) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.083675:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.083678:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607cd4780 x1874669227948160/t219043334294(219043334294) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.083684:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.083685:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.083687:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9856012aa800. 02000000:00000001:2.0:1787827022.083712:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.083713:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.083715:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=218 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.083717:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.083718:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9856095b4e00. 02000000:00000001:2.0:1787827022.083721:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.083722:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.083723:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607cd4780. 00000100:00000001:2.0:1787827022.083726:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.083727:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.083731:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607cd4000 x1874669227948544/t219043334295(219043334295) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.083738:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.083741:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607cd4000 x1874669227948544/t219043334295(219043334295) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.083747:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.083748:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.083749:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9856012abc00. 02000000:00000001:2.0:1787827022.083752:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.083753:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.083754:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=217 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.083756:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.083757:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9856095b5400. 02000000:00000001:2.0:1787827022.083759:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.083760:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.083762:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607cd4000. 00000100:00000001:2.0:1787827022.083764:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.083765:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.083770:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607cd7840 x1874669227948928/t219043334296(219043334296) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.083781:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.083784:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607cd7840 x1874669227948928/t219043334296(219043334296) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.083789:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.083791:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.083792:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9856012a9400. 02000000:00000001:2.0:1787827022.083794:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.083795:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.083796:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=216 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.083797:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.083798:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985614212000. 02000000:00000001:2.0:1787827022.083803:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.083804:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.083806:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607cd7840. 00000100:00000001:2.0:1787827022.083808:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.083809:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.083814:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607cd7c00 x1874669227949312/t219043334297(219043334297) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.083819:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.083822:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607cd7c00 x1874669227949312/t219043334297(219043334297) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.083828:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.083829:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.083831:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985612889400. 02000000:00000001:2.0:1787827022.083846:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.083847:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.083848:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=215 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.083850:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.083851:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9856095b5600. 02000000:00000001:2.0:1787827022.083853:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.083854:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.083856:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607cd7c00. 00000100:00000001:2.0:1787827022.083859:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.083860:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.083864:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607cd4b40 x1874669227949696/t219043334298(219043334298) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.083871:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.083874:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607cd4b40 x1874669227949696/t219043334298(219043334298) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.083880:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.083881:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.083882:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98561288ac00. 02000000:00000001:2.0:1787827022.083884:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.083885:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.083886:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=214 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.083887:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.083888:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985610d2b800. 02000000:00000001:2.0:1787827022.083891:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.083892:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.083893:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607cd4b40. 00000100:00000001:2.0:1787827022.083895:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.083896:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.083900:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607cd6940 x1874669227950080/t219043334299(219043334299) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.083906:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.083909:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607cd6940 x1874669227950080/t219043334299(219043334299) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.083915:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.083916:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.083917:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98561288b800. 02000000:00000001:2.0:1787827022.083919:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.083921:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.083922:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=213 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.083924:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.083925:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985610d2ae00. 02000000:00000001:2.0:1787827022.083927:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.083928:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.083930:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607cd6940. 00000100:00000001:2.0:1787827022.083932:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.083933:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.083937:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607cd61c0 x1874669227950464/t219043334300(219043334300) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.083943:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.083946:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607cd61c0 x1874669227950464/t219043334300(219043334300) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.083951:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.083952:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.083953:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98561288b000. 02000000:00000001:2.0:1787827022.083955:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.083956:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.083957:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=212 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.083958:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.083959:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985610d2b400. 02000000:00000001:2.0:1787827022.083961:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.083962:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.083963:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607cd61c0. 00000100:00000001:2.0:1787827022.083966:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.083967:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.083970:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607cd6580 x1874669227950848/t219043334301(219043334301) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.083976:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.083979:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607cd6580 x1874669227950848/t219043334301(219043334301) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.083984:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.083985:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.083986:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98561288a400. 02000000:00000001:2.0:1787827022.083988:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.083989:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.083990:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=211 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.083992:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.083994:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985610d2a600. 02000000:00000001:2.0:1787827022.083996:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.083998:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.083999:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607cd6580. 00000100:00000001:2.0:1787827022.084001:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.084003:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.084008:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607cd4f00 x1874669227951232/t219043334302(219043334302) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.084015:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.084018:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607cd4f00 x1874669227951232/t219043334302(219043334302) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.084024:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.084025:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.084027:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985612888000. 02000000:00000001:2.0:1787827022.084030:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.084031:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.084032:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=210 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.084034:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.084035:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985614213400. 02000000:00000001:2.0:1787827022.084038:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.084039:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.084040:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607cd4f00. 00000100:00000001:2.0:1787827022.084043:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.084044:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.084049:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607cd5e00 x1874669227951616/t219043334303(219043334303) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.084055:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.084058:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607cd5e00 x1874669227951616/t219043334303(219043334303) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.084064:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.084065:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.084067:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98561288bc00. 02000000:00000001:2.0:1787827022.084069:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.084070:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.084072:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=209 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.084073:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.084075:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985604fdbe00. 02000000:00000001:2.0:1787827022.084077:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.084078:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.084079:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607cd5e00. 00000100:00000001:2.0:1787827022.084081:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.084082:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.084087:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607cd52c0 x1874669227952000/t219043334304(219043334304) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.084094:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.084097:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607cd52c0 x1874669227952000/t219043334304(219043334304) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.084103:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.084105:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.084106:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98561288b400. 02000000:00000001:2.0:1787827022.084108:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.084109:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.084111:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=208 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.084112:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.084114:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985610d2ac00. 02000000:00000001:2.0:1787827022.084116:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.084117:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.084118:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607cd52c0. 00000100:00000001:2.0:1787827022.084120:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.084121:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.084126:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985608652580 x1874669227952384/t219043334305(219043334305) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.084132:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.084135:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985608652580 x1874669227952384/t219043334305(219043334305) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.084141:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.084142:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.084144:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9856067c1c00. 02000000:00000001:2.0:1787827022.084147:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.084148:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.084149:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=207 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.084151:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.084152:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985610ded400. 02000000:00000001:2.0:1787827022.084154:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.084155:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.084156:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985608652580. 00000100:00000001:2.0:1787827022.084158:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.084160:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.084164:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985608650f00 x1874669227952768/t219043334306(219043334306) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.084170:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.084173:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985608650f00 x1874669227952768/t219043334306(219043334306) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.084179:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.084180:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.084182:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9856067c2800. 02000000:00000001:2.0:1787827022.084184:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.084185:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.084186:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=206 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.084188:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.084190:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985610ded600. 02000000:00000001:2.0:1787827022.084191:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.084193:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.084194:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985608650f00. 00000100:00000001:2.0:1787827022.084196:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.084197:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.084202:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985608651e00 x1874669227953152/t219043334307(219043334307) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.084213:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.084216:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985608651e00 x1874669227953152/t219043334307(219043334307) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.084222:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.084223:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.084224:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9856067c3400. 02000000:00000001:2.0:1787827022.084227:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.084228:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.084229:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=205 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.084231:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.084232:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985610dede00. 02000000:00000001:2.0:1787827022.084234:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.084235:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.084237:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985608651e00. 00000100:00000001:2.0:1787827022.084239:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.084240:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.084244:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607cd5680 x1874669227953536/t219043334308(219043334308) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.084251:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.084253:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607cd5680 x1874669227953536/t219043334308(219043334308) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.084259:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.084260:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.084261:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98561288a000. 02000000:00000001:2.0:1787827022.084263:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.084264:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.084265:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=204 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.084267:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.084268:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985610ced600. 02000000:00000001:2.0:1787827022.084271:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.084272:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.084274:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607cd5680. 00000100:00000001:2.0:1787827022.084276:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.084277:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.084281:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607cd43c0 x1874669227953920/t219043334309(219043334309) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.084287:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.084290:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607cd43c0 x1874669227953920/t219043334309(219043334309) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.084295:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.084296:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.084298:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985612889000. 02000000:00000001:2.0:1787827022.084300:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.084301:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.084302:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=203 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.084304:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.084306:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985610cec800. 02000000:00000001:2.0:1787827022.084308:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.084309:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.084310:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607cd43c0. 00000100:00000001:2.0:1787827022.084312:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.084313:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.084317:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607cd7480 x1874669227954304/t219043334310(219043334310) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.084324:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.084326:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607cd7480 x1874669227954304/t219043334310(219043334310) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.084332:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.084334:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.084335:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985612889800. 02000000:00000001:2.0:1787827022.084337:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.084339:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.084340:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=202 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.084342:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.084343:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985610ced200. 02000000:00000001:2.0:1787827022.084345:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.084346:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.084347:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607cd7480. 00000100:00000001:2.0:1787827022.084350:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.084351:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.084355:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607cd6d00 x1874669227954688/t219043334311(219043334311) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.084361:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.084364:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607cd6d00 x1874669227954688/t219043334311(219043334311) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.084370:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.084372:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.084373:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985612888c00. 02000000:00000001:2.0:1787827022.084376:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.084377:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.084378:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=201 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.084380:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.084381:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985610ceca00. 02000000:00000001:2.0:1787827022.084383:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.084384:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.084385:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607cd6d00. 00000100:00000001:2.0:1787827022.084388:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.084389:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.084393:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff98560855e940 x1874669227955072/t219043334312(219043334312) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.084399:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.084402:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff98560855e940 x1874669227955072/t219043334312(219043334312) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.084408:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.084409:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.084410:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985612888400. 02000000:00000001:2.0:1787827022.084412:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.084413:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.084414:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=200 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.084416:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.084418:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985604fdb400. 02000000:00000001:2.0:1787827022.084419:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.084420:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.084422:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff98560855e940. 00000100:00000001:2.0:1787827022.084424:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.084425:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.084430:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff98560855e1c0 x1874669227955456/t219043334313(219043334313) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.084437:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.084439:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff98560855e1c0 x1874669227955456/t219043334313(219043334313) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.084445:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.084446:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.084448:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985606a71400. 02000000:00000001:2.0:1787827022.084451:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.084453:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.084454:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=199 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.084456:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.084457:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9856143d0400. 02000000:00000001:2.0:1787827022.084461:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.084462:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.084463:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff98560855e1c0. 00000100:00000001:2.0:1787827022.084465:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.084466:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.084470:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff98560855e580 x1874669227955840/t219043334314(219043334314) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.084477:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.084479:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff98560855e580 x1874669227955840/t219043334314(219043334314) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.084485:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.084486:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.084488:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985606a71000. 02000000:00000001:2.0:1787827022.084490:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.084491:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.084492:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=198 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.084494:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.084495:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9856143d1800. 02000000:00000001:2.0:1787827022.084497:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.084498:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.084500:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff98560855e580. 00000100:00000001:2.0:1787827022.084502:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.084503:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.084506:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff98560855cf00 x1874669227956224/t219043334315(219043334315) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.084512:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.084514:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff98560855cf00 x1874669227956224/t219043334315(219043334315) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.084520:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.084521:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.084522:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985606a73400. 02000000:00000001:2.0:1787827022.084525:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.084526:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.084527:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=197 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.084529:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.084530:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985610cede00. 02000000:00000001:2.0:1787827022.084532:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.084533:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.084534:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff98560855cf00. 00000100:00000001:2.0:1787827022.084537:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.084538:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.084542:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff98560855de00 x1874669227956608/t219043334316(219043334316) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.084548:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.084551:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff98560855de00 x1874669227956608/t219043334316(219043334316) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827033 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.084557:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.084558:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.084560:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985606624c00. 02000000:00000001:2.0:1787827022.084563:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.084564:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.084565:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=196 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.084567:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.084568:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985611118e00. 02000000:00000001:2.0:1787827022.084571:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.084572:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.084573:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff98560855de00. 00000100:00000001:2.0:1787827022.084576:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.084577:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.084581:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff98560855d2c0 x1874669227956992/t219043334317(219043334317) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.084587:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.084590:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff98560855d2c0 x1874669227956992/t219043334317(219043334317) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.084596:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.084598:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.084599:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985606624400. 02000000:00000001:2.0:1787827022.084601:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.084602:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.084604:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=195 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.084605:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.084607:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985611118600. 02000000:00000001:2.0:1787827022.084608:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.084610:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.084611:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff98560855d2c0. 00000100:00000001:2.0:1787827022.084613:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.084614:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.084618:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff98560855ed00 x1874669227957376/t219043334318(219043334318) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.084629:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.084632:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff98560855ed00 x1874669227957376/t219043334318(219043334318) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.084638:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.084639:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.084641:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985606624800. 02000000:00000001:2.0:1787827022.084643:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.084644:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.084645:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=194 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.084647:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.084648:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985611118200. 02000000:00000001:2.0:1787827022.084650:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.084651:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.084653:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff98560855ed00. 00000100:00000001:2.0:1787827022.084655:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.084656:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.084661:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff98560855d680 x1874669227957760/t219043334319(219043334319) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.084667:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.084670:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff98560855d680 x1874669227957760/t219043334319(219043334319) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.084675:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.084677:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.084678:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985606625400. 02000000:00000001:2.0:1787827022.084681:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.084682:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.084683:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=193 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.084685:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.084686:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9856143d0c00. 02000000:00000001:2.0:1787827022.084688:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.084712:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.084714:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff98560855d680. 00000100:00000001:2.0:1787827022.084717:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.084718:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.084724:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff98560855c3c0 x1874669227958144/t219043334320(219043334320) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.084730:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.084732:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff98560855c3c0 x1874669227958144/t219043334320(219043334320) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.084738:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.084739:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.084741:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985607c4d000. 02000000:00000001:2.0:1787827022.084744:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.084745:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.084746:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=192 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.084748:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.084749:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985602f03a00. 02000000:00000001:2.0:1787827022.084751:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.084752:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.084753:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff98560855c3c0. 00000100:00000001:2.0:1787827022.084755:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.084756:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.084760:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff98560855f480 x1874669227958528/t219043334321(219043334321) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.084767:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.084770:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff98560855f480 x1874669227958528/t219043334321(219043334321) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.084776:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.084777:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.084778:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985607c4c400. 02000000:00000001:2.0:1787827022.084781:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.084782:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.084783:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=191 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.084785:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.084787:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985602f02400. 02000000:00000001:2.0:1787827022.084789:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.084790:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.084792:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff98560855f480. 00000100:00000001:2.0:1787827022.084794:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.084795:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.084799:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff98560855da40 x1874669227958912/t219043334322(219043334322) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.084805:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.084808:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff98560855da40 x1874669227958912/t219043334322(219043334322) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.084814:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.084815:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.084816:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985607c4d400. 02000000:00000001:2.0:1787827022.084818:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.084819:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.084820:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=190 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.084822:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.084823:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985602f02c00. 02000000:00000001:2.0:1787827022.084825:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.084825:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.084827:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff98560855da40. 00000100:00000001:2.0:1787827022.084828:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.084829:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.084845:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff98560855f0c0 x1874669227959296/t219043334323(219043334323) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.084852:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.084855:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff98560855f0c0 x1874669227959296/t219043334323(219043334323) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.084861:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.084862:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.084864:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985607c4e000. 02000000:00000001:2.0:1787827022.084866:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.084867:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.084869:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=189 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.084871:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.084872:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985602f03200. 02000000:00000001:2.0:1787827022.084874:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.084875:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.084876:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff98560855f0c0. 00000100:00000001:2.0:1787827022.084878:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.084879:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.084883:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff98560855c780 x1874669227959680/t219043334324(219043334324) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.084890:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.084892:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff98560855c780 x1874669227959680/t219043334324(219043334324) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.084898:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.084899:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.084900:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985607c4c000. 02000000:00000001:2.0:1787827022.084902:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.084903:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.084904:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=188 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.084906:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.084908:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985611119a00. 02000000:00000001:2.0:1787827022.084911:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.084912:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.084913:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff98560855c780. 00000100:00000001:2.0:1787827022.084915:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.084916:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.084921:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff98560855c000 x1874669227960064/t219043334325(219043334325) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.084927:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.084930:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff98560855c000 x1874669227960064/t219043334325(219043334325) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.084937:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.084938:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.084939:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985607c4f000. 02000000:00000001:2.0:1787827022.084942:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.084943:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.084944:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=187 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.084946:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.084947:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9856118ac600. 02000000:00000001:2.0:1787827022.084949:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.084950:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.084951:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff98560855c000. 00000100:00000001:2.0:1787827022.084953:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.084954:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.084958:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff98560855f840 x1874669227960448/t219043334326(219043334326) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.084965:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.084967:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff98560855f840 x1874669227960448/t219043334326(219043334326) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.084973:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.084974:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.084976:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985607c4c800. 02000000:00000001:2.0:1787827022.084978:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.084979:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.084980:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=186 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.084982:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.084983:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9856118aca00. 02000000:00000001:2.0:1787827022.084985:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.084986:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.084987:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff98560855f840. 00000100:00000001:2.0:1787827022.084989:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.084990:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.084994:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff98560855fc00 x1874669227960832/t219043334327(219043334327) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.085000:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.085003:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff98560855fc00 x1874669227960832/t219043334327(219043334327) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.085009:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.085010:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.085012:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985607c4e400. 02000000:00000001:2.0:1787827022.085014:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.085014:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.085016:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=185 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.085017:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.085018:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9856118ace00. 02000000:00000001:2.0:1787827022.085020:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.085021:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.085023:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff98560855fc00. 00000100:00000001:2.0:1787827022.085025:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.085026:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.085030:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff98560855cb40 x1874669227961216/t219043334328(219043334328) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.085036:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.085039:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff98560855cb40 x1874669227961216/t219043334328(219043334328) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.085045:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.085046:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.085047:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985607c4f800. 02000000:00000001:2.0:1787827022.085049:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.085051:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.085052:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=184 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.085054:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.085055:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9856118ada00. 02000000:00000001:2.0:1787827022.085057:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.085058:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.085059:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff98560855cb40. 00000100:00000001:2.0:1787827022.085062:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.085063:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.085066:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff98560841a580 x1874669227961600/t219043334329(219043334329) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.085077:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.085080:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff98560841a580 x1874669227961600/t219043334329(219043334329) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.085086:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.085087:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.085088:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985607c4e800. 02000000:00000001:2.0:1787827022.085090:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.085091:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.085093:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=183 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.085094:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.085095:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9856118ac800. 02000000:00000001:2.0:1787827022.085097:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.085098:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.085099:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff98560841a580. 00000100:00000001:2.0:1787827022.085101:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.085102:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.085106:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985608418f00 x1874669227961984/t219043334330(219043334330) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.085112:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.085115:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985608418f00 x1874669227961984/t219043334330(219043334330) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.085121:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.085122:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.085123:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985607c4ec00. 02000000:00000001:2.0:1787827022.085125:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.085126:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.085127:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=182 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.085129:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.085130:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9856118adc00. 02000000:00000001:2.0:1787827022.085132:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.085133:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.085135:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985608418f00. 00000100:00000001:2.0:1787827022.085137:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.085138:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.085142:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985608419e00 x1874669227962368/t219043334331(219043334331) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.085148:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.085151:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985608419e00 x1874669227962368/t219043334331(219043334331) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.085157:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.085159:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.085160:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985607c4fc00. 02000000:00000001:2.0:1787827022.085162:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.085164:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.085165:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=181 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.085167:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.085168:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9856118ade00. 02000000:00000001:2.0:1787827022.085170:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.085171:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.085173:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985608419e00. 00000100:00000001:2.0:1787827022.085175:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.085176:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.085180:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9856084192c0 x1874669227962752/t219043334332(219043334332) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.085186:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.085189:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9856084192c0 x1874669227962752/t219043334332(219043334332) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.085195:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.085197:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.085198:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985607c4dc00. 02000000:00000001:2.0:1787827022.085200:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.085201:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.085202:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=180 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.085204:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.085206:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9856118acc00. 02000000:00000001:2.0:1787827022.085208:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.085209:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.085210:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff9856084192c0. 00000100:00000001:2.0:1787827022.085212:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.085213:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.085218:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff98560841ad00 x1874669227963136/t219043334333(219043334333) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.085224:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.085227:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff98560841ad00 x1874669227963136/t219043334333(219043334333) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.085234:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.085235:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.085237:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985606680000. 02000000:00000001:2.0:1787827022.085240:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.085241:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.085242:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=179 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.085244:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.085245:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9856118ad000. 02000000:00000001:2.0:1787827022.085247:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.085248:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.085250:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff98560841ad00. 00000100:00000001:2.0:1787827022.085252:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.085253:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.085258:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985608419680 x1874669227963520/t219043334334(219043334334) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.085264:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.085267:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985608419680 x1874669227963520/t219043334334(219043334334) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.085273:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.085274:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.085276:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985606682c00. 02000000:00000001:2.0:1787827022.085278:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.085279:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.085280:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=178 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.085282:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.085283:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985602f03c00. 02000000:00000001:2.0:1787827022.085285:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.085286:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.085287:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985608419680. 00000100:00000001:2.0:1787827022.085289:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.085290:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.085295:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9856084183c0 x1874669227963904/t219043334335(219043334335) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.085301:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.085304:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9856084183c0 x1874669227963904/t219043334335(219043334335) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.085310:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.085311:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.085313:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985606681000. 02000000:00000001:2.0:1787827022.085315:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.085316:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.085317:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=177 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.085319:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.085321:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9856084f5800. 02000000:00000001:2.0:1787827022.085323:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.085324:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.085325:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff9856084183c0. 00000100:00000001:2.0:1787827022.085327:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.085328:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.085332:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff98560841b480 x1874669227964288/t219043334336(219043334336) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.085339:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.085341:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff98560841b480 x1874669227964288/t219043334336(219043334336) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.085347:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.085348:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.085349:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985606680800. 02000000:00000001:2.0:1787827022.085351:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.085352:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.085353:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=176 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.085355:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.085357:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9856084f4600. 02000000:00000001:2.0:1787827022.085358:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.085359:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.085361:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff98560841b480. 00000100:00000001:2.0:1787827022.085363:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.085364:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.085367:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985608419a40 x1874669227964672/t219043334337(219043334337) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.085373:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.085375:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985608419a40 x1874669227964672/t219043334337(219043334337) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.085381:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.085382:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.085384:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985606683800. 02000000:00000001:2.0:1787827022.085386:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.085387:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.085388:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=175 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.085390:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.085391:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9856084f5600. 02000000:00000001:2.0:1787827022.085392:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.085393:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.085395:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985608419a40. 00000100:00000001:2.0:1787827022.085396:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.085398:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.085401:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff98560841b0c0 x1874669227965056/t219043334338(219043334338) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.085408:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.085410:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff98560841b0c0 x1874669227965056/t219043334338(219043334338) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.085416:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.085418:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.085419:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985606680400. 02000000:00000001:2.0:1787827022.085421:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.085422:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.085423:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=174 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.085425:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.085426:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9856084f4c00. 02000000:00000001:2.0:1787827022.085427:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.085428:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.085429:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff98560841b0c0. 00000100:00000001:2.0:1787827022.085431:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.085432:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.085436:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985608418780 x1874669227965440/t219043334339(219043334339) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.085443:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.085446:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985608418780 x1874669227965440/t219043334339(219043334339) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.085452:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.085453:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.085454:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985606683400. 02000000:00000001:2.0:1787827022.085456:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.085457:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.085459:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=173 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.085460:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.085462:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9856084f4800. 02000000:00000001:2.0:1787827022.085464:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.085465:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.085466:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985608418780. 00000100:00000001:2.0:1787827022.085468:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.085469:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.085474:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985608418000 x1874669227965824/t219043334340(219043334340) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.085485:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.085487:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985608418000 x1874669227965824/t219043334340(219043334340) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.085493:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.085494:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.085495:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985606682000. 02000000:00000001:2.0:1787827022.085498:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.085499:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.085500:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=172 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.085502:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.085503:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9856084f5000. 02000000:00000001:2.0:1787827022.085505:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.085506:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.085507:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985608418000. 00000100:00000001:2.0:1787827022.085510:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.085510:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.085514:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff98560841b840 x1874669227966208/t219043334341(219043334341) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.085521:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.085524:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff98560841b840 x1874669227966208/t219043334341(219043334341) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.085529:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.085530:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.085532:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985606683c00. 02000000:00000001:2.0:1787827022.085534:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.085535:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.085536:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=171 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.085537:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.085539:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9856084f4000. 02000000:00000001:2.0:1787827022.085541:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.085542:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.085543:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff98560841b840. 00000100:00000001:2.0:1787827022.085545:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.085547:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.085550:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9856086512c0 x1874669227966592/t219043334342(219043334342) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.085557:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.085559:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9856086512c0 x1874669227966592/t219043334342(219043334342) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.085565:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.085566:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.085567:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9856067c0000. 02000000:00000001:2.0:1787827022.085571:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.085572:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.085573:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=170 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.085575:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.085576:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985610dedc00. 02000000:00000001:2.0:1787827022.085578:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.085579:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.085580:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff9856086512c0. 00000100:00000001:2.0:1787827022.085582:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.085583:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.085587:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985608652d00 x1874669227966976/t219043334343(219043334343) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.085594:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.085596:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985608652d00 x1874669227966976/t219043334343(219043334343) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.085602:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.085603:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.085604:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9856067c0800. 02000000:00000001:2.0:1787827022.085607:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.085608:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.085609:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=169 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.085611:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.085612:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985610ded800. 02000000:00000001:2.0:1787827022.085614:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.085615:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.085616:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985608652d00. 00000100:00000001:2.0:1787827022.085618:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.085619:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.085623:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985608651680 x1874669227967360/t219043334344(219043334344) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.085630:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.085632:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985608651680 x1874669227967360/t219043334344(219043334344) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.085637:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.085638:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.085639:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9856054c9c00. 02000000:00000001:2.0:1787827022.085642:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.085643:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.085644:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=168 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.085645:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.085646:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985610dec000. 02000000:00000001:2.0:1787827022.085648:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.085649:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.085651:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985608651680. 00000100:00000001:2.0:1787827022.085653:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.085654:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.085657:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9856086503c0 x1874669227967744/t219043334345(219043334345) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.085663:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.085666:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9856086503c0 x1874669227967744/t219043334345(219043334345) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.085672:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.085674:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.085675:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9856054c9000. 02000000:00000001:2.0:1787827022.085678:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.085679:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.085680:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=167 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.085683:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.085684:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985610db8200. 02000000:00000001:2.0:1787827022.085686:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.085687:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.085688:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff9856086503c0. 00000100:00000001:2.0:1787827022.085710:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.085712:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.085717:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985608653480 x1874669227968128/t219043334346(219043334346) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.085723:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.085726:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985608653480 x1874669227968128/t219043334346(219043334346) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.085732:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.085733:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.085734:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9856054ca400. 02000000:00000001:2.0:1787827022.085737:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.085738:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.085739:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=166 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.085741:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.085743:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985603a95a00. 02000000:00000001:2.0:1787827022.085746:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.085747:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.085749:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985608653480. 00000100:00000001:2.0:1787827022.085751:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.085752:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.085756:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985608651a40 x1874669227968512/t219043334347(219043334347) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.085764:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.085766:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985608651a40 x1874669227968512/t219043334347(219043334347) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.085773:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.085774:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.085775:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9856054cb000. 02000000:00000001:2.0:1787827022.085778:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.085779:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.085780:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=165 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.085782:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.085783:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9856069c6a00. 02000000:00000001:2.0:1787827022.085785:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.085786:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.085787:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985608651a40. 00000100:00000001:2.0:1787827022.085789:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.085791:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.085795:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9856086530c0 x1874669227968896/t219043334348(219043334348) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.085801:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.085804:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9856086530c0 x1874669227968896/t219043334348(219043334348) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.085810:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.085811:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.085813:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9856054cbc00. 02000000:00000001:2.0:1787827022.085815:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.085816:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.085817:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=164 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.085819:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.085820:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607cc9600. 02000000:00000001:2.0:1787827022.085822:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.085824:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.085825:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff9856086530c0. 00000100:00000001:2.0:1787827022.085827:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.085828:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.085844:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985608650780 x1874669227969280/t219043334349(219043334349) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.085851:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.085854:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985608650780 x1874669227969280/t219043334349(219043334349) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.085860:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.085861:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.085863:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9856054ca000. 02000000:00000001:2.0:1787827022.085865:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.085866:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.085868:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=163 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.085869:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.085871:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607cc9e00. 02000000:00000001:2.0:1787827022.085873:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.085874:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.085875:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985608650780. 00000100:00000001:2.0:1787827022.085877:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.085878:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.085882:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985608650000 x1874669227969664/t219043334350(219043334350) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.085888:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.085891:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985608650000 x1874669227969664/t219043334350(219043334350) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.085897:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.085898:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.085899:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9856054cb400. 02000000:00000001:2.0:1787827022.085902:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.085903:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.085904:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=162 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.085905:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.085906:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607cc9800. 02000000:00000001:2.0:1787827022.085908:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.085909:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.085911:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985608650000. 00000100:00000001:2.0:1787827022.085913:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.085914:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.085918:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985608653840 x1874669227970048/t219043334351(219043334351) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.085929:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.085932:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985608653840 x1874669227970048/t219043334351(219043334351) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.085938:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.085939:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.085941:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9856054c8400. 02000000:00000001:2.0:1787827022.085943:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.085944:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.085945:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=161 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.085947:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.085948:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607cc9400. 02000000:00000001:2.0:1787827022.085950:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.085951:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.085952:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985608653840. 00000100:00000001:2.0:1787827022.085954:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.085955:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.085958:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607f37c00 x1874669227970432/t219043334352(219043334352) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.085964:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.085966:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607f37c00 x1874669227970432/t219043334352(219043334352) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.085972:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.085973:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.085974:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9856054ca800. 02000000:00000001:2.0:1787827022.085977:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.085978:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.085979:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=160 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.085980:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.085981:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607cc9000. 02000000:00000001:2.0:1787827022.085983:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.085984:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.085985:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607f37c00. 00000100:00000001:2.0:1787827022.085988:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.085989:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.085993:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607f34b40 x1874669227970816/t219043334353(219043334353) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.086000:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.086002:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607f34b40 x1874669227970816/t219043334353(219043334353) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.086009:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.086010:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.086011:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9856054c8c00. 02000000:00000001:2.0:1787827022.086014:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.086015:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.086016:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=159 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.086018:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.086019:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607cc8200. 02000000:00000001:2.0:1787827022.086022:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.086023:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.086025:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607f34b40. 00000100:00000001:2.0:1787827022.086028:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.086029:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.086033:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607f36940 x1874669227971200/t219043334354(219043334354) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.086039:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.086042:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607f36940 x1874669227971200/t219043334354(219043334354) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.086048:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.086049:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.086051:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9856054c9800. 02000000:00000001:2.0:1787827022.086053:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.086054:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.086055:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=158 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.086057:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.086059:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607cc9c00. 02000000:00000001:2.0:1787827022.086061:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.086062:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.086063:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607f36940. 00000100:00000001:2.0:1787827022.086065:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.086066:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.086070:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607f361c0 x1874669227971584/t219043334355(219043334355) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.086076:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.086079:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607f361c0 x1874669227971584/t219043334355(219043334355) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.086085:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.086087:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.086088:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9856054cb800. 02000000:00000001:2.0:1787827022.086090:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.086091:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.086092:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=157 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.086094:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.086095:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607cc9200. 02000000:00000001:2.0:1787827022.086097:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.086098:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.086099:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607f361c0. 00000100:00000001:2.0:1787827022.086101:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.086102:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.086106:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607f36580 x1874669227971968/t219043334356(219043334356) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.086113:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.086115:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607f36580 x1874669227971968/t219043334356(219043334356) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.086121:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.086122:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.086123:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9856054c9400. 02000000:00000001:2.0:1787827022.086125:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.086127:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.086128:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=156 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.086130:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.086131:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607cc8000. 02000000:00000001:2.0:1787827022.086133:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.086134:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.086135:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607f36580. 00000100:00000001:2.0:1787827022.086137:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.086138:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.086143:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607f34f00 x1874669227972352/t219043334357(219043334357) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.086149:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.086152:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607f34f00 x1874669227972352/t219043334357(219043334357) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.086158:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.086159:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.086160:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9856054cac00. 02000000:00000001:2.0:1787827022.086163:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.086164:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.086165:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=155 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.086167:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.086168:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607cc9a00. 02000000:00000001:2.0:1787827022.086170:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.086171:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.086172:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607f34f00. 00000100:00000001:2.0:1787827022.086175:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.086176:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.086180:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607f35e00 x1874669227972736/t219043334358(219043334358) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.086186:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.086188:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607f35e00 x1874669227972736/t219043334358(219043334358) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.086194:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.086196:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.086197:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985612308c00. 02000000:00000001:2.0:1787827022.086199:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.086200:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.086201:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=154 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.086203:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.086204:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607cc8a00. 02000000:00000001:2.0:1787827022.086206:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.086207:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.086208:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607f35e00. 00000100:00000001:2.0:1787827022.086210:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.086211:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.086215:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607f352c0 x1874669227973120/t219043334359(219043334359) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.086221:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.086223:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607f352c0 x1874669227973120/t219043334359(219043334359) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.086229:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.086230:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.086232:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985612308000. 02000000:00000001:2.0:1787827022.086234:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.086235:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.086236:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=153 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.086238:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.086239:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607cc8e00. 02000000:00000001:2.0:1787827022.086241:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.086242:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.086243:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607f352c0. 00000100:00000001:2.0:1787827022.086245:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.086246:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.086251:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607f36d00 x1874669227973504/t219043334360(219043334360) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.086257:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.086260:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607f36d00 x1874669227973504/t219043334360(219043334360) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.086267:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.086268:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.086269:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985612309000. 02000000:00000001:2.0:1787827022.086272:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.086273:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.086274:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=152 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.086276:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.086277:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607cc8c00. 02000000:00000001:2.0:1787827022.086279:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.086280:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.086281:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607f36d00. 00000100:00000001:2.0:1787827022.086284:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.086285:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.086289:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607f35680 x1874669227973888/t219043334361(219043334361) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.086296:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.086298:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607f35680 x1874669227973888/t219043334361(219043334361) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.086304:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.086306:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.086307:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98561230a400. 02000000:00000001:2.0:1787827022.086309:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.086311:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.086312:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=151 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.086314:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.086315:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607cc8400. 02000000:00000001:2.0:1787827022.086317:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.086318:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.086319:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607f35680. 00000100:00000001:2.0:1787827022.086322:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.086323:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.086327:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607f343c0 x1874669227974272/t219043334362(219043334362) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.086338:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.086341:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607f343c0 x1874669227974272/t219043334362(219043334362) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.086347:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.086348:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.086350:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985608563000. 02000000:00000001:2.0:1787827022.086352:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.086353:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.086354:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=150 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.086356:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.086357:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607cc8600. 02000000:00000001:2.0:1787827022.086359:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.086360:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.086361:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607f343c0. 00000100:00000001:2.0:1787827022.086364:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.086365:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.086369:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607f37480 x1874669227974656/t219043334363(219043334363) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.086375:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.086378:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607f37480 x1874669227974656/t219043334363(219043334363) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.086384:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.086385:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.086387:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985608560800. 02000000:00000001:2.0:1787827022.086389:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.086390:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.086391:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=149 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.086392:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.086394:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985610dece00. 02000000:00000001:2.0:1787827022.086396:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.086397:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.086398:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607f37480. 00000100:00000001:2.0:1787827022.086400:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.086401:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.086406:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607f35a40 x1874669227975040/t219043334364(219043334364) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.086412:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.086415:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607f35a40 x1874669227975040/t219043334364(219043334364) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.086421:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.086422:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.086423:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985608560000. 02000000:00000001:2.0:1787827022.086425:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.086426:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.086427:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=148 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.086430:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.086431:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff98560570ec00. 02000000:00000001:2.0:1787827022.086433:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.086434:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.086436:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607f35a40. 00000100:00000001:2.0:1787827022.086438:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.086439:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.086443:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607f370c0 x1874669227975424/t219043334365(219043334365) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.086449:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.086452:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607f370c0 x1874669227975424/t219043334365(219043334365) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.086458:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.086459:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.086461:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985608563400. 02000000:00000001:2.0:1787827022.086463:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.086464:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.086465:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=147 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.086467:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.086468:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff98560570fa00. 02000000:00000001:2.0:1787827022.086471:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.086472:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.086473:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607f370c0. 00000100:00000001:2.0:1787827022.086475:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.086476:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.086480:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607f34780 x1874669227975808/t219043334366(219043334366) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.086485:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.086488:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607f34780 x1874669227975808/t219043334366(219043334366) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.086494:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.086495:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.086497:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985608562c00. 02000000:00000001:2.0:1787827022.086499:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.086500:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.086501:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=146 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.086502:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.086503:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff98560570f600. 02000000:00000001:2.0:1787827022.086505:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.086506:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.086507:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607f34780. 00000100:00000001:2.0:1787827022.086509:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.086510:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.086513:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607f34000 x1874669227976192/t219043334367(219043334367) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.086520:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.086523:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607f34000 x1874669227976192/t219043334367(219043334367) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.086529:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.086530:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.086531:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985608561800. 02000000:00000001:2.0:1787827022.086533:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.086535:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.086536:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=145 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.086538:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.086539:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff98560570fe00. 02000000:00000001:2.0:1787827022.086541:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.086542:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.086543:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607f34000. 00000100:00000001:2.0:1787827022.086545:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.086546:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.086551:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607f37840 x1874669227976576/t219043334368(219043334368) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.086557:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.086560:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607f37840 x1874669227976576/t219043334368(219043334368) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.086566:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.086567:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.086568:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985608561c00. 02000000:00000001:2.0:1787827022.086570:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.086571:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.086572:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=144 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.086574:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.086575:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff98560570fc00. 02000000:00000001:2.0:1787827022.086577:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.086578:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.086579:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607f37840. 00000100:00000001:2.0:1787827022.086581:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.086582:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.086587:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff98560575f480 x1874669227976960/t219043334369(219043334369) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.086593:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.086596:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff98560575f480 x1874669227976960/t219043334369(219043334369) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.086602:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.086604:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.086605:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985608561000. 02000000:00000001:2.0:1787827022.086608:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.086609:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.086610:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=143 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.086612:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.086613:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff98560570e000. 02000000:00000001:2.0:1787827022.086615:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.086616:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.086617:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff98560575f480. 00000100:00000001:2.0:1787827022.086620:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.086621:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.086625:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff98560575da40 x1874669227977344/t219043334370(219043334370) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.086631:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.086634:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff98560575da40 x1874669227977344/t219043334370(219043334370) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.086640:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.086641:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.086642:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985608562800. 02000000:00000001:2.0:1787827022.086644:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.086645:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.086646:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=142 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.086648:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.086649:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff98560570f400. 02000000:00000001:2.0:1787827022.086650:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.086651:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.086652:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff98560575da40. 00000100:00000001:2.0:1787827022.086654:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.086655:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.086659:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff98560575f0c0 x1874669227977728/t219043334371(219043334371) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.086665:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.086667:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff98560575f0c0 x1874669227977728/t219043334371(219043334371) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.086673:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.086675:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.086676:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985608562400. 02000000:00000001:2.0:1787827022.086678:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.086679:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.086680:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=141 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.086682:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.086683:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff98560570ee00. 02000000:00000001:2.0:1787827022.086685:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.086686:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.086688:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff98560575f0c0. 00000100:00000001:2.0:1787827022.086707:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.086709:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.086714:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff98560575c780 x1874669227978112/t219043334372(219043334372) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.086720:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.086723:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff98560575c780 x1874669227978112/t219043334372(219043334372) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.086728:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.086730:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.086731:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985608560400. 02000000:00000001:2.0:1787827022.086734:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.086735:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.086736:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=140 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.086738:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.086739:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff98560570e600. 02000000:00000001:2.0:1787827022.086741:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.086742:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.086743:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff98560575c780. 00000100:00000001:2.0:1787827022.086748:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.086749:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.086753:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff98560575c000 x1874669227978496/t219043334373(219043334373) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.086764:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.086767:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff98560575c000 x1874669227978496/t219043334373(219043334373) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.086773:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.086774:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.086776:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985608563c00. 02000000:00000001:2.0:1787827022.086778:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.086779:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.086780:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=139 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.086782:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.086783:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff98560570f800. 02000000:00000001:2.0:1787827022.086785:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.086787:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.086788:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff98560575c000. 00000100:00000001:2.0:1787827022.086790:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.086791:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.086795:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff98560575f840 x1874669227978880/t219043334374(219043334374) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.086801:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.086804:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff98560575f840 x1874669227978880/t219043334374(219043334374) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.086810:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.086811:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.086813:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985608562000. 02000000:00000001:2.0:1787827022.086815:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.086816:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.086817:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=138 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.086819:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.086820:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff98560570e200. 02000000:00000001:2.0:1787827022.086822:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.086823:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.086824:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff98560575f840. 00000100:00000001:2.0:1787827022.086826:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.086827:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.086842:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff98560575fc00 x1874669227979264/t219043334375(219043334375) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.086849:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.086852:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff98560575fc00 x1874669227979264/t219043334375(219043334375) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.086858:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.086860:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.086861:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985608561400. 02000000:00000001:2.0:1787827022.086864:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.086865:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.086866:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=137 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.086868:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.086869:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff98560570e800. 02000000:00000001:2.0:1787827022.086871:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.086872:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.086873:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff98560575fc00. 00000100:00000001:2.0:1787827022.086875:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.086876:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.086881:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff98560575cb40 x1874669227979648/t219043334376(219043334376) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.086887:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.086890:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff98560575cb40 x1874669227979648/t219043334376(219043334376) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.086896:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.086897:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.086898:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985608563800. 02000000:00000001:2.0:1787827022.086900:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.086901:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.086902:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=136 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.086904:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.086906:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607cc8800. 02000000:00000001:2.0:1787827022.086908:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.086909:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.086910:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff98560575cb40. 00000100:00000001:2.0:1787827022.086913:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.086914:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.086918:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff98560575e940 x1874669227980032/t219043334377(219043334377) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.086924:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.086927:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff98560575e940 x1874669227980032/t219043334377(219043334377) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.086933:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.086934:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.086935:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985608560c00. 02000000:00000001:2.0:1787827022.086937:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.086939:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.086940:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=135 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.086942:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.086943:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff98560571ec00. 02000000:00000001:2.0:1787827022.086945:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.086946:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.086947:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff98560575e940. 00000100:00000001:2.0:1787827022.086949:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.086951:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.086954:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff98560575e1c0 x1874669227980416/t219043334378(219043334378) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.086961:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.086964:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff98560575e1c0 x1874669227980416/t219043334378(219043334378) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.086971:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.086972:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.086973:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985605662c00. 02000000:00000001:2.0:1787827022.086991:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.086993:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.086994:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=134 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.086997:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.086998:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff98560571f000. 02000000:00000001:2.0:1787827022.087001:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.087002:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.087003:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff98560575e1c0. 00000100:00000001:2.0:1787827022.087005:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.087007:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.087012:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff98560575e580 x1874669227980800/t219043334379(219043334379) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.087018:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.087021:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff98560575e580 x1874669227980800/t219043334379(219043334379) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.087027:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.087028:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.087029:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985605661800. 02000000:00000001:2.0:1787827022.087031:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.087033:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.087034:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=133 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.087036:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.087037:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff98560571fe00. 02000000:00000001:2.0:1787827022.087039:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.087040:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.087041:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff98560575e580. 00000100:00000001:2.0:1787827022.087043:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.087044:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.087048:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff98560575cf00 x1874669227981184/t219043334380(219043334380) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.087053:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.087056:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff98560575cf00 x1874669227981184/t219043334380(219043334380) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.087062:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.087063:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.087065:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985605661c00. 02000000:00000001:2.0:1787827022.087067:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.087068:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.087069:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=132 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.087071:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.087072:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff98560571fa00. 02000000:00000001:2.0:1787827022.087073:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.087074:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.087075:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff98560575cf00. 00000100:00000001:2.0:1787827022.087078:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.087079:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.087083:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff98560575de00 x1874669227981568/t219043334381(219043334381) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.087089:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.087091:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff98560575de00 x1874669227981568/t219043334381(219043334381) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.087097:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.087098:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.087099:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985605661000. 02000000:00000001:2.0:1787827022.087101:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.087103:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.087104:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=131 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.087106:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.087107:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff98560571f400. 02000000:00000001:2.0:1787827022.087109:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.087110:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.087111:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff98560575de00. 00000100:00000001:2.0:1787827022.087114:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.087115:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.087119:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff98560575d2c0 x1874669227981952/t219043334382(219043334382) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.087125:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.087127:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff98560575d2c0 x1874669227981952/t219043334382(219043334382) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.087134:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.087135:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.087136:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985605662800. 02000000:00000001:2.0:1787827022.087138:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.087139:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.087141:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=130 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.087142:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.087143:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff98560571fc00. 02000000:00000001:2.0:1787827022.087145:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.087146:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.087148:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff98560575d2c0. 00000100:00000001:2.0:1787827022.087150:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.087151:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.087155:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff98560575ed00 x1874669227982336/t219043334383(219043334383) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.087162:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.087164:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff98560575ed00 x1874669227982336/t219043334383(219043334383) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.087170:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.087171:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.087172:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985605662400. 02000000:00000001:2.0:1787827022.087174:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.087175:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.087177:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=129 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.087178:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.087179:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff98560571e600. 02000000:00000001:2.0:1787827022.087181:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.087182:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.087183:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff98560575ed00. 00000100:00000001:2.0:1787827022.087185:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.087186:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.087191:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff98560575d680 x1874669227982720/t219043334384(219043334384) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.087202:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.087205:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff98560575d680 x1874669227982720/t219043334384(219043334384) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.087210:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.087212:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.087213:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985605660400. 02000000:00000001:2.0:1787827022.087215:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.087216:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.087217:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=128 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.087219:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.087220:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff98560571ea00. 02000000:00000001:2.0:1787827022.087222:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.087223:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.087224:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff98560575d680. 00000100:00000001:2.0:1787827022.087227:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.087228:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.087232:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff98560575c3c0 x1874669227983104/t219043334385(219043334385) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.087238:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.087241:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff98560575c3c0 x1874669227983104/t219043334385(219043334385) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.087247:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.087248:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.087250:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985605663c00. 02000000:00000001:2.0:1787827022.087252:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.087253:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.087254:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=127 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.087255:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.087257:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff98560571e400. 02000000:00000001:2.0:1787827022.087259:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.087260:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.087261:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff98560575c3c0. 00000100:00000001:2.0:1787827022.087264:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.087265:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.087268:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607c2de00 x1874669227983488/t219043334386(219043334386) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.087274:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.087277:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607c2de00 x1874669227983488/t219043334386(219043334386) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.087282:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.087283:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.087284:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985605662000. 02000000:00000001:2.0:1787827022.087286:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.087287:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.087288:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=126 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.087290:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.087292:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff98560571e800. 02000000:00000001:2.0:1787827022.087293:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.087294:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.087296:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607c2de00. 00000100:00000001:2.0:1787827022.087326:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.087328:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.087333:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607c2d2c0 x1874669227983872/t219043334387(219043334387) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.087340:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.087343:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607c2d2c0 x1874669227983872/t219043334387(219043334387) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.087349:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.087350:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.087352:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985605661400. 02000000:00000001:2.0:1787827022.087354:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.087356:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.087357:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=125 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.087359:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.087360:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff98560571f200. 02000000:00000001:2.0:1787827022.087362:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.087363:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.087365:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607c2d2c0. 00000100:00000001:2.0:1787827022.087367:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.087368:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.087372:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607c2ed00 x1874669227984256/t219043334388(219043334388) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.087379:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.087382:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607c2ed00 x1874669227984256/t219043334388(219043334388) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.087387:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.087389:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.087390:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985605663800. 02000000:00000001:2.0:1787827022.087392:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.087393:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.087394:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=124 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.087396:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.087398:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff98560571e200. 02000000:00000001:2.0:1787827022.087399:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.087400:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.087402:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607c2ed00. 00000100:00000001:2.0:1787827022.087404:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.087405:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.087410:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607c2d680 x1874669227984640/t219043334389(219043334389) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.087416:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.087419:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607c2d680 x1874669227984640/t219043334389(219043334389) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.087424:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.087425:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.087427:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985605660c00. 02000000:00000001:2.0:1787827022.087429:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.087431:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.087432:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=123 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.087434:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.087435:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff98560571f600. 02000000:00000001:2.0:1787827022.087437:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.087438:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.087439:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607c2d680. 00000100:00000001:2.0:1787827022.087441:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.087442:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.087446:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607c2c3c0 x1874669227985024/t219043334390(219043334390) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.087453:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.087455:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607c2c3c0 x1874669227985024/t219043334390(219043334390) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.087461:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.087462:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.087464:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985605663000. 02000000:00000001:2.0:1787827022.087466:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.087467:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.087468:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=122 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.087470:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.087471:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff98560571e000. 02000000:00000001:2.0:1787827022.087473:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.087474:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.087475:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607c2c3c0. 00000100:00000001:2.0:1787827022.087478:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.087479:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.087483:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607c2f480 x1874669227985408/t219043334391(219043334391) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.087489:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.087492:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607c2f480 x1874669227985408/t219043334391(219043334391) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.087498:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.087499:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.087500:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985605660800. 02000000:00000001:2.0:1787827022.087503:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.087504:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.087505:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=121 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.087507:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.087508:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff98560571ee00. 02000000:00000001:2.0:1787827022.087510:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.087511:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.087512:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607c2f480. 00000100:00000001:2.0:1787827022.087514:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.087515:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.087520:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607c2da40 x1874669227985792/t219043334392(219043334392) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.087527:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.087529:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607c2da40 x1874669227985792/t219043334392(219043334392) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.087535:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.087536:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.087537:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985605660000. 02000000:00000001:2.0:1787827022.087539:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.087540:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.087541:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=120 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.087543:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.087544:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff98560570e400. 02000000:00000001:2.0:1787827022.087546:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.087547:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.087548:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607c2da40. 00000100:00000001:2.0:1787827022.087560:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.087562:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.087569:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607c2f0c0 x1874669227986176/t219043334393(219043334393) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.087576:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.087579:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607c2f0c0 x1874669227986176/t219043334393(219043334393) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.087585:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.087586:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.087587:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985605663400. 02000000:00000001:2.0:1787827022.087590:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.087591:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.087592:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=119 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.087594:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.087595:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985608530c00. 02000000:00000001:2.0:1787827022.087605:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.087606:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.087608:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607c2f0c0. 00000100:00000001:2.0:1787827022.087610:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.087612:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.087617:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607c2c780 x1874669227986560/t219043334394(219043334394) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.087623:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.087626:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607c2c780 x1874669227986560/t219043334394(219043334394) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.087632:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.087633:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.087635:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9856085be400. 02000000:00000001:2.0:1787827022.087637:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.087639:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.087640:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=118 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.087642:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.087643:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985608531000. 02000000:00000001:2.0:1787827022.087645:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.087646:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.087647:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607c2c780. 00000100:00000001:2.0:1787827022.087649:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.087650:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.087654:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607c2c000 x1874669227986944/t219043334395(219043334395) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.087665:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.087668:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607c2c000 x1874669227986944/t219043334395(219043334395) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.087674:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.087675:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.087676:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9856085bc400. 02000000:00000001:2.0:1787827022.087679:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.087680:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.087681:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=117 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.087683:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.087684:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985608531e00. 02000000:00000001:2.0:1787827022.087686:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.087687:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.087688:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607c2c000. 00000100:00000001:2.0:1787827022.087709:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.087711:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.087715:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607c2f840 x1874669227987328/t219043334396(219043334396) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.087722:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.087725:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607c2f840 x1874669227987328/t219043334396(219043334396) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.087730:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.087732:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.087733:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9856085bfc00. 02000000:00000001:2.0:1787827022.087735:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.087736:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.087737:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=116 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.087739:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.087740:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985608531a00. 02000000:00000001:2.0:1787827022.087742:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.087743:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.087745:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607c2f840. 00000100:00000001:2.0:1787827022.087747:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.087748:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.087752:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607c2fc00 x1874669227987712/t219043334397(219043334397) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.087758:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.087760:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607c2fc00 x1874669227987712/t219043334397(219043334397) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.087766:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.087768:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.087769:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9856085be000. 02000000:00000001:2.0:1787827022.087771:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.087772:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.087773:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=115 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.087775:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.087777:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985608531400. 02000000:00000001:2.0:1787827022.087778:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.087779:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.087781:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607c2fc00. 00000100:00000001:2.0:1787827022.087782:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.087783:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.087788:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607c2cb40 x1874669227988096/t219043334398(219043334398) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.087794:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.087797:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607c2cb40 x1874669227988096/t219043334398(219043334398) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.087802:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.087804:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.087805:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9856085bd400. 02000000:00000001:2.0:1787827022.087807:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.087808:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.087808:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=114 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.087810:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.087811:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985608531c00. 02000000:00000001:2.0:1787827022.087813:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.087814:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.087815:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607c2cb40. 00000100:00000001:2.0:1787827022.087817:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.087818:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.087822:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607c2e940 x1874669227988480/t219043334399(219043334399) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.087828:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.087842:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607c2e940 x1874669227988480/t219043334399(219043334399) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.087849:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.087851:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.087852:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9856085bf800. 02000000:00000001:2.0:1787827022.087855:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.087856:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.087857:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=113 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.087859:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.087860:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985608530600. 02000000:00000001:2.0:1787827022.087862:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.087863:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.087864:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607c2e940. 00000100:00000001:2.0:1787827022.087866:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.087867:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.087872:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607c2e1c0 x1874669227988864/t219043334400(219043334400) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.087878:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.087881:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607c2e1c0 x1874669227988864/t219043334400(219043334400) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.087886:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.087887:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.087889:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9856085bcc00. 02000000:00000001:2.0:1787827022.087891:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.087892:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.087893:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=112 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.087895:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.087896:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985608530a00. 02000000:00000001:2.0:1787827022.087898:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.087899:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.087900:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607c2e1c0. 00000100:00000001:2.0:1787827022.087902:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.087903:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.087908:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607c2e580 x1874669227989248/t219043334401(219043334401) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.087914:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.087917:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607c2e580 x1874669227989248/t219043334401(219043334401) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.087924:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.087925:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.087927:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9856085bf000. 02000000:00000001:2.0:1787827022.087929:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.087930:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.087931:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=111 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.087933:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.087934:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985608530400. 02000000:00000001:2.0:1787827022.087936:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.087937:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.087938:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607c2e580. 00000100:00000001:2.0:1787827022.087941:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.087942:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.087947:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607c2cf00 x1874669227989632/t219043334402(219043334402) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.087953:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.087956:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607c2cf00 x1874669227989632/t219043334402(219043334402) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.087962:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.087963:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.087965:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9856085bc800. 02000000:00000001:2.0:1787827022.087967:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.087968:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.087969:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=110 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.087971:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.087972:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985608530800. 02000000:00000001:2.0:1787827022.087974:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.087975:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.087976:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607c2cf00. 00000100:00000001:2.0:1787827022.087978:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.087979:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.087984:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9856086aad00 x1874669227990016/t219043334403(219043334403) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.087990:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.087993:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9856086aad00 x1874669227990016/t219043334403(219043334403) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.087999:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.088000:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.088002:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9856085bc000. 02000000:00000001:2.0:1787827022.088004:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.088005:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.088006:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=109 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.088008:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.088009:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985608531200. 02000000:00000001:2.0:1787827022.088011:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.088012:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.088013:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff9856086aad00. 00000100:00000001:2.0:1787827022.088016:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.088018:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.088022:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff98560841bc00 x1874669227990400/t219043334404(219043334404) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.088028:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.088030:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff98560841bc00 x1874669227990400/t219043334404(219043334404) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.088037:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.088038:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.088039:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985606681400. 02000000:00000001:2.0:1787827022.088042:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.088043:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.088044:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=108 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.088046:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.088047:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9856084f4a00. 02000000:00000001:2.0:1787827022.088049:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.088050:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.088052:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff98560841bc00. 00000100:00000001:2.0:1787827022.088054:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.088055:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.088059:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985608418b40 x1874669227990784/t219043334405(219043334405) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.088066:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.088068:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985608418b40 x1874669227990784/t219043334405(219043334405) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.088075:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.088076:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.088077:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985606680c00. 02000000:00000001:2.0:1787827022.088080:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.088081:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.088082:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=107 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.088083:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.088085:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9856084f4400. 02000000:00000001:2.0:1787827022.088086:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.088087:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.088088:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985608418b40. 00000100:00000001:2.0:1787827022.088090:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.088091:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.088096:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff98560841a940 x1874669227991168/t219043334406(219043334406) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.088107:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.088110:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff98560841a940 x1874669227991168/t219043334406(219043334406) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.088116:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.088117:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.088118:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985606682800. 02000000:00000001:2.0:1787827022.088120:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.088121:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.088122:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=106 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.088124:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.088125:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9856084f5400. 02000000:00000001:2.0:1787827022.088127:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.088127:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.088128:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff98560841a940. 00000100:00000001:2.0:1787827022.088131:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.088132:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.088136:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff98560841a1c0 x1874669227991552/t219043334407(219043334407) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.088142:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.088145:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff98560841a1c0 x1874669227991552/t219043334407(219043334407) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.088151:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.088152:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.088153:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985605639c00. 02000000:00000001:2.0:1787827022.088156:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.088157:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.088158:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=105 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.088159:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.088161:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9856084f5c00. 02000000:00000001:2.0:1787827022.088163:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.088164:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.088166:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff98560841a1c0. 00000100:00000001:2.0:1787827022.088176:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.088178:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.088183:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607fd0780 x1874669227991936/t219043334408(219043334408) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.088190:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.088193:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607fd0780 x1874669227991936/t219043334408(219043334408) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.088199:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.088201:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.088202:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985605639000. 02000000:00000001:2.0:1787827022.088204:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.088205:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.088207:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=104 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.088209:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.088210:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9856084f5a00. 02000000:00000001:2.0:1787827022.088212:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.088213:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.088214:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607fd0780. 00000100:00000001:2.0:1787827022.088218:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.088219:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.088223:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607fd0000 x1874669227992320/t219043334409(219043334409) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.088229:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.088232:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607fd0000 x1874669227992320/t219043334409(219043334409) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.088238:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.088239:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.088240:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98560563a800. 02000000:00000001:2.0:1787827022.088243:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.088244:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.088245:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=103 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.088247:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.088249:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9856084f5e00. 02000000:00000001:2.0:1787827022.088250:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.088252:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.088253:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607fd0000. 00000100:00000001:2.0:1787827022.088255:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.088256:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.088260:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607fd3840 x1874669227992704/t219043334410(219043334410) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.088265:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.088268:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607fd3840 x1874669227992704/t219043334410(219043334410) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.088274:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.088275:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.088276:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98560563a400. 02000000:00000001:2.0:1787827022.088278:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.088279:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.088280:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=102 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.088282:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.088283:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9856084f4200. 02000000:00000001:2.0:1787827022.088285:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.088286:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.088287:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607fd3840. 00000100:00000001:2.0:1787827022.088289:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.088290:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.088294:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607fd3c00 x1874669227993088/t219043334411(219043334411) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.088301:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.088303:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607fd3c00 x1874669227993088/t219043334411(219043334411) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.088309:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.088310:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.088312:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985605638400. 02000000:00000001:2.0:1787827022.088314:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.088316:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.088317:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=101 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.088319:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.088320:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9856118ac000. 02000000:00000001:2.0:1787827022.088322:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.088323:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.088324:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607fd3c00. 00000100:00000001:2.0:1787827022.088326:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.088327:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.088332:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607fd0b40 x1874669227993472/t219043334412(219043334412) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.088338:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.088341:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607fd0b40 x1874669227993472/t219043334412(219043334412) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.088346:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.088347:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.088349:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98560563bc00. 02000000:00000001:2.0:1787827022.088351:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.088352:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.088352:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=100 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.088354:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.088355:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607ee1000. 02000000:00000001:2.0:1787827022.088357:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.088358:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.088359:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607fd0b40. 00000100:00000001:2.0:1787827022.088361:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.088363:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.088366:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607fd2940 x1874669227993856/t219043334413(219043334413) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.088372:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.088375:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607fd2940 x1874669227993856/t219043334413(219043334413) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.088381:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.088382:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.088383:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98560563a000. 02000000:00000001:2.0:1787827022.088385:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.088386:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.088387:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=99 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.088389:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.088390:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607ee1a00. 02000000:00000001:2.0:1787827022.088392:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.088393:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.088394:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607fd2940. 00000100:00000001:2.0:1787827022.088397:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.088398:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.088402:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607fd21c0 x1874669227994240/t219043334414(219043334414) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.088409:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.088412:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607fd21c0 x1874669227994240/t219043334414(219043334414) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.088417:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.088419:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.088420:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985605639400. 02000000:00000001:2.0:1787827022.088422:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.088424:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.088425:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=98 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.088426:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.088428:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607ee1400. 02000000:00000001:2.0:1787827022.088430:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.088431:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.088432:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607fd21c0. 00000100:00000001:2.0:1787827022.088434:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.088435:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.088439:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607fd2580 x1874669227994624/t219043334415(219043334415) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.088445:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.088448:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607fd2580 x1874669227994624/t219043334415(219043334415) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.088454:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.088455:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.088457:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98560563b800. 02000000:00000001:2.0:1787827022.088459:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.088460:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.088461:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=97 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.088463:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.088464:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607ee1c00. 02000000:00000001:2.0:1787827022.088466:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.088467:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.088468:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607fd2580. 00000100:00000001:2.0:1787827022.088471:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.088472:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.088477:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607fd0f00 x1874669227995008/t219043334416(219043334416) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.088483:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.088485:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607fd0f00 x1874669227995008/t219043334416(219043334416) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.088491:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.088493:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.088494:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985605638c00. 02000000:00000001:2.0:1787827022.088496:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.088497:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.088498:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=96 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.088500:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.088501:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607ee0600. 02000000:00000001:2.0:1787827022.088503:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.088504:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.088506:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607fd0f00. 00000100:00000001:2.0:1787827022.088510:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.088511:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.088516:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607fd1e00 x1874669227995392/t219043334417(219043334417) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.088526:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.088529:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607fd1e00 x1874669227995392/t219043334417(219043334417) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.088535:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.088537:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.088538:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98560563b000. 02000000:00000001:2.0:1787827022.088540:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.088541:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.088542:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=95 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.088544:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.088545:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607ee0a00. 02000000:00000001:2.0:1787827022.088547:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.088548:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.088550:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607fd1e00. 00000100:00000001:2.0:1787827022.088552:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.088554:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.088558:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607fd12c0 x1874669227995776/t219043334418(219043334418) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.088564:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.088567:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607fd12c0 x1874669227995776/t219043334418(219043334418) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.088573:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.088574:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.088575:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985605638800. 02000000:00000001:2.0:1787827022.088577:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.088578:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.088579:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=94 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.088581:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.088583:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607ee0400. 02000000:00000001:2.0:1787827022.088585:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.088586:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.088587:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607fd12c0. 00000100:00000001:2.0:1787827022.088589:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.088591:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.088595:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607fd2d00 x1874669227996160/t219043334419(219043334419) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.088601:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.088604:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607fd2d00 x1874669227996160/t219043334419(219043334419) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.088610:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.088611:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.088613:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985605638000. 02000000:00000001:2.0:1787827022.088615:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.088616:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.088617:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=93 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.088619:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.088620:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607ee0800. 02000000:00000001:2.0:1787827022.088622:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.088623:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.088624:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607fd2d00. 00000100:00000001:2.0:1787827022.088626:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.088627:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.088631:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607fd1680 x1874669227996544/t219043334420(219043334420) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.088637:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.088640:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607fd1680 x1874669227996544/t219043334420(219043334420) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.088646:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.088647:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.088648:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98560563b400. 02000000:00000001:2.0:1787827022.088650:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.088651:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.088652:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=92 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.088654:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.088655:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607ee1200. 02000000:00000001:2.0:1787827022.088657:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.088658:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.088659:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607fd1680. 00000100:00000001:2.0:1787827022.088662:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.088663:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.088666:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607fd03c0 x1874669227996928/t219043334421(219043334421) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.088673:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.088675:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607fd03c0 x1874669227996928/t219043334421(219043334421) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.088681:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.088683:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.088684:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98560563ac00. 02000000:00000001:2.0:1787827022.088687:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.088688:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.088689:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=91 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.088712:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.088713:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607ee0200. 02000000:00000001:2.0:1787827022.088715:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.088717:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.088718:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607fd03c0. 00000100:00000001:2.0:1787827022.088720:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.088722:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.088726:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607fd3480 x1874669227997312/t219043334422(219043334422) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.088733:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.088736:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607fd3480 x1874669227997312/t219043334422(219043334422) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.088741:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.088742:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.088744:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985605639800. 02000000:00000001:2.0:1787827022.088746:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.088748:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.088749:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=90 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.088751:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.088752:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607ee1600. 02000000:00000001:2.0:1787827022.088754:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.088755:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.088756:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607fd3480. 00000100:00000001:2.0:1787827022.088758:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.088759:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.088763:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607fd1a40 x1874669227997696/t219043334423(219043334423) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.088770:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.088772:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607fd1a40 x1874669227997696/t219043334423(219043334423) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.088778:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.088779:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.088780:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98561041a400. 02000000:00000001:2.0:1787827022.088783:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.088784:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.088785:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=89 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.088787:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.088788:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607ee0000. 02000000:00000001:2.0:1787827022.088789:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.088791:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.088792:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607fd1a40. 00000100:00000001:2.0:1787827022.088794:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.088796:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.088800:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607fd30c0 x1874669227998080/t219043334424(219043334424) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.088807:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.088809:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607fd30c0 x1874669227998080/t219043334424(219043334424) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.088815:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.088817:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.088818:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985610418400. 02000000:00000001:2.0:1787827022.088820:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.088822:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.088823:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=88 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.088825:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.088826:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607ee0e00. 02000000:00000001:2.0:1787827022.088828:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.088829:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.088830:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607fd30c0. 00000100:00000001:2.0:1787827022.088845:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.088846:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.088850:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985608590f00 x1874669227998464/t219043334425(219043334425) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.088857:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.088859:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985608590f00 x1874669227998464/t219043334425(219043334425) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.088865:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.088866:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.088867:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98561041bc00. 02000000:00000001:2.0:1787827022.088869:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.088870:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.088871:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=87 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.088874:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.088875:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607ee0c00. 02000000:00000001:2.0:1787827022.088877:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.088878:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.088879:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985608590f00. 00000100:00000001:2.0:1787827022.088882:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.088883:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.088887:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985608591e00 x1874669227998848/t219043334426(219043334426) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.088893:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.088896:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985608591e00 x1874669227998848/t219043334426(219043334426) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.088902:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.088903:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.088905:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98561041a000. 02000000:00000001:2.0:1787827022.088907:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.088908:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.088909:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=86 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.088911:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.088912:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607ee1800. 02000000:00000001:2.0:1787827022.088914:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.088915:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.088917:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985608591e00. 00000100:00000001:2.0:1787827022.088919:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.088920:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.088924:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9856085912c0 x1874669227999232/t219043334427(219043334427) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.088931:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.088934:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9856085912c0 x1874669227999232/t219043334427(219043334427) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.088939:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.088940:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.088942:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985610419400. 02000000:00000001:2.0:1787827022.088943:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.088944:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.088945:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=85 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.088947:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.088948:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9856084f4e00. 02000000:00000001:2.0:1787827022.088950:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.088951:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.088952:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff9856085912c0. 00000100:00000001:2.0:1787827022.088954:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.088955:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.088960:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985608592d00 x1874669227999616/t219043334428(219043334428) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.088999:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.089003:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985608592d00 x1874669227999616/t219043334428(219043334428) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.089008:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.089009:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.089011:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98561041b800. 02000000:00000001:2.0:1787827022.089014:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.089015:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.089017:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=84 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.089018:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.089020:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607e33800. 02000000:00000001:2.0:1787827022.089023:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.089024:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.089031:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985608592d00. 00000100:00000001:2.0:1787827022.089034:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.089035:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.089040:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985608591680 x1874669228000000/t219043334429(219043334429) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.089047:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.089051:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985608591680 x1874669228000000/t219043334429(219043334429) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.089057:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.089058:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.089059:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985610418c00. 02000000:00000001:2.0:1787827022.089061:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.089062:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.089064:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=83 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.089065:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.089067:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607e33e00. 02000000:00000001:2.0:1787827022.089069:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.089070:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.089072:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985608591680. 00000100:00000001:2.0:1787827022.089074:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.089075:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.089079:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9856085903c0 x1874669228000384/t219043334430(219043334430) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.089085:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.089088:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9856085903c0 x1874669228000384/t219043334430(219043334430) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.089094:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.089095:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.089096:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98561041b000. 02000000:00000001:2.0:1787827022.089098:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.089099:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.089101:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=82 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.089103:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.089104:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607e33a00. 02000000:00000001:2.0:1787827022.089106:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.089107:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.089109:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff9856085903c0. 00000100:00000001:2.0:1787827022.089112:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.089113:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.089118:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985608593480 x1874669228000768/t219043334431(219043334431) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.089124:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.089128:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985608593480 x1874669228000768/t219043334431(219043334431) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.089134:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.089135:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.089137:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985610418800. 02000000:00000001:2.0:1787827022.089139:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.089140:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.089141:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=81 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.089143:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.089145:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607e33400. 02000000:00000001:2.0:1787827022.089146:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.089148:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.089149:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985608593480. 00000100:00000001:2.0:1787827022.089151:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.089152:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.089156:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985608591a40 x1874669228001152/t219043334432(219043334432) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.089163:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.089166:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985608591a40 x1874669228001152/t219043334432(219043334432) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.089172:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.089173:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.089175:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985610418000. 02000000:00000001:2.0:1787827022.089177:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.089178:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.089179:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=80 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.089181:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.089183:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607e33c00. 02000000:00000001:2.0:1787827022.089184:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.089185:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.089186:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985608591a40. 00000100:00000001:2.0:1787827022.089189:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.089190:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.089194:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9856085930c0 x1874669228001536/t219043334433(219043334433) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.089200:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.089203:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9856085930c0 x1874669228001536/t219043334433(219043334433) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.089209:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.089210:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.089211:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98561041b400. 02000000:00000001:2.0:1787827022.089213:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.089215:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.089216:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=79 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.089218:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.089219:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607e32600. 02000000:00000001:2.0:1787827022.089221:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.089222:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.089223:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff9856085930c0. 00000100:00000001:2.0:1787827022.089225:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.089226:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.089229:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985608590780 x1874669228001920/t219043334434(219043334434) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.089236:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.089238:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985608590780 x1874669228001920/t219043334434(219043334434) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.089244:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.089245:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.089246:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98561041ac00. 02000000:00000001:2.0:1787827022.089248:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.089248:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.089250:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=78 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.089251:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.089252:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607e32a00. 02000000:00000001:2.0:1787827022.089254:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.089255:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.089256:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985608590780. 00000100:00000001:2.0:1787827022.089258:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.089259:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.089263:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985608590000 x1874669228002304/t219043334435(219043334435) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.089268:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.089271:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985608590000 x1874669228002304/t219043334435(219043334435) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.089277:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.089278:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.089279:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985610419800. 02000000:00000001:2.0:1787827022.089281:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.089282:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.089284:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=77 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.089285:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.089287:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607e32400. 02000000:00000001:2.0:1787827022.089288:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.089289:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.089290:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985608590000. 00000100:00000001:2.0:1787827022.089293:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.089294:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.089298:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985608593840 x1874669228002688/t219043334436(219043334436) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.089304:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.089307:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985608593840 x1874669228002688/t219043334436(219043334436) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.089312:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.089313:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.089315:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985610419c00. 02000000:00000001:2.0:1787827022.089317:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.089319:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.089320:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=76 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.089322:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.089323:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607e32800. 02000000:00000001:2.0:1787827022.089325:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.089326:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.089328:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985608593840. 00000100:00000001:2.0:1787827022.089330:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.089331:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.089335:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985608593c00 x1874669228003072/t219043334437(219043334437) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.089341:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.089344:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985608593c00 x1874669228003072/t219043334437(219043334437) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.089351:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.089352:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.089354:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985610419000. 02000000:00000001:2.0:1787827022.089356:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.089357:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.089358:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=75 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.089360:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.089361:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607e33200. 02000000:00000001:2.0:1787827022.089363:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.089364:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.089365:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985608593c00. 00000100:00000001:2.0:1787827022.089367:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.089368:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.089372:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985608590b40 x1874669228003456/t219043334438(219043334438) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.089378:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.089381:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985608590b40 x1874669228003456/t219043334438(219043334438) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.089387:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.089388:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.089389:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98561041a800. 02000000:00000001:2.0:1787827022.089391:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.089392:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.089393:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=74 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.089395:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.089396:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607e32200. 02000000:00000001:2.0:1787827022.089398:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.089399:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.089400:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985608590b40. 00000100:00000001:2.0:1787827022.089402:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.089403:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.089407:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985608592940 x1874669228003840/t219043334439(219043334439) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.089418:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.089420:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985608592940 x1874669228003840/t219043334439(219043334439) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.089426:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.089427:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.089429:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985610dfc800. 02000000:00000001:2.0:1787827022.089432:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.089433:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.089434:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=73 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.089436:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.089437:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607e33600. 02000000:00000001:2.0:1787827022.089439:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.089440:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.089441:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985608592940. 00000100:00000001:2.0:1787827022.089443:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.089445:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.089449:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9856085921c0 x1874669228004224/t219043334440(219043334440) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.089456:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.089458:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9856085921c0 x1874669228004224/t219043334440(219043334440) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.089464:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.089465:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.089466:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985610dfc000. 02000000:00000001:2.0:1787827022.089468:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.089469:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.089470:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=72 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.089472:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.089473:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607e32000. 02000000:00000001:2.0:1787827022.089475:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.089475:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.089477:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff9856085921c0. 00000100:00000001:2.0:1787827022.089479:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.089480:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.089484:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985608592580 x1874669228004608/t219043334441(219043334441) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.089490:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.089493:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985608592580 x1874669228004608/t219043334441(219043334441) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.089498:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.089500:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.089501:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985610dff400. 02000000:00000001:2.0:1787827022.089503:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.089504:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.089506:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=71 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.089507:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.089509:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607e32e00. 02000000:00000001:2.0:1787827022.089510:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.089511:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.089512:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985608592580. 00000100:00000001:2.0:1787827022.089515:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.089516:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.089520:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985610fb2d00 x1874669228004992/t219043334442(219043334442) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.089526:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.089528:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610fb2d00 x1874669228004992/t219043334442(219043334442) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.089535:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.089536:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.089537:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985610dfec00. 02000000:00000001:2.0:1787827022.089540:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.089541:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.089542:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=70 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.089544:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.089545:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607e32c00. 02000000:00000001:2.0:1787827022.089547:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.089548:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.089550:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610fb2d00. 00000100:00000001:2.0:1787827022.089552:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.089553:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.089557:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985610fb1680 x1874669228005376/t219043334443(219043334443) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.089564:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.089567:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610fb1680 x1874669228005376/t219043334443(219043334443) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.089573:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.089574:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.089575:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985610dfd800. 02000000:00000001:2.0:1787827022.089578:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.089579:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.089580:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=69 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.089582:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.089583:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607ee1e00. 02000000:00000001:2.0:1787827022.089585:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.089586:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.089587:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610fb1680. 00000100:00000001:2.0:1787827022.089589:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.089590:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.089594:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985610fb03c0 x1874669228005760/t219043334444(219043334444) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.089601:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.089603:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610fb03c0 x1874669228005760/t219043334444(219043334444) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.089609:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.089610:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.089612:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985610dfdc00. 02000000:00000001:2.0:1787827022.089614:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.089615:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.089617:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=68 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.089619:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.089620:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985610c60a00. 02000000:00000001:2.0:1787827022.089622:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.089623:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.089624:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610fb03c0. 00000100:00000001:2.0:1787827022.089626:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.089628:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.089632:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985610fb3480 x1874669228006144/t219043334445(219043334445) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.089638:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.089641:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610fb3480 x1874669228006144/t219043334445(219043334445) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.089647:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.089648:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.089649:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985610dfd000. 02000000:00000001:2.0:1787827022.089651:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.089652:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.089653:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=67 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.089655:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.089657:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985610c60800. 02000000:00000001:2.0:1787827022.089658:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.089659:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.089660:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610fb3480. 00000100:00000001:2.0:1787827022.089662:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.089663:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.089667:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985610fb1a40 x1874669228006528/t219043334446(219043334446) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.089673:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.089675:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610fb1a40 x1874669228006528/t219043334446(219043334446) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.089681:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.089684:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.089685:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985610dfe800. 02000000:00000001:2.0:1787827022.089687:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.089688:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.089708:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=66 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.089711:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.089712:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985610c61200. 02000000:00000001:2.0:1787827022.089715:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.089716:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.089717:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610fb1a40. 00000100:00000001:2.0:1787827022.089719:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.089721:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.089726:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985610fb30c0 x1874669228006912/t219043334447(219043334447) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.089732:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.089735:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610fb30c0 x1874669228006912/t219043334447(219043334447) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.089740:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.089741:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.089743:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985610dfe400. 02000000:00000001:2.0:1787827022.089745:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.089746:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.089747:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=65 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.089749:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.089750:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985610c60200. 02000000:00000001:2.0:1787827022.089752:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.089753:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.089755:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610fb30c0. 00000100:00000001:2.0:1787827022.089756:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.089768:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.089775:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985610fb0780 x1874669228007296/t219043334448(219043334448) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.089782:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.089785:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610fb0780 x1874669228007296/t219043334448(219043334448) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.089791:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.089792:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.089794:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985610dfc400. 02000000:00000001:2.0:1787827022.089796:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.089797:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.089798:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=64 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.089800:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.089801:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985610c61600. 02000000:00000001:2.0:1787827022.089803:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.089804:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.089805:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610fb0780. 00000100:00000001:2.0:1787827022.089807:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.089808:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.089812:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985610fb0000 x1874669228007680/t219043334449(219043334449) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.089818:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.089821:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610fb0000 x1874669228007680/t219043334449(219043334449) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.089827:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.089828:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.089830:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985610dffc00. 02000000:00000001:2.0:1787827022.089843:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.089845:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.089846:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=63 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.089848:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.089849:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985610c60000. 02000000:00000001:2.0:1787827022.089851:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.089852:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.089853:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610fb0000. 00000100:00000001:2.0:1787827022.089855:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.089856:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.089862:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985610fb3840 x1874669228008064/t219043334450(219043334450) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.089873:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.089875:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610fb3840 x1874669228008064/t219043334450(219043334450) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.089881:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.089882:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.089884:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985610dfe000. 02000000:00000001:2.0:1787827022.089886:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.089887:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.089888:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=62 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.089890:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.089891:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985610c60e00. 02000000:00000001:2.0:1787827022.089893:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.089894:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.089895:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610fb3840. 00000100:00000001:2.0:1787827022.089897:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.089899:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.089903:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985610fb3c00 x1874669228008448/t219043334451(219043334451) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.089909:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.089912:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610fb3c00 x1874669228008448/t219043334451(219043334451) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.089917:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.089918:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.089920:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985610dfd400. 02000000:00000001:2.0:1787827022.089922:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.089923:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.089924:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=61 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.089926:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.089927:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985610c60c00. 02000000:00000001:2.0:1787827022.089929:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.089930:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.089932:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610fb3c00. 00000100:00000001:2.0:1787827022.089934:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.089935:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.089938:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985610fb0b40 x1874669228008832/t219043334452(219043334452) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.089944:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.089947:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610fb0b40 x1874669228008832/t219043334452(219043334452) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.089952:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.089953:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.089954:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985610dff800. 02000000:00000001:2.0:1787827022.089956:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.089957:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.089958:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=60 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.089959:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.089961:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985610c61800. 02000000:00000001:2.0:1787827022.089962:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.089963:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.089965:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610fb0b40. 00000100:00000001:2.0:1787827022.089967:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.089968:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.089972:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985610fb2940 x1874669228009216/t219043334453(219043334453) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.089978:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.089981:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610fb2940 x1874669228009216/t219043334453(219043334453) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.089987:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.089988:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.089989:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985610dfcc00. 02000000:00000001:2.0:1787827022.089992:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.089993:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.089994:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=59 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.089996:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.089998:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985610c61000. 02000000:00000001:2.0:1787827022.090000:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.090001:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.090002:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610fb2940. 00000100:00000001:2.0:1787827022.090004:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.090005:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.090010:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985610fb21c0 x1874669228009600/t219043334454(219043334454) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.090016:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.090019:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610fb21c0 x1874669228009600/t219043334454(219043334454) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.090024:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.090026:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.090027:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985610dff000. 02000000:00000001:2.0:1787827022.090029:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.090030:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.090031:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=58 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.090033:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.090035:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985610c61e00. 02000000:00000001:2.0:1787827022.090036:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.090037:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.090038:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610fb21c0. 00000100:00000001:2.0:1787827022.090040:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.090041:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.090045:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985610fb2580 x1874669228009984/t219043334455(219043334455) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.090050:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.090053:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610fb2580 x1874669228009984/t219043334455(219043334455) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.090059:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.090060:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.090061:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985608538400. 02000000:00000001:2.0:1787827022.090064:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.090065:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.090067:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=57 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.090068:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.090070:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985610c61a00. 02000000:00000001:2.0:1787827022.090071:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.090073:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.090074:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610fb2580. 00000100:00000001:2.0:1787827022.090075:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.090077:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.090081:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985610fb0f00 x1874669228010368/t219043334456(219043334456) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.090087:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.090089:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610fb0f00 x1874669228010368/t219043334456(219043334456) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.090095:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.090096:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.090098:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98560853bc00. 02000000:00000001:2.0:1787827022.090100:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.090101:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.090102:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=56 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.090104:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.090105:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985610c61400. 02000000:00000001:2.0:1787827022.090107:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.090108:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.090110:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610fb0f00. 00000100:00000001:2.0:1787827022.090112:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.090113:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.090117:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985610fb1e00 x1874669228010752/t219043334457(219043334457) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.090123:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.090126:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610fb1e00 x1874669228010752/t219043334457(219043334457) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.090132:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.090134:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.090135:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98560853a000. 02000000:00000001:2.0:1787827022.090137:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.090139:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.090140:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=55 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.090141:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.090143:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985610c61c00. 02000000:00000001:2.0:1787827022.090145:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.090146:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.090147:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610fb1e00. 00000100:00000001:2.0:1787827022.090149:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.090150:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.090155:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985610fb12c0 x1874669228011136/t219043334458(219043334458) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.090162:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.090165:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610fb12c0 x1874669228011136/t219043334458(219043334458) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.090171:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.090172:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.090173:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985608539400. 02000000:00000001:2.0:1787827022.090176:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.090177:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.090178:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=54 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.090180:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.090182:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985610c60600. 02000000:00000001:2.0:1787827022.090184:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.090185:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.090186:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610fb12c0. 00000100:00000001:2.0:1787827022.090188:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.090189:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.090193:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607e15e00 x1874669228011520/t219043334459(219043334459) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.090200:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.090203:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607e15e00 x1874669228011520/t219043334459(219043334459) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.090210:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.090211:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.090212:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98560853b800. 02000000:00000001:2.0:1787827022.090214:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.090215:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.090217:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=53 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.090218:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.090219:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607e33000. 02000000:00000001:2.0:1787827022.090222:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.090223:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.090224:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607e15e00. 00000100:00000001:2.0:1787827022.090227:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.090228:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.090232:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607e152c0 x1874669228011904/t219043334460(219043334460) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.090239:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.090241:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607e152c0 x1874669228011904/t219043334460(219043334460) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.090247:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.090248:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.090249:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985608538c00. 02000000:00000001:2.0:1787827022.090251:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.090252:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.090253:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=52 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.090255:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.090256:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff98561288ce00. 02000000:00000001:2.0:1787827022.090258:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.090259:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.090260:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607e152c0. 00000100:00000001:2.0:1787827022.090263:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.090264:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.090268:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607e16d00 x1874669228012288/t219043334461(219043334461) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.090279:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.090282:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607e16d00 x1874669228012288/t219043334461(219043334461) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.090288:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.090289:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.090290:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98560853b000. 02000000:00000001:2.0:1787827022.090292:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.090293:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.090294:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=51 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.090296:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.090297:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff98561288d800. 02000000:00000001:2.0:1787827022.090300:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.090301:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.090302:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607e16d00. 00000100:00000001:2.0:1787827022.090304:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.090305:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.090309:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607e15680 x1874669228012672/t219043334462(219043334462) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.090315:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.090318:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607e15680 x1874669228012672/t219043334462(219043334462) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.090324:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.090325:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.090326:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985608538800. 02000000:00000001:2.0:1787827022.090328:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.090329:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.090330:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=50 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.090332:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.090334:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff98561288d000. 02000000:00000001:2.0:1787827022.090335:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.090336:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.090338:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607e15680. 00000100:00000001:2.0:1787827022.090340:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.090341:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.090345:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607e143c0 x1874669228013056/t219043334463(219043334463) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.090352:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.090355:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607e143c0 x1874669228013056/t219043334463(219043334463) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.090360:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.090361:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.090362:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985608538000. 02000000:00000001:2.0:1787827022.090364:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.090365:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.090366:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=49 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.090368:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.090369:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff98561288de00. 02000000:00000001:2.0:1787827022.090370:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.090371:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.090372:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607e143c0. 00000100:00000001:2.0:1787827022.090374:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.090376:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.090379:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607e17480 x1874669228013440/t219043334464(219043334464) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.090385:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.090388:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607e17480 x1874669228013440/t219043334464(219043334464) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.090394:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.090395:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.090396:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98560853b400. 02000000:00000001:2.0:1787827022.090399:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.090400:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.090401:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=48 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.090403:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.090404:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff98561288da00. 02000000:00000001:2.0:1787827022.090406:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.090407:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.090409:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607e17480. 00000100:00000001:2.0:1787827022.090411:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.090412:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.090416:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607e15a40 x1874669228013824/t219043334465(219043334465) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.090422:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.090425:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607e15a40 x1874669228013824/t219043334465(219043334465) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.090430:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.090432:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.090433:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98560853ac00. 02000000:00000001:2.0:1787827022.090435:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.090436:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.090437:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=47 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.090439:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.090440:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff98561288d400. 02000000:00000001:2.0:1787827022.090442:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.090443:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.090444:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607e15a40. 00000100:00000001:2.0:1787827022.090446:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.090447:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.090451:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607e170c0 x1874669228014208/t219043334466(219043334466) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.090458:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.090460:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607e170c0 x1874669228014208/t219043334466(219043334466) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.090466:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.090467:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.090468:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985608539800. 02000000:00000001:2.0:1787827022.090471:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.090472:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.090473:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=46 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.090475:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.090476:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff98561288dc00. 02000000:00000001:2.0:1787827022.090478:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.090479:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.090480:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607e170c0. 00000100:00000001:2.0:1787827022.090482:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.090483:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.090488:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9856140d9680 x1874669228014592/t219043334467(219043334467) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.090494:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.090497:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9856140d9680 x1874669228014592/t219043334467(219043334467) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.090502:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.090503:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.090504:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9856110f1c00. 02000000:00000001:2.0:1787827022.090507:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.090508:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.090509:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=45 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.090511:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.090512:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff98560540a400. 02000000:00000001:2.0:1787827022.090515:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.090516:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.090517:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff9856140d9680. 00000100:00000001:2.0:1787827022.090520:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.090521:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.090526:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9856086a9680 x1874669228014976/t219043334468(219043334468) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.090533:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.090535:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9856086a9680 x1874669228014976/t219043334468(219043334468) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.090541:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.090543:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.090544:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9856085bf400. 02000000:00000001:2.0:1787827022.090547:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.090548:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.090549:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=44 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.090551:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.090552:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985608531600. 02000000:00000001:2.0:1787827022.090554:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.090555:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.090556:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff9856086a9680. 00000100:00000001:2.0:1787827022.090558:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.090559:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.090563:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9856086a83c0 x1874669228015360/t219043334469(219043334469) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.090570:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.090572:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9856086a83c0 x1874669228015360/t219043334469(219043334469) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.090577:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.090578:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.090579:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9856085bec00. 02000000:00000001:2.0:1787827022.090581:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.090583:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.090584:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=43 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.090585:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.090586:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985608530000. 02000000:00000001:2.0:1787827022.090588:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.090589:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.090590:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff9856086a83c0. 00000100:00000001:2.0:1787827022.090592:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.090593:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.090597:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9856086ab480 x1874669228015744/t219043334470(219043334470) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.090603:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.090605:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9856086ab480 x1874669228015744/t219043334470(219043334470) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.090612:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.090613:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.090614:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9856085bd800. 02000000:00000001:2.0:1787827022.090616:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.090617:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.090618:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=42 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.090620:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.090621:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985608530e00. 02000000:00000001:2.0:1787827022.090623:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.090624:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.090626:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff9856086ab480. 00000100:00000001:2.0:1787827022.090628:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.090629:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.090633:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9856086a9a40 x1874669228016128/t219043334471(219043334471) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.090639:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.090642:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9856086a9a40 x1874669228016128/t219043334471(219043334471) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.090647:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.090648:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.090650:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9856085bdc00. 02000000:00000001:2.0:1787827022.090652:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.090653:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.090654:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=41 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.090655:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.090657:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985608531800. 02000000:00000001:2.0:1787827022.090659:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.090660:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.090661:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff9856086a9a40. 00000100:00000001:2.0:1787827022.090663:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.090664:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.090668:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9856086ab0c0 x1874669228016512/t219043334472(219043334472) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.090678:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.090681:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9856086ab0c0 x1874669228016512/t219043334472(219043334472) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.090687:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.090689:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.090710:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9856085bd000. 02000000:00000001:2.0:1787827022.090713:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.090714:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.090715:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=40 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.090717:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.090719:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff98560571f800. 02000000:00000001:2.0:1787827022.090734:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.090735:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.090737:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff9856086ab0c0. 00000100:00000001:2.0:1787827022.090740:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.090741:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.090746:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9856086a8780 x1874669228016896/t219043334473(219043334473) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.090753:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.090756:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9856086a8780 x1874669228016896/t219043334473(219043334473) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.090763:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.090764:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.090765:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9856085be800. 02000000:00000001:2.0:1787827022.090767:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.090769:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.090770:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=39 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.090771:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.090773:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607c39400. 02000000:00000001:2.0:1787827022.090775:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.090777:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.090778:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff9856086a8780. 00000100:00000001:2.0:1787827022.090780:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.090781:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.090785:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9856086a8000 x1874669228017280/t219043334474(219043334474) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.090792:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.090794:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9856086a8000 x1874669228017280/t219043334474(219043334474) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.090800:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.090801:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.090803:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98560860d000. 02000000:00000001:2.0:1787827022.090805:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.090806:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.090807:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=38 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.090809:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.090810:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607c38600. 02000000:00000001:2.0:1787827022.090812:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.090813:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.090814:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff9856086a8000. 00000100:00000001:2.0:1787827022.090816:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.090817:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.090821:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9856086ab840 x1874669228017664/t219043334475(219043334475) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.090827:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.090829:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9856086ab840 x1874669228017664/t219043334475(219043334475) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.090848:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.090849:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.090851:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98560860e800. 02000000:00000001:2.0:1787827022.090853:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.090854:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.090855:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=37 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.090858:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.090859:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607c38a00. 02000000:00000001:2.0:1787827022.090861:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.090862:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.090863:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff9856086ab840. 00000100:00000001:2.0:1787827022.090866:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.090867:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.090872:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9856086abc00 x1874669228018048/t219043334476(219043334476) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.090879:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.090881:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9856086abc00 x1874669228018048/t219043334476(219043334476) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.090886:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.090887:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.090889:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98560860e400. 02000000:00000001:2.0:1787827022.090891:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.090892:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.090893:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=36 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.090895:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.090897:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607c38400. 02000000:00000001:2.0:1787827022.090898:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.090899:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.090900:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff9856086abc00. 00000100:00000001:2.0:1787827022.090902:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.090903:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.090907:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9856086a8b40 x1874669228018432/t219043334477(219043334477) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.090913:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.090915:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9856086a8b40 x1874669228018432/t219043334477(219043334477) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.090921:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.090922:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.090924:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98560860c400. 02000000:00000001:2.0:1787827022.090926:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.090927:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.090928:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=35 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.090930:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.090932:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607c38800. 02000000:00000001:2.0:1787827022.090934:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.090935:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.090936:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff9856086a8b40. 00000100:00000001:2.0:1787827022.090938:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.090939:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.090942:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9856086aa940 x1874669228018816/t219043334478(219043334478) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.090948:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.090951:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9856086aa940 x1874669228018816/t219043334478(219043334478) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.090957:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.090958:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.090959:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98560860fc00. 02000000:00000001:2.0:1787827022.090962:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.090963:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.090964:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=34 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.090966:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.090967:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607c39200. 02000000:00000001:2.0:1787827022.090969:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.090971:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.090972:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff9856086aa940. 00000100:00000001:2.0:1787827022.090974:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.090975:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.090979:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9856086aa1c0 x1874669228019200/t219043334479(219043334479) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.090985:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.090988:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9856086aa1c0 x1874669228019200/t219043334479(219043334479) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.090994:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.090996:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.090998:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98560860e000. 02000000:00000001:2.0:1787827022.091000:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.091001:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.091002:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=33 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.091004:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.091005:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607c38200. 02000000:00000001:2.0:1787827022.091007:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.091008:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.091009:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff9856086aa1c0. 00000100:00000001:2.0:1787827022.091011:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.091012:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.091016:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9856086aa580 x1874669228019584/t219043334480(219043334480) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.091022:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.091025:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9856086aa580 x1874669228019584/t219043334480(219043334480) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.091031:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.091032:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.091033:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98560860d400. 02000000:00000001:2.0:1787827022.091035:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.091036:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.091037:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=32 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.091039:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.091040:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607c39600. 02000000:00000001:2.0:1787827022.091042:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.091043:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.091044:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff9856086aa580. 00000100:00000001:2.0:1787827022.091046:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.091047:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.091050:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9856086a8f00 x1874669228019968/t219043334481(219043334481) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.091056:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.091059:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9856086a8f00 x1874669228019968/t219043334481(219043334481) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.091066:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.091067:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.091068:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98560860f800. 02000000:00000001:2.0:1787827022.091070:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.091071:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.091072:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=31 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.091074:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.091075:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607c38000. 02000000:00000001:2.0:1787827022.091077:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.091078:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.091080:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff9856086a8f00. 00000100:00000001:2.0:1787827022.091082:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.091083:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.091088:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9856086a9e00 x1874669228020352/t219043334482(219043334482) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.091094:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.091097:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9856086a9e00 x1874669228020352/t219043334482(219043334482) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.091103:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.091104:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.091105:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98560860cc00. 02000000:00000001:2.0:1787827022.091108:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.091109:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.091110:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=30 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.091112:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.091113:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607c38e00. 02000000:00000001:2.0:1787827022.091115:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.091116:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.091117:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff9856086a9e00. 00000100:00000001:2.0:1787827022.091119:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.091120:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.091125:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9856086a92c0 x1874669228020736/t219043334483(219043334483) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.091136:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.091139:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9856086a92c0 x1874669228020736/t219043334483(219043334483) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.091145:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.091146:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.091147:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98560860f000. 02000000:00000001:2.0:1787827022.091149:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.091150:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.091151:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=29 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.091153:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.091154:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607c38c00. 02000000:00000001:2.0:1787827022.091156:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.091157:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.091158:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff9856086a92c0. 00000100:00000001:2.0:1787827022.091160:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.091161:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.091165:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607eb1e00 x1874669228021120/t219043334484(219043334484) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.091170:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.091172:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607eb1e00 x1874669228021120/t219043334484(219043334484) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.091178:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.091179:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.091180:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98560860c800. 02000000:00000001:2.0:1787827022.091182:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.091183:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.091184:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=28 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.091186:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.091187:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607c39800. 02000000:00000001:2.0:1787827022.091189:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.091190:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.091192:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607eb1e00. 00000100:00000001:2.0:1787827022.091195:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.091196:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.091200:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607eb12c0 x1874669228021504/t219043334485(219043334485) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.091206:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.091209:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607eb12c0 x1874669228021504/t219043334485(219043334485) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.091215:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.091216:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.091218:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98560860c000. 02000000:00000001:2.0:1787827022.091220:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.091221:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.091222:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=27 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.091224:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.091225:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607c39000. 02000000:00000001:2.0:1787827022.091227:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.091228:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.091229:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607eb12c0. 00000100:00000001:2.0:1787827022.091231:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.091232:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.091237:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607eb2d00 x1874669228021888/t219043334486(219043334486) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.091242:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.091245:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607eb2d00 x1874669228021888/t219043334486(219043334486) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.091251:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.091252:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.091253:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98560860f400. 02000000:00000001:2.0:1787827022.091255:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.091256:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.091258:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=26 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.091260:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.091261:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607c39e00. 02000000:00000001:2.0:1787827022.091263:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.091264:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.091266:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607eb2d00. 00000100:00000001:2.0:1787827022.091268:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.091269:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.091273:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607eb1680 x1874669228022272/t219043334487(219043334487) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.091280:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.091283:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607eb1680 x1874669228022272/t219043334487(219043334487) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.091289:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.091290:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.091292:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98560860ec00. 02000000:00000001:2.0:1787827022.091294:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.091295:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.091296:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=25 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.091298:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.091299:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607c39a00. 02000000:00000001:2.0:1787827022.091301:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.091302:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.091303:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607eb1680. 00000100:00000001:2.0:1787827022.091305:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.091307:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.091311:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607eb03c0 x1874669228022656/t219043334488(219043334488) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.091317:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.091320:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607eb03c0 x1874669228022656/t219043334488(219043334488) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.091327:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.091328:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.091329:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98560860d800. 02000000:00000001:2.0:1787827022.091331:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.091333:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.091334:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=24 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.091335:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.091337:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985608530200. 02000000:00000001:2.0:1787827022.091339:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.091340:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.091341:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607eb03c0. 00000100:00000001:2.0:1787827022.091345:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.091346:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.091350:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607eb3480 x1874669228023040/t219043334489(219043334489) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.091357:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.091360:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607eb3480 x1874669228023040/t219043334489(219043334489) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.091365:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.091367:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.091368:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98560860dc00. 02000000:00000001:2.0:1787827022.091370:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.091371:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.091372:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=23 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.091374:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.091376:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607f55200. 02000000:00000001:2.0:1787827022.091378:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.091379:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.091380:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607eb3480. 00000100:00000001:2.0:1787827022.091382:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.091383:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.091388:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607eb1a40 x1874669228023424/t219043334490(219043334490) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.091394:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.091396:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607eb1a40 x1874669228023424/t219043334490(219043334490) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.091402:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.091403:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.091405:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985607eb5c00. 02000000:00000001:2.0:1787827022.091407:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.091408:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.091409:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=22 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.091411:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.091412:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607f55600. 02000000:00000001:2.0:1787827022.091414:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.091415:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.091416:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607eb1a40. 00000100:00000001:2.0:1787827022.091418:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.091419:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.091423:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607eb30c0 x1874669228023808/t219043334491(219043334491) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.091429:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.091431:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607eb30c0 x1874669228023808/t219043334491(219043334491) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.091437:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.091438:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.091439:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985607eb5000. 02000000:00000001:2.0:1787827022.091441:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.091442:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.091443:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=21 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.091445:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.091446:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607f54000. 02000000:00000001:2.0:1787827022.091448:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.091449:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.091450:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607eb30c0. 00000100:00000001:2.0:1787827022.091452:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.091453:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.091457:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607eb0780 x1874669228024192/t219043334492(219043334492) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.091463:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.091465:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607eb0780 x1874669228024192/t219043334492(219043334492) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.091471:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.091473:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.091474:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985607eb6800. 02000000:00000001:2.0:1787827022.091477:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.091478:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.091479:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=20 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.091480:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.091481:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607f54e00. 02000000:00000001:2.0:1787827022.091483:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.091484:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.091485:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607eb0780. 00000100:00000001:2.0:1787827022.091487:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.091489:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.091492:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607eb0000 x1874669228024576/t219043334493(219043334493) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.091499:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.091502:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607eb0000 x1874669228024576/t219043334493(219043334493) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.091507:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.091509:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.091510:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985607eb6400. 02000000:00000001:2.0:1787827022.091512:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.091514:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.091515:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=19 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.091516:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.091518:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607f54c00. 02000000:00000001:2.0:1787827022.091520:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.091521:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.091522:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607eb0000. 00000100:00000001:2.0:1787827022.091525:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.091526:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.091530:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607eb3840 x1874669228024960/t219043334494(219043334494) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.091541:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.091544:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607eb3840 x1874669228024960/t219043334494(219043334494) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.091550:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.091551:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.091553:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985607eb4400. 02000000:00000001:2.0:1787827022.091555:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.091556:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.091557:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=18 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.091559:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.091560:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607f55800. 02000000:00000001:2.0:1787827022.091562:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.091563:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.091564:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607eb3840. 00000100:00000001:2.0:1787827022.091566:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.091568:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.091572:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607eb3c00 x1874669228025344/t219043334495(219043334495) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.091577:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.091581:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607eb3c00 x1874669228025344/t219043334495(219043334495) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.091587:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.091588:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.091589:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985607eb7c00. 02000000:00000001:2.0:1787827022.091591:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.091592:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.091593:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=17 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.091595:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.091596:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607f55000. 02000000:00000001:2.0:1787827022.091598:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.091599:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.091600:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607eb3c00. 00000100:00000001:2.0:1787827022.091602:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.091603:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.091607:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607eb0b40 x1874669228025728/t219043334496(219043334496) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.091612:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.091615:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607eb0b40 x1874669228025728/t219043334496(219043334496) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.091620:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.091622:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.091623:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985607eb6000. 02000000:00000001:2.0:1787827022.091625:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.091626:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.091626:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=16 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.091628:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.091629:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607f55e00. 02000000:00000001:2.0:1787827022.091630:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.091631:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.091632:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607eb0b40. 00000100:00000001:2.0:1787827022.091634:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.091635:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.091639:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607eb2940 x1874669228026112/t219043334497(219043334497) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.091644:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.091647:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607eb2940 x1874669228026112/t219043334497(219043334497) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.091653:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.091654:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.091655:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985607eb5400. 02000000:00000001:2.0:1787827022.091658:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.091659:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.091660:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=15 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.091662:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.091663:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607f55a00. 02000000:00000001:2.0:1787827022.091665:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.091666:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.091667:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607eb2940. 00000100:00000001:2.0:1787827022.091669:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.091670:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.091675:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9856140d83c0 x1874669228026496/t219043334498(219043334498) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.091681:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.091684:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9856140d83c0 x1874669228026496/t219043334498(219043334498) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.091709:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.091710:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.091712:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9856110f0000. 02000000:00000001:2.0:1787827022.091715:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.091716:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.091717:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=14 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.091719:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.091721:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff98560540ac00. 02000000:00000001:2.0:1787827022.091723:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.091724:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.091725:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff9856140d83c0. 00000100:00000001:2.0:1787827022.091728:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.091729:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.091733:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607eb2580 x1874669228026880/t219043334499(219043334499) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.091741:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.091743:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607eb2580 x1874669228026880/t219043334499(219043334499) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.091749:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.091750:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.091751:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985607eb4c00. 02000000:00000001:2.0:1787827022.091754:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.091756:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.091757:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=13 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.091759:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.091760:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607f55c00. 02000000:00000001:2.0:1787827022.091762:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.091764:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.091765:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607eb2580. 00000100:00000001:2.0:1787827022.091767:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.091768:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.091772:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607eb0f00 x1874669228027264/t219043334500(219043334500) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.091779:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.091782:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607eb0f00 x1874669228027264/t219043334500(219043334500) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.091788:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.091789:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.091790:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985607eb7000. 02000000:00000001:2.0:1787827022.091793:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.091794:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.091795:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=12 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.091797:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.091798:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607f54600. 02000000:00000001:2.0:1787827022.091800:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.091801:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.091802:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607eb0f00. 00000100:00000001:2.0:1787827022.091805:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.091806:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.091810:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607eb21c0 x1874669228027648/t219043334501(219043334501) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.091817:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.091820:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607eb21c0 x1874669228027648/t219043334501(219043334501) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.091826:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.091827:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.091829:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985607eb4800. 02000000:00000001:2.0:1787827022.091831:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.091844:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.091845:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=11 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.091847:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.091848:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607f54a00. 02000000:00000001:2.0:1787827022.091850:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.091851:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.091853:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607eb21c0. 00000100:00000001:2.0:1787827022.091855:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.091856:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.091860:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607f092c0 x1874669228028032/t219043334502(219043334502) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.091868:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.091870:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607f092c0 x1874669228028032/t219043334502(219043334502) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.091876:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.091878:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.091879:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985607eb4000. 02000000:00000001:2.0:1787827022.091881:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.091882:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.091884:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=10 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.091885:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.091886:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607f54400. 02000000:00000001:2.0:1787827022.091888:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.091889:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.091890:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607f092c0. 00000100:00000001:2.0:1787827022.091893:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.091894:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.091899:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607f0ad00 x1874669228028416/t219043334503(219043334503) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.091905:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.091908:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607f0ad00 x1874669228028416/t219043334503(219043334503) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.091914:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.091915:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.091917:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985607eb7400. 02000000:00000001:2.0:1787827022.091919:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.091920:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.091921:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=9 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.091923:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.091924:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607f54800. 02000000:00000001:2.0:1787827022.091926:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.091927:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.091928:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607f0ad00. 00000100:00000001:2.0:1787827022.091930:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.091932:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.091935:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607f09680 x1874669228028800/t219043334504(219043334504) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.091942:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.091944:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607f09680 x1874669228028800/t219043334504(219043334504) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.091951:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.091952:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.091953:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985607eb6c00. 02000000:00000001:2.0:1787827022.091955:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.091956:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.091958:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=8 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.091959:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.091961:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607f54200. 02000000:00000001:2.0:1787827022.091963:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.091964:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.091965:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607f09680. 00000100:00000001:2.0:1787827022.091967:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.091968:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.091972:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607f083c0 x1874669228029184/t219043334505(219043334505) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.091984:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.091986:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607f083c0 x1874669228029184/t219043334505(219043334505) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.091992:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.091993:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.091995:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985607eb5800. 02000000:00000001:2.0:1787827022.091996:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.091997:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.091998:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=7 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.092000:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.092001:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607c39c00. 02000000:00000001:2.0:1787827022.092003:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.092004:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.092005:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607f083c0. 00000100:00000001:2.0:1787827022.092007:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.092008:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.092013:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607e14000 x1874669228029568/t219043334506(219043334506) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.092019:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.092021:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607e14000 x1874669228029568/t219043334506(219043334506) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.092028:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.092029:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.092030:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985608539000. 02000000:00000001:2.0:1787827022.092032:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.092033:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.092034:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=6 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.092036:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.092037:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff98561288ca00. 02000000:00000001:2.0:1787827022.092039:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.092040:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.092041:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607e14000. 00000100:00000001:2.0:1787827022.092043:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.092044:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.092048:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607e17840 x1874669228029952/t219043334507(219043334507) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.092054:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.092057:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607e17840 x1874669228029952/t219043334507(219043334507) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.092062:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.092064:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.092065:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98560853a800. 02000000:00000001:2.0:1787827022.092067:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.092068:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.092069:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=5 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.092071:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.092072:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff98561288c400. 02000000:00000001:2.0:1787827022.092074:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.092075:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.092076:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607e17840. 00000100:00000001:2.0:1787827022.092078:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.092079:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.092083:0:2385:0:(client.c:3181:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff985607e17c00 x1874669228030336/t219043334508(219043334508) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.092089:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.092092:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607e17c00 x1874669228030336/t219043334508(219043334508) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 504/456 e 0 to 0 dl 1787827034 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.092097:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.092099:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.092100:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff98560853a400. 02000000:00000001:2.0:1787827022.092102:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.092103:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.092104:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=4 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.092106:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.092108:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff98561288c800. 02000000:00000001:2.0:1787827022.092110:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.092111:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.092113:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607e17c00. 00000100:00000001:2.0:1787827022.092114:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.092116:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1787827022.092118:0:2385:0:(client.c:3188:ptlrpc_free_committed()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000040:2.0:1787827022.092121:0:2385:0:(client.c:3235:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: normal: skip=1 replay=402/402 committed=0/0 00000100:00000001:2.0:1787827022.092124:0:2385:0:(client.c:3236:ptlrpc_free_committed()) Process leaving 00000100:00000001:2.0:1787827022.092126:0:2385:0:(client.c:1797:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1787827022.092132:0:2385:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9856140dad00 x1874669228030592/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827038 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.092140:0:2385:0:(client.c:2354:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1787827022.092143:0:2385:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1787827022.092145:0:2385:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.092148:0:2385:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9856140dad00 x1874669228030592/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827038 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827022.092156:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669228030592:192.168.203.114@tcp:400:kworker.0 00000400:00000001:2.0:1787827022.092168:0:2385:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1787827022.092170:0:2385:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1787827022.092173:0:2385:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff98561080ef78. 00000400:00000200:2.0:1787827022.092176:0:2385:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985610afa880 00000400:00000010:2.0:1787827022.092179:0:2385:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985610afa880. 00000100:00000001:2.0:1787827022.092182:0:2385:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1787827022.092186:0:2385:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff9856140db0c0 x1874669228030848/t0(0) o400->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827038 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1787827022.092195:0:2385:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff9856140db0c0 x1874669228030848/t0(0) o400->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827038 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.092202:0:2385:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1787827022.092204:0:2385:0:(client.c:3001:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1787827022.092206:0:2385:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:2.0:1787827022.092208:0:2385:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1787827022.092211:0:2385:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1787827022.092213:0:2385:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1787827022.092216:0:2385:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1787827022.092224:0:2385:0:(import.c:2149:obd_at_measure()) add 0 to ffff9856110b0460 time=100 v=5 (5 5 5 5) 00000100:00001000:2.0:1787827022.092227:0:2385:0:(import.c:2149:obd_at_measure()) add 1 to ffff9856110b0418 time=151 v=5 (1 1 1 1) 00000100:00000001:2.0:1787827022.092230:0:2385:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1787827022.092232:0:2385:0:(client.c:1506:ptlrpc_check_status()) Process leaving (rc=18446744073709551509 : -107 : ffffffffffffff95) 00000100:00000001:2.0:1787827022.092236:0:2385:0:(recover.c:202:ptlrpc_request_handle_notconn()) Process entered 00000100:00080000:2.0:1787827022.092238:0:2385:0:(recover.c:205:ptlrpc_request_handle_notconn()) import lustre-OST0000-osc-ffff985602bc9800 of lustre-OST0000_UUID@192.168.203.114@tcp abruptly disconnected: reconnecting 00000100:00080000:2.0:1787827022.092243:0:2385:0:(import.c:65:import_set_state_nolock()) ffff9856110b0000 lustre-OST0000_UUID: changing import state from FULL to DISCONN 00000100:02000400:2.0:1787827022.092246:0:2385:0:(import.c:182:ptlrpc_set_import_discon()) lustre-OST0000-osc-ffff985602bc9800: Connection to lustre-OST0000 (at 192.168.203.114@tcp) was lost; in progress operations using this service will wait for recovery to complete 00000100:00000001:2.0:1787827022.092250:0:2385:0:(obd_class.h:1222:obd_import_event()) Process entered 00000008:00000001:2.0:1787827022.092253:0:2385:0:(osc_request.c:4091:osc_import_event()) Process entered 00000008:00000001:2.0:1787827022.092256:0:2385:0:(osc_request.c:4158:osc_import_event()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1787827022.092258:0:2385:0:(obd_class.h:1233:obd_import_event()) Process leaving 00000100:00000001:2.0:1787827022.092260:0:2385:0:(import.c:701:ptlrpc_connect_import_locked()) Process entered 00000100:00080000:2.0:1787827022.092262:0:2385:0:(import.c:65:import_set_state_nolock()) ffff9856110b0000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00000001:2.0:1787827022.092265:0:2385:0:(import.c:485:import_select_connection()) Process entered 00000100:00080000:2.0:1787827022.092268:0:2385:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 922 00000400:00000200:2.0:1787827022.092274:0:2385:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827022.092279:0:2385:0:(peer.c:1685:LNetPeerDiscovered()) Peer NID 192.168.203.114@tcp is discovered: rc = 1 00000100:00000040:2.0:1787827022.092283:0:2385:0:(lustre_net.h:1965:ptlrpc_connection_put()) PUT conn=ffff98561201f640 refcount 14 to 192.168.203.114@tcp 00000100:00000001:2.0:1787827022.092286:0:2385:0:(connection.c:156:ptlrpc_connection_addref()) Process entered 00000100:00000040:2.0:1787827022.092288:0:2385:0:(connection.c:161:ptlrpc_connection_addref()) conn=ffff98561201f640 refcount 15 to 192.168.203.114@tcp 00000100:00000001:2.0:1787827022.092290:0:2385:0:(connection.c:163:ptlrpc_connection_addref()) Process leaving (rc=18446630094169568832 : -113979539982784 : ffff98561201f640) 00000020:00000001:2.0:1787827022.092293:0:2385:0:(genops.c:792:class_conn2export()) Process entered 00000020:00000040:2.0:1787827022.092295:0:2385:0:(genops.c:804:class_conn2export()) looking for export cookie 0xb3f83dad1b7dcdd6 00000020:00000001:2.0:1787827022.092297:0:2385:0:(lustre_handles.c:130:class_handle2object()) Process entered 00000020:00000040:2.0:1787827022.092300:0:2385:0:(lustre_handles.c:148:class_handle2object()) GET export ffff985602cdf000 refcount=4 00000020:00000001:2.0:1787827022.092303:0:2385:0:(lustre_handles.c:155:class_handle2object()) Process leaving (rc=18446630093914501120 : -113979795050496 : ffff985602cdf000) 00000020:00000001:2.0:1787827022.092305:0:2385:0:(genops.c:806:class_conn2export()) Process leaving (rc=18446630093914501120 : -113979795050496 : ffff985602cdf000) 00000100:00000040:2.0:1787827022.092308:0:2385:0:(lustre_net.h:1965:ptlrpc_connection_put()) PUT conn=ffff98561201f640 refcount 14 to 192.168.203.114@tcp 00000100:00000001:2.0:1787827022.092310:0:2385:0:(connection.c:156:ptlrpc_connection_addref()) Process entered 00000100:00000040:2.0:1787827022.092312:0:2385:0:(connection.c:161:ptlrpc_connection_addref()) conn=ffff98561201f640 refcount 15 to 192.168.203.114@tcp 00000100:00000001:2.0:1787827022.092314:0:2385:0:(connection.c:163:ptlrpc_connection_addref()) Process leaving (rc=18446630094169568832 : -113979539982784 : ffff98561201f640) 00000020:00000040:2.0:1787827022.092317:0:2385:0:(genops.c:879:class_export_put()) PUTting export ffff985602cdf000 : new refcount 3 00000100:00080000:2.0:1787827022.092320:0:2385:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff985602bc9800: import ffff9856110b0000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00000001:2.0:1787827022.092323:0:2385:0:(import.c:628:import_select_connection()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1787827022.092326:0:2385:0:(sec.c:1579:sptlrpc_import_sec_adapt()) Process entered 02000000:00000001:2.0:1787827022.092332:0:2385:0:(sec.c:1619:sptlrpc_import_sec_adapt()) Process leaving via out (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1787827022.092335:0:2385:0:(sec.c:1647:sptlrpc_import_sec_adapt()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1787827022.092336:0:2385:0:(obd_class.h:889:obd_reconnect()) Process entered 00000008:00100000:2.0:1787827022.092340:0:2385:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683216 ocd_grant: 8388608, lost: 0. 00000008:00000001:2.0:1787827022.092343:0:2385:0:(osc_request.c:4022:osc_reconnect()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1787827022.092345:0:2385:0:(obd_class.h:903:obd_reconnect()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:2.0:1787827022.092351:0:2385:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 904 at ffff9856140d83c0. 00000020:00000040:2.0:1787827022.092355:0:2385:0:(genops.c:1210:class_import_get()) import ffff9856110b0000 refcount=4 obd=lustre-OST0000-osc-ffff985602bc9800 02000000:00000001:2.0:1787827022.092358:0:2385:0:(sec.c:2038:sptlrpc_sepol_get()) Process entered 02000000:00000001:2.0:1787827022.092360:0:2385:0:(sec.c:2048:sptlrpc_sepol_get()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1787827022.092363:0:2385:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1787827022.092366:0:2385:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1787827022.092369:0:2385:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1787827022.092374:0:2385:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 1024 at ffff9856110f0000. 00000100:00000001:2.0:1787827022.092380:0:2385:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1787827022.092386:0:2385:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d83c0 x1874669228031104/t0(0) o8->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1787827022.092395:0:2385:0:(jobid.c:939:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1787827022.092399:0:2385:0:(jobid.c:979:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1787827022.092404:0:2385:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000008c51c1e6] to pc [ptlrpcd_rcv+-1] req@ffff9856140d83c0 x1874669228031104/t0(0) o8->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'ptlrpcd_01_00.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.092432:0:2385:0:(import.c:833:ptlrpc_connect_import_locked()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1787827022.092435:0:2385:0:(recover.c:221:ptlrpc_request_handle_notconn()) Process leaving 00000100:00000001:2.0:1787827022.092437:0:2385:0:(client.c:1714:after_reply()) Process leaving (rc=18446744073709551509 : -107 : ffffffffffffff95) 00000100:00000040:2.0:1787827022.092442:0:2385:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9856140db0c0 x1874669228030848/t0(0) o400->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827038 ref 1 fl Rpc:RNQU/200/0 rc -107/-107 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.092448:0:2385:0:(client.c:2354:ptlrpc_check_set()) Process leaving via interpret (rc=18446744073709551509 : -107 : 0xffffffffffffff95) 00000100:00000001:2.0:1787827022.092451:0:2385:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1787827022.092452:0:2385:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827022.092455:0:2385:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9856140db0c0 x1874669228030848/t0(0) o400->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827038 ref 1 fl Interpret:RNQU/200/0 rc -107/-107 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827022.092463:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669228030848:192.168.203.114@tcp:400:kworker.0 00000100:00000001:2.0:1787827022.092467:0:2385:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1787827022.092469:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.092472:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9856140dad00 x1874669228030592/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827038 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.092477:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.092479:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.092480:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff985611861800. 02000000:00000001:2.0:1787827022.092483:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.092485:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.092486:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=3 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:2.0:1787827022.092488:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.092489:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff985615779800. 02000000:00000001:2.0:1787827022.092492:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.092494:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.092495:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff9856140dad00. 00000100:00000001:2.0:1787827022.092499:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.092500:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1787827022.092502:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827022.092505:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9856140db0c0 x1874669228030848/t0(0) o400->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827038 ref 1 fl Complete:RNQU/200/0 rc -107/-107 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.092511:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827022.092512:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827022.092514:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff985611860000. 02000000:00000001:2.0:1787827022.092516:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827022.092517:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827022.092518:0:2385:0:(genops.c:1223:class_import_put()) import ffff9856110b0000 refcount=3 obd=lustre-OST0000-osc-ffff985602bc9800 00000020:00000001:2.0:1787827022.092520:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827022.092521:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff985615779b00. 02000000:00000001:2.0:1787827022.092523:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827022.092524:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827022.092525:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff9856140db0c0. 00000100:00000001:2.0:1787827022.092527:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827022.092529:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1787827022.092530:0:2385:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1787827022.092537:0:2385:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1787827022.092539:0:2385:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1787827022.092541:0:2385:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1787827022.092544:0:2385:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827022.092625:0:2382:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1787827022.092629:0:2382:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1787827022.092631:0:2382:0:(client.c:1813:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1787827022.092637:0:2382:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9856140d83c0 x1874669228031104/t0(0) o8->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'ptlrpcd_01_00.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827022.092644:0:2382:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1787827022.092646:0:2382:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1787827022.092649:0:2382:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1787827022.092652:0:2382:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1787827022.092654:0:2382:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1787827022.092657:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228031104:192.168.203.114@tcp:8:ptlrpcd_01_00.0 00000100:00000001:1.0:1787827022.092661:0:2382:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1787827022.092664:0:2382:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1787827022.092666:0:2382:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1787827022.092669:0:2382:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1787827022.092673:0:2382:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 1024 at ffff985607d03400. 02000000:00000001:1.0:1787827022.092675:0:2382:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1787827022.092680:0:2382:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000010:1.0:1787827022.092684:0:2382:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff985602ecf898. 00000400:00000010:1.0:1787827022.092709:0:2382:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985606618aa0. 00000100:00000200:1.0:1787827022.092715:0:2382:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 1024 bytes, xid 1874669228031104, portal 4 00000100:00000001:1.0:1787827022.092717:0:2382:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1787827022.092719:0:2382:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446630094203880384 : -113979505671232 : ffff9856140d83c0) 00000100:00000040:1.0:1787827022.092725:0:2382:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9856140d83c0 x1874669228031104/t0(0) o8->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 520/544 e 0 to 0 dl 1787827032 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'ptlrpcd_01_00.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827022.092732:0:2382:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1787827022.092734:0:2382:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.203.114@tcp 00000400:00000010:1.0:1787827022.092737:0:2382:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985606618ee0. 00000100:00000200:1.0:1787827022.092740:0:2382:0:(niobuf.c:94:ptl_send_buf()) Sending 520 bytes to portal 28, xid 1874669228031104, offset 0 00000400:00000200:1.0:1787827022.092746:0:2382:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.203.114@tcp 00000400:00000200:1.0:1787827022.092752:0:2382:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:1.0:1787827022.092758:0:2382:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.203.114@tcp local destination 00000400:00000200:1.0:1787827022.092763:0:2382:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1787827022.092768:0:2382:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.203.14@tcp 00000400:00000200:1.0:1787827022.092771:0:2382:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.203.14@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1787827022.092774:0:2382:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.203.114@tcp 00000400:00000200:1.0:1787827022.092778:0:2382:0:(lib-move.c:1745:lnet_handle_send()) 192.168.203.14@tcp NI seq info: [21879:21879:256:4294967295] 192.168.203.114@tcp LPNI seq info [21879:21879:8:4294967295] 00000400:00000200:1.0:1787827022.092782:0:2382:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:1.0:1787827022.092792:0:2382:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp:) -> 192.168.203.114@tcp(192.168.203.114@tcp:192.168.203.114@tcp) : PUT try# 0 00000800:00000200:1.0:1787827022.092798:0:2382:0:(socklnd_cb.c:931:ksocknal_send()) sending 520 bytes in 1 frags to 12345-192.168.203.114@tcp 00000800:00000010:1.0:1787827022.092803:0:2382:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff985607d95900. 00000800:00000200:1.0:1787827022.092809:0:2382:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff985606751480] -> 12345-192.168.203.114@tcp (5) 00000800:00000200:1.0:1787827022.092816:0:2382:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.203.114@tcp ip 192.168.203.114:988 00000800:00000200:1.0:1787827022.092820:0:2382:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff985607d95900 type 1, nob 616 niov 1 nkiov 1 00000100:00000001:1.0:1787827022.092845:0:2382:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1787827022.092849:0:2382:0:(niobuf.c:635:kick_cpu_latency()) lustre-OST0000-osc-ffff985602bc9800: using a timeout of 14625 usecs (15 jiffies) 00000400:00000200:1.0:1787827022.092853:0:2382:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000001:1.0:1787827022.092855:0:2382:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff985602bc9800: PM QoS new deadline estimation for cpu 2 is 4297840167 00000100:00000001:1.0:1787827022.092858:0:2382:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff985602bc9800: PM QoS request active for cpu 2, keep current deadline 4297850108 00000100:00000001:1.0:1787827022.092861:0:2382:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff985602bc9800: PM QoS new deadline estimation for cpu 3 is 4297840167 00000100:00000001:1.0:1787827022.092863:0:2382:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff985602bc9800: PM QoS request active for cpu 3, keep current deadline 4297850108 00000100:00000001:1.0:1787827022.092865:0:2382:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1787827022.092868:0:2382:0:(client.c:1941:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827022.092871:0:2382:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827022.092873:0:2382:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787827022.092877:0:2382:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1787827022.092879:0:2382:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=5 : 5 : 5) 00000100:00000001:1.0:1787827022.092881:0:2382:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1787827022.092883:0:2382:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1787827022.092884:0:2382:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827022.092886:0:2382:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1787827022.093026:0:2378:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1787827022.093036:0:2378:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff985607d95900. 00000400:00000200:3.0:1787827022.093044:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1787827022.093052:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000400:00000200:3.0:1787827022.093060:0:2378:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985606618ee0 00000400:00000010:3.0:1787827022.093062:0:2378:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985606618ee0. 00000100:00000001:3.0:1787827022.093068:0:2378:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1787827022.093075:0:2378:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff9856140d83c0 x1874669228031104/t0(0) o8->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 520/544 e 0 to 0 dl 1787827032 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'ptlrpcd_01_00.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827022.093086:0:2378:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1787827022.093090:0:2378:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9856140d83c0 x1874669228031104/t0(0) o8->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 520/544 e 0 to 0 dl 1787827032 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'ptlrpcd_01_00.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827022.093160:0:2378:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827022.093162:0:2378:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1787827022.096311:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827022.096330:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1787827022.096334:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827022.096337:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827022.096345:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827022.096352:0:2377:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp) <- 192.168.203.114@tcp : PUT - for me 00000400:00000200:2.0:1787827022.096371:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827022.096376:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827022.096381:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827022.096385:0:2377:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.203.114@tcp of length 224 into portal 4 MB=0x6a90071687c80 00000400:00000200:2.0:1787827022.096390:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827022.096395:0:2377:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 4 from 12345-192.168.203.114@tcp of length 224/224 into md 0x32285 [1] + 0 00000800:00000001:2.0:1787827022.096402:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827022.096413:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827022.096416:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1787827022.096420:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000100:00000001:2.0:1787827022.096428:0:2377:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1787827022.096435:0:2377:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff9856140d83c0 x1874669228031104/t0(0) o8->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 520/544 e 0 to 0 dl 1787827032 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'ptlrpcd_01_00.0' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1787827022.096453:0:2377:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=0 replen=544 req@ffff9856140d83c0 x1874669228031104/t0(0) o8->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 520/544 e 0 to 0 dl 1787827032 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'ptlrpcd_01_00.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827022.096476:0:2377:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1787827022.096484:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827022.096487:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787827022.096521:0:2382:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1787827022.096525:0:2382:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1787827022.096531:0:2382:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1787827022.096532:0:2382:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1787827022.096536:0:2382:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff985602ecf898. 00000400:00000200:1.0:1787827022.096539:0:2382:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985606618aa0 00000400:00000010:1.0:1787827022.096540:0:2382:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985606618aa0. 00000100:00000001:1.0:1787827022.096543:0:2382:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1787827022.096547:0:2382:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff9856140d83c0 x1874669228031104/t0(0) o8->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 520/544 e 0 to 0 dl 1787827032 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'ptlrpcd_01_00.0' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1787827022.096554:0:2382:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff9856140d83c0 x1874669228031104/t0(0) o8->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 520/544 e 0 to 0 dl 1787827032 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'ptlrpcd_01_00.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827022.096559:0:2382:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1787827022.096566:0:2382:0:(client.c:3001:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787827022.096569:0:2382:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:1.0:1787827022.096575:0:2382:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1787827022.096577:0:2382:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1787827022.096579:0:2382:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1787827022.096583:0:2382:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1787827022.096592:0:2382:0:(import.c:2149:obd_at_measure()) add 0 to ffff9856110b0460 time=100 v=5 (5 5 5 5) 00000100:00001000:1.0:1787827022.096596:0:2382:0:(import.c:2149:obd_at_measure()) add 1 to ffff9856110b0418 time=1 v=5 (1 1 1 1) 00000100:00000001:1.0:1787827022.096598:0:2382:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1787827022.096600:0:2382:0:(client.c:1506:ptlrpc_check_status()) Process leaving (rc=18446744073709551597 : -19 : ffffffffffffffed) 00000100:00000001:1.0:1787827022.096603:0:2382:0:(client.c:1711:after_reply()) Process leaving (rc=18446744073709551597 : -19 : ffffffffffffffed) 00000100:00000040:1.0:1787827022.096608:0:2382:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9856140d83c0 x1874669228031104/t0(0) o8->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 520/224 e 0 to 0 dl 1787827032 ref 1 fl Rpc:RNQU/200/0 rc -19/-19 job:'ptlrpcd_01_00.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827022.096616:0:2382:0:(client.c:2354:ptlrpc_check_set()) Process leaving via interpret (rc=18446744073709551597 : -19 : 0xffffffffffffffed) 00000100:00000001:1.0:1787827022.096620:0:2382:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1787827022.096622:0:2382:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787827022.096624:0:2382:0:(import.c:1057:ptlrpc_connect_interpret()) Process entered 00000100:00000001:1.0:1787827022.096626:0:2382:0:(import.c:1092:ptlrpc_connect_interpret()) Process leaving via out (rc=18446744073709551597 : -19 : 0xffffffffffffffed) 00000100:00080000:1.0:1787827022.096629:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9856110b0000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827022.096633:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.114@tcp failed (-19) 00000100:00000001:1.0:1787827022.096636:0:2382:0:(import.c:1517:ptlrpc_connect_interpret()) Process leaving (rc=18446744073709551597 : -19 : ffffffffffffffed) 00000100:00000040:1.0:1787827022.096639:0:2382:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9856140d83c0 x1874669228031104/t0(0) o8->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 520/224 e 0 to 0 dl 1787827032 ref 1 fl Interpret:RNQU/200/0 rc -19/-19 job:'ptlrpcd_01_00.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827022.096648:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228031104:192.168.203.114@tcp:8:ptlrpcd_01_00.0 00000100:00000001:1.0:1787827022.096653:0:2382:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787827022.096656:0:2382:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1787827022.096659:0:2382:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9856140d83c0 x1874669228031104/t0(0) o8->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 520/224 e 0 to 0 dl 1787827032 ref 1 fl Complete:RNQU/200/0 rc -19/-19 job:'ptlrpcd_01_00.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827022.096663:0:2382:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1787827022.096665:0:2382:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1787827022.096668:0:2382:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985607d03400. 02000000:00000001:1.0:1787827022.096671:0:2382:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1787827022.096673:0:2382:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:1.0:1787827022.096675:0:2382:0:(genops.c:1223:class_import_put()) import ffff9856110b0000 refcount=2 obd=lustre-OST0000-osc-ffff985602bc9800 00000020:00000001:1.0:1787827022.096676:0:2382:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:1.0:1787827022.096678:0:2382:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 1024 at ffff9856110f0000. 02000000:00000001:1.0:1787827022.096681:0:2382:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1787827022.096683:0:2382:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1787827022.096685:0:2382:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff9856140d83c0. 00000100:00000001:1.0:1787827022.096688:0:2382:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1787827022.096714:0:2382:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787827022.096717:0:2382:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787827022.096723:0:2382:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1787827022.096725:0:2382:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827022.096728:0:2382:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1787827022.096729:0:2382:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827022.644797:0:74885:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827022.644802:0:74885:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827022.644806:0:74885:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827022.644807:0:74885:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827022.644809:0:74885:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827022.644810:0:74885:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827022.644812:0:74885:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827022.644813:0:74885:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1787827022.644869:0:79887:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1787827022.644872:0:79887:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1787827022.644875:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:0.0:1787827023.668841:0:74885:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827023.668848:0:74885:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827023.668854:0:74885:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827023.668855:0:74885:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827023.668858:0:74885:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827023.668859:0:74885:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827023.668862:0:74885:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827023.668863:0:74885:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1787827023.668915:0:15163:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1787827023.668918:0:15163:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1787827023.668920:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:0.0:1787827024.692803:0:74885:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827024.692810:0:74885:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827024.692979:0:74885:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827024.692981:0:74885:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827024.692985:0:74885:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827024.692986:0:74885:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827024.692989:0:74885:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827024.692990:0:74885:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1787827024.693059:0:33341:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1787827024.693063:0:33341:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1787827024.693067:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:0.0:1787827025.716820:0:74885:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827025.716826:0:74885:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827025.716833:0:74885:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827025.716834:0:74885:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827025.716837:0:74885:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827025.716838:0:74885:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827025.716841:0:74885:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827025.716845:0:74885:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:0.0:1787827025.716846:0:74885:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:0.0:1787827025.716852:0:74885:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:0.0:1787827025.716855:0:74885:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:0.0:1787827025.716857:0:74885:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827025.716859:0:74885:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:0.0:1787827025.716862:0:74885:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827025.716863:0:74885:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827025.716866:0:74885:0:(ldlm_pool.c:496:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1787827025.716891:0:15164:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1787827025.716894:0:15164:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1787827025.716896:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:0.0:1787827026.740795:0:74885:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827026.740812:0:74885:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827026.740819:0:74885:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827026.740822:0:74885:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:0.0:1787827026.740825:0:74885:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:0.0:1787827026.740828:0:74885:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827026.740831:0:74885:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:0.0:1787827026.740832:0:74885:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827026.740834:0:74885:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827026.740837:0:74885:0:(ldlm_pool.c:496:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827026.740840:0:74885:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827026.740842:0:74885:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827026.740845:0:74885:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827026.740846:0:74885:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1787827026.740869:0:33342:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1787827026.740872:0:33342:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1787827026.740875:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:3.0:1787827027.188829:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1787827027.188839:0:37:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1787827027.188847:0:37:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 904 at ffff985610f83c00. 00000020:00000040:3.0:1787827027.188853:0:37:0:(genops.c:1210:class_import_get()) import ffff98561118e000 refcount=4 obd=MGC192.168.203.114@tcp 00000100:00000001:3.0:1787827027.188857:0:37:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1787827027.188860:0:37:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1787827027.188864:0:37:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1787827027.188871:0:37:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff985609025a00. 00000100:00000001:3.0:1787827027.188878:0:37:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1787827027.188889:0:37:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS req@ffff985610f83c00 x1874669228031232/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1787827027.188900:0:37:0:(jobid.c:939:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1787827027.188905:0:37:0:(jobid.c:979:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1787827027.188911:0:37:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000759c64fa] to pc [ptlrpcd_01_00+0] req@ffff985610f83c00 x1874669228031232/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827027.188938:0:37:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1787827027.188942:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1787827027.188946:0:37:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1787827027.188949:0:37:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 904 at ffff985610f82940. 00000020:00000040:3.0:1787827027.188952:0:37:0:(genops.c:1210:class_import_get()) import ffff985611182000 refcount=4 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000100:00000001:3.0:1787827027.188955:0:37:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:2.0:1787827027.188956:0:2385:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:3.0:1787827027.188957:0:37:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1787827027.188960:0:37:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1787827027.188962:0:2385:0:(client.c:1990:ptlrpc_check_set()) Process entered 02000000:00000010:3.0:1787827027.188965:0:37:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff985609025f00. 00000100:00000001:2.0:1787827027.188967:0:2385:0:(client.c:1813:ptlrpc_send_new_req()) Process entered 00000100:00000001:3.0:1787827027.188970:0:37:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1787827027.188976:0:2385:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff985610f83c00 x1874669228031232/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1787827027.188980:0:37:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID req@ffff985610f82940 x1874669228031360/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1787827027.188999:0:2385:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1787827027.189002:0:2385:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1787827027.189009:0:2385:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:3.0:1787827027.189011:0:37:0:(jobid.c:939:lustre_get_jobid()) Process entered 02000000:00000001:2.0:1787827027.189013:0:2385:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1787827027.189016:0:37:0:(jobid.c:979:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1787827027.189018:0:2385:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1787827027.189023:0:37:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000abc5c067] to pc [ptlrpcd_01_01+1] req@ffff985610f82940 x1874669228031360/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827027.189025:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2385:1874669228031232:192.168.203.114@tcp:400:kworker.0 00000100:00000001:2.0:1787827027.189033:0:2385:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1787827027.189041:0:2385:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:3.0:1787827027.189041:0:37:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1787827027.189045:0:2385:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1787827027.189045:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000001:2.0:1787827027.189050:0:2385:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:3.0:1787827027.189052:0:37:0:(pinger.c:119:ptlrpc_ping()) Process entered 02000000:00000010:2.0:1787827027.189056:0:2385:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff985611860000. 00000100:00000010:3.0:1787827027.189057:0:37:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 904 at ffff985610f821c0. 02000000:00000001:2.0:1787827027.189059:0:2385:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:3.0:1787827027.189062:0:37:0:(genops.c:1210:class_import_get()) import ffff985611185800 refcount=4 obd=lustre-MDT0001-mdc-ffff985602bc9800 00000100:00000001:3.0:1787827027.189066:0:37:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 00000400:00000200:2.0:1787827027.189068:0:2385:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 02000000:00000001:3.0:1787827027.189069:0:37:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 00000400:00000010:2.0:1787827027.189072:0:2385:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff98561080ef78. 02000000:00000001:3.0:1787827027.189073:0:37:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1787827027.189079:0:37:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff985609025200. 00000400:00000010:2.0:1787827027.189080:0:2385:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985610afa880. 00000100:00000200:2.0:1787827027.189087:0:2385:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1874669228031232, portal 25 00000100:00000001:2.0:1787827027.189090:0:2385:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1787827027.189090:0:37:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1787827027.189091:0:2385:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446630094152154112 : -113979557397504 : ffff985610f83c00) 00000100:00000040:2.0:1787827027.189102:0:2385:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff985610f83c00 x1874669228031232/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827043 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1787827027.189102:0:37:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID req@ffff985610f821c0 x1874669228031488/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1787827027.189119:0:2385:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:2.0:1787827027.189124:0:2385:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.203.114@tcp 00000100:00000001:3.0:1787827027.189125:0:37:0:(jobid.c:939:lustre_get_jobid()) Process entered 00000400:00000010:2.0:1787827027.189126:0:2385:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985610afa220. 00000100:00000200:2.0:1787827027.189130:0:2385:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1874669228031232, offset 0 00000100:00000001:3.0:1787827027.189130:0:37:0:(jobid.c:979:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1787827027.189140:0:37:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000fa701676] to pc [ptlrpcd_01_00+0] req@ffff985610f821c0 x1874669228031488/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1787827027.189141:0:2385:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.203.114@tcp 00000100:00000001:3.0:1787827027.189153:0:37:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1787827027.189154:0:2385:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00080000:3.0:1787827027.189157:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827027.189162:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000400:00000200:2.0:1787827027.189165:0:2385:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.203.114@tcp local destination 00000100:00000001:3.0:1787827027.189166:0:37:0:(import.c:701:ptlrpc_connect_import_locked()) Process entered 00000100:00080000:3.0:1787827027.189171:0:37:0:(import.c:65:import_set_state_nolock()) ffff9856110b0000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000400:00000200:2.0:1787827027.189173:0:2385:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:3.0:1787827027.189175:0:37:0:(import.c:485:import_select_connection()) Process entered 00000400:00000200:2.0:1787827027.189180:0:2385:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.203.14@tcp 00000100:00080000:3.0:1787827027.189184:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3172 00000400:00000200:2.0:1787827027.189188:0:2385:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.203.14@tcp lpni_is_preferred = 0 00000400:00000200:2.0:1787827027.189191:0:2385:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.203.114@tcp 00000400:00000200:2.0:1787827027.189195:0:2385:0:(lib-move.c:1745:lnet_handle_send()) 192.168.203.14@tcp NI seq info: [21880:21880:256:4294967295] 192.168.203.114@tcp LPNI seq info [21880:21880:8:4294967295] 00000400:00000200:2.0:1787827027.189198:0:2385:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827027.189206:0:2385:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp:) -> 192.168.203.114@tcp(192.168.203.114@tcp:192.168.203.114@tcp) : PUT try# 0 00000800:00000200:2.0:1787827027.189213:0:2385:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.203.114@tcp 00000400:00000200:3.0:1787827027.189214:0:37:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000800:00000010:2.0:1787827027.189218:0:2385:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff985615779b00. 00000400:00000200:3.0:1787827027.189222:0:37:0:(peer.c:1685:LNetPeerDiscovered()) Peer NID 192.168.203.114@tcp is discovered: rc = 1 00000800:00000200:2.0:1787827027.189227:0:2385:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff985606751480] -> 12345-192.168.203.114@tcp (5) 00000100:00001000:3.0:1787827027.189229:0:37:0:(import.c:2149:obd_at_measure()) add 10 to ffff9856110b0418 time=6 v=5 (1 1 1 1) 00000100:00001000:3.0:1787827027.189234:0:37:0:(import.c:2215:obd_at_measure()) AT ffff9856110b0418 change: old=5 new=10 delta=5 (val=10) hist 10 1 1 1 00000100:00080000:3.0:1787827027.189239:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff985602bc9800: tried all connections, increasing latency to 5s 00000800:00000200:2.0:1787827027.189242:0:2385:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.203.114@tcp ip 192.168.203.114:988 00000100:00000040:3.0:1787827027.189243:0:37:0:(lustre_net.h:1965:ptlrpc_connection_put()) PUT conn=ffff98561201f640 refcount 14 to 192.168.203.114@tcp 00000800:00000200:2.0:1787827027.189247:0:2385:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff985615779b00 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1787827027.189248:0:37:0:(connection.c:156:ptlrpc_connection_addref()) Process entered 00000100:00000040:3.0:1787827027.189253:0:37:0:(connection.c:161:ptlrpc_connection_addref()) conn=ffff98561201f640 refcount 15 to 192.168.203.114@tcp 00000100:00000001:3.0:1787827027.189256:0:37:0:(connection.c:163:ptlrpc_connection_addref()) Process leaving (rc=18446630094169568832 : -113979539982784 : ffff98561201f640) 00000100:00000001:2.0:1787827027.189257:0:2385:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1787827027.189262:0:37:0:(genops.c:792:class_conn2export()) Process entered 00000400:00000200:2.0:1787827027.189265:0:2385:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000020:00000040:3.0:1787827027.189266:0:37:0:(genops.c:804:class_conn2export()) looking for export cookie 0xb3f83dad1b7dcdd6 00000100:00000001:2.0:1787827027.189268:0:2385:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.203.114@tcp: PM QoS new deadline estimation for cpu 2 is 4297855228 00000020:00000001:3.0:1787827027.189271:0:37:0:(lustre_handles.c:130:class_handle2object()) Process entered 00000100:00000001:2.0:1787827027.189275:0:2385:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.203.114@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4297850108 00000020:00000040:3.0:1787827027.189276:0:37:0:(lustre_handles.c:148:class_handle2object()) GET export ffff985602cdf000 refcount=4 00000100:00000001:2.0:1787827027.189281:0:2385:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.203.114@tcp: PM QoS new deadline estimation for cpu 3 is 4297855228 00000020:00000001:3.0:1787827027.189281:0:37:0:(lustre_handles.c:155:class_handle2object()) Process leaving (rc=18446630093914501120 : -113979795050496 : ffff985602cdf000) 00000100:00000001:2.0:1787827027.189287:0:2385:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.203.114@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4297850108 00000020:00000001:3.0:1787827027.189287:0:37:0:(genops.c:806:class_conn2export()) Process leaving (rc=18446630093914501120 : -113979795050496 : ffff985602cdf000) 00000100:00000001:2.0:1787827027.189291:0:2385:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1787827027.189295:0:2385:0:(client.c:1941:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1787827027.189295:0:37:0:(lustre_net.h:1965:ptlrpc_connection_put()) PUT conn=ffff98561201f640 refcount 14 to 192.168.203.114@tcp 00000100:00000001:2.0:1787827027.189300:0:2385:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827027.189300:0:37:0:(connection.c:156:ptlrpc_connection_addref()) Process entered 00000100:00000001:2.0:1787827027.189302:0:2385:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1787827027.189306:0:37:0:(connection.c:161:ptlrpc_connection_addref()) conn=ffff98561201f640 refcount 15 to 192.168.203.114@tcp 00000100:00000001:2.0:1787827027.189310:0:2385:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1787827027.189310:0:37:0:(connection.c:163:ptlrpc_connection_addref()) Process leaving (rc=18446630094169568832 : -113979539982784 : ffff98561201f640) 00000100:00000001:2.0:1787827027.189314:0:2385:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000020:00000040:3.0:1787827027.189315:0:37:0:(genops.c:879:class_export_put()) PUTting export ffff985602cdf000 : new refcount 3 00000100:00000001:2.0:1787827027.189316:0:2385:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1787827027.189320:0:2385:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1787827027.189322:0:2385:0:(client.c:1813:ptlrpc_send_new_req()) Process entered 00000100:00080000:3.0:1787827027.189323:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff985602bc9800: import ffff9856110b0000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00000040:2.0:1787827027.189327:0:2385:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff985610f821c0 x1874669228031488/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827027.189329:0:37:0:(import.c:628:import_select_connection()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1787827027.189335:0:37:0:(sec.c:1579:sptlrpc_import_sec_adapt()) Process entered 00000100:00000001:2.0:1787827027.189336:0:2385:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1787827027.189338:0:2385:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1787827027.189340:0:2385:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1787827027.189342:0:37:0:(sec.c:1619:sptlrpc_import_sec_adapt()) Process leaving via out (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1787827027.189343:0:2385:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1787827027.189345:0:2385:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1787827027.189347:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f821c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669228031488:192.168.203.114@tcp:400:kworker.0 00000100:00000001:2.0:1787827027.189349:0:2385:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1787827027.189351:0:2385:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1787827027.189353:0:2385:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1787827027.189354:0:37:0:(sec.c:1647:sptlrpc_import_sec_adapt()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1787827027.189355:0:2385:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:2.0:1787827027.189357:0:2385:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff985611861800. 00000100:00000001:3.0:1787827027.189359:0:37:0:(obd_class.h:889:obd_reconnect()) Process entered 02000000:00000001:2.0:1787827027.189360:0:2385:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1787827027.189363:0:2385:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000010:2.0:1787827027.189365:0:2385:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff98561080ee70. 00000008:00100000:3.0:1787827027.189365:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683216 ocd_grant: 8388608, lost: 0. 00000400:00000010:2.0:1787827027.189370:0:2385:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985610afa110. 00000008:00000001:3.0:1787827027.189371:0:37:0:(osc_request.c:4022:osc_reconnect()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:2.0:1787827027.189373:0:2385:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1874669228031488, portal 10 00000100:00000001:3.0:1787827027.189375:0:37:0:(obd_class.h:903:obd_reconnect()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1787827027.189376:0:2385:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1787827027.189377:0:2385:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446630094152147392 : -113979557404224 : ffff985610f821c0) 00000100:00000010:3.0:1787827027.189382:0:37:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 904 at ffff985610f81680. 00000100:00000040:2.0:1787827027.189384:0:2385:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff985610f821c0 x1874669228031488/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827043 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000020:00000040:3.0:1787827027.189388:0:37:0:(genops.c:1210:class_import_get()) import ffff9856110b0000 refcount=3 obd=lustre-OST0000-osc-ffff985602bc9800 00000100:00000001:2.0:1787827027.189392:0:2385:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:2.0:1787827027.189395:0:2385:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.203.114@tcp 02000000:00000001:3.0:1787827027.189395:0:37:0:(sec.c:2038:sptlrpc_sepol_get()) Process entered 00000400:00000010:2.0:1787827027.189399:0:2385:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985610afa330. 02000000:00000001:3.0:1787827027.189400:0:37:0:(sec.c:2048:sptlrpc_sepol_get()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:2.0:1787827027.189401:0:2385:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1874669228031488, offset 0 00000100:00000001:3.0:1787827027.189405:0:37:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 00000400:00000200:2.0:1787827027.189406:0:2385:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.203.114@tcp 00000400:00000200:2.0:1787827027.189409:0:2385:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827027.189411:0:2385:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.203.114@tcp local destination 00000400:00000200:2.0:1787827027.189413:0:2385:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:2.0:1787827027.189414:0:2385:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.203.14@tcp 00000400:00000200:2.0:1787827027.189418:0:2385:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.203.14@tcp lpni_is_preferred = 0 02000000:00000001:3.0:1787827027.189418:0:37:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 00000400:00000200:2.0:1787827027.189422:0:2385:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.203.114@tcp 02000000:00000001:3.0:1787827027.189423:0:37:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1787827027.189424:0:2385:0:(lib-move.c:1745:lnet_handle_send()) 192.168.203.14@tcp NI seq info: [21881:21881:255:4294967295] 192.168.203.114@tcp LPNI seq info [21881:21881:7:4294967295] 00000400:00000200:2.0:1787827027.189428:0:2385:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 02000000:00000010:3.0:1787827027.189430:0:37:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 1024 at ffff985608492400. 00000400:00000200:2.0:1787827027.189437:0:2385:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp:) -> 192.168.203.114@tcp(192.168.203.114@tcp:192.168.203.114@tcp) : PUT try# 0 00000100:00000001:3.0:1787827027.189437:0:37:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1787827027.189446:0:2385:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787827027.189448:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81680 x1874669228031616/t0(0) o8->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000010:2.0:1787827027.189451:0:2385:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff985615779800. 00000800:00000200:2.0:1787827027.189459:0:2385:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff985606751480] -> 12345-192.168.203.114@tcp (5) 00000100:00000001:3.0:1787827027.189465:0:37:0:(jobid.c:939:lustre_get_jobid()) Process entered 00000800:00000200:2.0:1787827027.189468:0:2385:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.203.114@tcp ip 192.168.203.114:988 00000100:00000001:3.0:1787827027.189470:0:37:0:(jobid.c:979:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1787827027.189473:0:2385:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff985615779800 type 1, nob 320 niov 1 nkiov 1 00000100:00000040:3.0:1787827027.189478:0:37:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000e6e8a1e6] to pc [ptlrpcd_rcv+-1] req@ffff985610f81680 x1874669228031616/t0(0) o8->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827027.189479:0:2385:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1787827027.189484:0:2385:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff985602bc9800: using a timeout of 11975 usecs (12 jiffies) 00000400:00000200:2.0:1787827027.189492:0:2385:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000001:2.0:1787827027.189496:0:2385:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff985602bc9800: PM QoS new deadline estimation for cpu 2 is 4297845260 00000100:00000001:2.0:1787827027.189498:0:2385:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff985602bc9800: PM QoS request active for cpu 2, keep current deadline 4297855228 00000100:00000001:2.0:1787827027.189500:0:2385:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff985602bc9800: PM QoS new deadline estimation for cpu 3 is 4297845260 00000100:00000001:2.0:1787827027.189502:0:2385:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff985602bc9800: PM QoS request active for cpu 3, keep current deadline 4297855228 00000100:00000001:2.0:1787827027.189503:0:2385:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1787827027.189505:0:2385:0:(client.c:1941:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1787827027.189507:0:2385:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1787827027.189509:0:2385:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1787827027.189509:0:37:0:(import.c:833:ptlrpc_connect_import_locked()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1787827027.189515:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-OST0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1787827027.189519:0:37:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000001:2.0:1787827027.189520:0:2385:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1787827027.189524:0:2385:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000010:3.0:1787827027.189524:0:37:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 904 at ffff985610f82580. 00000100:00000001:2.0:1787827027.189530:0:2385:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000020:00000040:3.0:1787827027.189530:0:37:0:(genops.c:1210:class_import_get()) import ffff9856110b1000 refcount=3 obd=lustre-OST0001-osc-ffff985602bc9800 00000100:00000001:2.0:1787827027.189532:0:2385:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1787827027.189536:0:2385:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827027.189536:0:37:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1787827027.189540:0:37:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 00000100:00100000:2.0:1787827027.189541:0:2385:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 02000000:00000001:3.0:1787827027.189543:0:37:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1787827027.189545:0:2385:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787827027.189548:0:2382:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000010:3.0:1787827027.189549:0:37:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff985609025b00. 00000100:00000001:2.0:1787827027.189550:0:2385:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1787827027.189553:0:2382:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1787827027.189554:0:2385:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1787827027.189555:0:37:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827027.189557:0:2382:0:(client.c:1813:ptlrpc_send_new_req()) Process entered 00000100:00000040:3.0:1787827027.189564:0:37:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-OST0001_UUID req@ffff985610f82580 x1874669228031744/t0(0) o400->lustre-OST0001-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1787827027.189566:0:2385:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:1.0:1787827027.189567:0:2382:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff985610f81680 x1874669228031616/t0(0) o8->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827027.189571:0:2385:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1787827027.189574:0:2385:0:(client.c:1813:ptlrpc_send_new_req()) Process entered 00000100:00000001:3.0:1787827027.189579:0:37:0:(jobid.c:939:lustre_get_jobid()) Process entered 00000100:00000040:2.0:1787827027.189582:0:2385:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff985610f82940 x1874669228031360/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827027.189583:0:37:0:(jobid.c:979:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1787827027.189594:0:37:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000db1fd97d] to pc [ptlrpcd_01_01+1] req@ffff985610f82580 x1874669228031744/t0(0) o400->lustre-OST0001-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827027.189601:0:2382:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1787827027.189605:0:2385:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1787827027.189606:0:2382:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1787827027.189609:0:2385:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1787827027.189612:0:2382:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1787827027.189615:0:2385:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1787827027.189616:0:2382:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1787827027.189620:0:2385:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1787827027.189620:0:37:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1787827027.189621:0:2382:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1787827027.189627:0:2385:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1787827027.189628:0:37:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (3182) 00000100:00100000:1.0:1787827027.189630:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228031616:192.168.203.114@tcp:8:kworker.0 00000100:00100000:2.0:1787827027.189635:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669228031360:192.168.203.114@tcp:400:kworker.0 00000100:00000001:1.0:1787827027.189643:0:2382:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1787827027.189646:0:2382:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1787827027.189647:0:2382:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1787827027.189651:0:2382:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:2.0:1787827027.189654:0:2385:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000010:1.0:1787827027.189657:0:2382:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 1024 at ffff985607d00800. 02000000:00000001:2.0:1787827027.189659:0:2385:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1787827027.189662:0:2382:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1787827027.189663:0:2385:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1787827027.189667:0:2385:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000200:1.0:1787827027.189671:0:2382:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 02000000:00000010:2.0:1787827027.189672:0:2385:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff985611860e00. 00000400:00000010:1.0:1787827027.189676:0:2382:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff985602ecf898. 02000000:00000001:2.0:1787827027.189676:0:2385:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1787827027.189683:0:2385:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000010:1.0:1787827027.189684:0:2382:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985606618aa0. 00000400:00000010:2.0:1787827027.189687:0:2385:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff98561080e790. 00000400:00000010:2.0:1787827027.189713:0:2385:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985610afad48. 00000100:00000200:2.0:1787827027.189717:0:2385:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1874669228031360, portal 10 00000100:00000001:2.0:1787827027.189719:0:2385:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1787827027.189720:0:2385:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446630094152149312 : -113979557402304 : ffff985610f82940) 00000100:00000200:1.0:1787827027.189722:0:2382:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 1024 bytes, xid 1874669228031616, portal 4 00000100:00000001:1.0:1787827027.189727:0:2382:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000040:2.0:1787827027.189728:0:2385:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff985610f82940 x1874669228031360/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827043 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827027.189730:0:2382:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446630094152144512 : -113979557407104 : ffff985610f81680) 00000100:00000001:2.0:1787827027.189739:0:2385:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1787827027.189741:0:2382:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff985610f81680 x1874669228031616/t0(0) o8->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 520/544 e 0 to 0 dl 1787827042 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1787827027.189746:0:2385:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.203.114@tcp 00000400:00000010:2.0:1787827027.189750:0:2385:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985610afaee0. 00000100:00000200:2.0:1787827027.189754:0:2385:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1874669228031360, offset 0 00000100:00000001:3.0:1787827027.189754:0:2386:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1787827027.189756:0:2382:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:3.0:1787827027.189760:0:2386:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000040:1.0:1787827027.189761:0:2382:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.203.114@tcp 00000100:00000001:3.0:1787827027.189764:0:2386:0:(client.c:1813:ptlrpc_send_new_req()) Process entered 00000400:00000200:2.0:1787827027.189765:0:2385:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.203.114@tcp 00000400:00000010:1.0:1787827027.189768:0:2382:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9856066186e8. 00000400:00000200:2.0:1787827027.189773:0:2385:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000040:3.0:1787827027.189774:0:2386:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff985610f82580 x1874669228031744/t0(0) o400->lustre-OST0001-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1787827027.189777:0:2385:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.203.114@tcp local destination 00000100:00000200:1.0:1787827027.189778:0:2382:0:(niobuf.c:94:ptl_send_buf()) Sending 520 bytes to portal 28, xid 1874669228031616, offset 0 00000400:00000200:2.0:1787827027.189785:0:2385:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:3.0:1787827027.189791:0:2386:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000400:00000200:1.0:1787827027.189792:0:2382:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.203.114@tcp 00000400:00000200:2.0:1787827027.189792:0:2385:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.203.14@tcp 00000100:00000001:3.0:1787827027.189794:0:2386:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1787827027.189808:0:2385:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.203.14@tcp lpni_is_preferred = 0 02000000:00000001:3.0:1787827027.189812:0:2386:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 00000400:00000200:1.0:1787827027.189813:0:2382:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827027.189815:0:2385:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.203.114@tcp 02000000:00000001:3.0:1787827027.189816:0:2386:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1787827027.189820:0:2386:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1787827027.189823:0:2382:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.203.114@tcp local destination 00000400:00000200:2.0:1787827027.189823:0:2385:0:(lib-move.c:1745:lnet_handle_send()) 192.168.203.14@tcp NI seq info: [21882:21882:254:4294967295] 192.168.203.114@tcp LPNI seq info [21882:21882:6:4294967295] 00000100:00100000:3.0:1787827027.189827:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228031744:192.168.203.114@tcp:400:kworker.0 00000400:00000200:1.0:1787827027.189832:0:2382:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:3.0:1787827027.189835:0:2386:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000200:2.0:1787827027.189836:0:2385:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:1.0:1787827027.189839:0:2382:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.203.14@tcp 02000000:00000001:3.0:1787827027.189840:0:2386:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1787827027.189844:0:2386:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1787827027.189846:0:2382:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.203.14@tcp lpni_is_preferred = 0 02000000:00000001:3.0:1787827027.189849:0:2386:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000200:1.0:1787827027.189854:0:2382:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.203.114@tcp 00000400:00000200:2.0:1787827027.189855:0:2385:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp:) -> 192.168.203.114@tcp(192.168.203.114@tcp:192.168.203.114@tcp) : PUT try# 0 02000000:00000010:3.0:1787827027.189856:0:2386:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff98561288dc00. 02000000:00000001:3.0:1787827027.189861:0:2386:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1787827027.189865:0:2382:0:(lib-move.c:1745:lnet_handle_send()) 192.168.203.14@tcp NI seq info: [21883:21883:253:4294967295] 192.168.203.114@tcp LPNI seq info [21883:21883:5:4294967295] 00000400:00000200:3.0:1787827027.189870:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000800:00000200:2.0:1787827027.189873:0:2385:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.203.114@tcp 00000400:00000010:3.0:1787827027.189876:0:2386:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff985602ad9c08. 00000400:00000200:1.0:1787827027.189878:0:2382:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000800:00000010:2.0:1787827027.189880:0:2385:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff985615779600. 00000400:00000010:3.0:1787827027.189883:0:2386:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985609573ee0. 00000800:00000200:2.0:1787827027.189888:0:2385:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff985606751480] -> 12345-192.168.203.114@tcp (5) 00000100:00000200:3.0:1787827027.189890:0:2386:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1874669228031744, portal 4 00000100:00000001:3.0:1787827027.189894:0:2386:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000800:00000200:2.0:1787827027.189895:0:2385:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.203.114@tcp ip 192.168.203.114:988 00000100:00000001:3.0:1787827027.189896:0:2386:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446630094152148352 : -113979557403264 : ffff985610f82580) 00000800:00000200:2.0:1787827027.189901:0:2385:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff985615779600 type 1, nob 320 niov 1 nkiov 1 00000100:00000040:3.0:1787827027.189906:0:2386:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff985610f82580 x1874669228031744/t0(0) o400->lustre-OST0001-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827043 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827027.189908:0:2385:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1787827027.189913:0:2385:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff985602bc9800: using a timeout of 38053 usecs (39 jiffies) 00000400:00000200:2.0:1787827027.189922:0:2385:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000001:3.0:1787827027.189922:0:2386:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:2.0:1787827027.189925:0:2385:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff985602bc9800: PM QoS new deadline estimation for cpu 2 is 4297845288 00000100:00000040:3.0:1787827027.189929:0:2386:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.203.114@tcp 00000100:00000001:2.0:1787827027.189930:0:2385:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff985602bc9800: PM QoS request active for cpu 2, keep current deadline 4297855228 00000400:00000010:3.0:1787827027.189933:0:2386:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985609573bb0. 00000100:00000001:2.0:1787827027.189935:0:2385:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff985602bc9800: PM QoS new deadline estimation for cpu 3 is 4297845288 00000100:00000200:3.0:1787827027.189938:0:2386:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 28, xid 1874669228031744, offset 0 00000100:00000001:2.0:1787827027.189940:0:2385:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff985602bc9800: PM QoS request active for cpu 3, keep current deadline 4297855228 00000100:00000001:2.0:1787827027.189944:0:2385:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000200:3.0:1787827027.189948:0:2386:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.203.114@tcp 00000100:00000001:2.0:1787827027.189949:0:2385:0:(client.c:1941:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1787827027.189954:0:2385:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1787827027.189956:0:2385:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1787827027.189957:0:2385:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1787827027.189958:0:2385:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1787827027.189999:0:2385:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1787827027.190000:0:2385:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1787827027.190634:0:2382:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp:) -> 192.168.203.114@tcp(192.168.203.114@tcp:192.168.203.114@tcp) : PUT try# 0 00000800:00000200:1.0:1787827027.190666:0:2382:0:(socklnd_cb.c:931:ksocknal_send()) sending 520 bytes in 1 frags to 12345-192.168.203.114@tcp 00000800:00000010:1.0:1787827027.190672:0:2382:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff985607d95f00. 00000400:00000200:3.0:1787827027.190674:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000800:00000200:1.0:1787827027.190680:0:2382:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff985606751480] -> 12345-192.168.203.114@tcp (5) 00000400:00000200:3.0:1787827027.190684:0:2386:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.203.114@tcp local destination 00000400:00000200:3.0:1787827027.190715:0:2386:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000800:00000200:1.0:1787827027.190718:0:2382:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.203.114@tcp ip 192.168.203.114:988 00000800:00000200:1.0:1787827027.190723:0:2382:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff985607d95f00 type 1, nob 616 niov 1 nkiov 1 00000400:00000200:3.0:1787827027.190724:0:2386:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.203.14@tcp 00000100:00000001:1.0:1787827027.190729:0:2382:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1787827027.190730:0:2386:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.203.14@tcp lpni_is_preferred = 0 00000100:00000040:1.0:1787827027.190736:0:2382:0:(niobuf.c:635:kick_cpu_latency()) lustre-OST0000-osc-ffff985602bc9800: using a timeout of 14617 usecs (15 jiffies) 00000400:00000200:3.0:1787827027.190736:0:2386:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.203.114@tcp 00000400:00000200:3.0:1787827027.190740:0:2386:0:(lib-move.c:1745:lnet_handle_send()) 192.168.203.14@tcp NI seq info: [21884:21884:252:4294967295] 192.168.203.114@tcp LPNI seq info [21884:21884:4:4294967295] 00000400:00000200:3.0:1787827027.190748:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:1.0:1787827027.190756:0:2382:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:3.0:1787827027.190758:0:2386:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp:) -> 192.168.203.114@tcp(192.168.203.114@tcp:192.168.203.114@tcp) : PUT try# 0 00000100:00000001:1.0:1787827027.190761:0:2382:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff985602bc9800: PM QoS new deadline estimation for cpu 2 is 4297845265 00000800:00000200:3.0:1787827027.190766:0:2386:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.203.114@tcp 00000100:00000001:1.0:1787827027.190767:0:2382:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff985602bc9800: PM QoS request active for cpu 2, keep current deadline 4297855228 00000800:00000010:3.0:1787827027.190770:0:2386:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff985609025400. 00000100:00000001:1.0:1787827027.190771:0:2382:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff985602bc9800: PM QoS new deadline estimation for cpu 3 is 4297845265 00000100:00000001:1.0:1787827027.190776:0:2382:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff985602bc9800: PM QoS request active for cpu 3, keep current deadline 4297855228 00000800:00000200:3.0:1787827027.190777:0:2386:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff985606751480] -> 12345-192.168.203.114@tcp (5) 00000100:00000001:1.0:1787827027.190782:0:2382:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000800:00000200:3.0:1787827027.190785:0:2386:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.203.114@tcp ip 192.168.203.114:988 00000100:00000001:1.0:1787827027.190788:0:2382:0:(client.c:1941:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1787827027.190789:0:2386:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff985609025400 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1787827027.190793:0:2382:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827027.190793:0:2386:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827027.190810:0:2382:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1787827027.190811:0:2386:0:(niobuf.c:635:kick_cpu_latency()) lustre-OST0001-osc-ffff985602bc9800: using a timeout of 13851 usecs (14 jiffies) 00000400:00000200:3.0:1787827027.190815:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000001:1.0:1787827027.190817:0:2382:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1787827027.190818:0:2386:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0001-osc-ffff985602bc9800: PM QoS new deadline estimation for cpu 2 is 4297845264 00000100:00000001:1.0:1787827027.190821:0:2382:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=5 : 5 : 5) 00000100:00000001:3.0:1787827027.190822:0:2386:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0001-osc-ffff985602bc9800: PM QoS request active for cpu 2, keep current deadline 4297855228 00000100:00000001:3.0:1787827027.190826:0:2386:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0001-osc-ffff985602bc9800: PM QoS new deadline estimation for cpu 3 is 4297845264 00000100:00000001:1.0:1787827027.190828:0:2382:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1787827027.190828:0:2386:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0001-osc-ffff985602bc9800: PM QoS request active for cpu 3, keep current deadline 4297855228 00000100:00000001:1.0:1787827027.190831:0:2382:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1787827027.190832:0:2386:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1787827027.190836:0:2382:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827027.190837:0:2386:0:(client.c:1941:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827027.190843:0:2382:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827027.190843:0:2386:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827027.190848:0:2386:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1787827027.190852:0:2386:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1787827027.190853:0:2386:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1787827027.190855:0:2386:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1787827027.190856:0:2386:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1787827027.190857:0:2386:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827027.190859:0:2386:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1787827027.190896:0:2378:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1787827027.190899:0:2378:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff985615779b00. 00000400:00000200:3.0:1787827027.190905:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1787827027.190910:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000400:00000200:3.0:1787827027.190915:0:2378:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985610afa220 00000400:00000010:3.0:1787827027.190917:0:2378:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985610afa220. 00000100:00000001:3.0:1787827027.190921:0:2378:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1787827027.190927:0:2378:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff985610f83c00 x1874669228031232/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827043 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827027.190937:0:2378:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1787827027.190940:0:2378:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff985610f83c00 x1874669228031232/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827043 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827027.190947:0:2378:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827027.190949:0:2378:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1787827027.190960:0:2378:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1787827027.190962:0:2378:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff985615779800. 00000400:00000200:3.0:1787827027.190963:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1787827027.190965:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000400:00000200:3.0:1787827027.190967:0:2378:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985610afa330 00000400:00000010:3.0:1787827027.190968:0:2378:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985610afa330. 00000100:00000001:3.0:1787827027.190970:0:2378:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1787827027.190972:0:2378:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff985610f821c0 x1874669228031488/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827043 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827027.190977:0:2378:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1787827027.190988:0:2378:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff985610f821c0 x1874669228031488/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827043 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827027.190995:0:2378:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827027.190996:0:2378:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1787827027.191004:0:2378:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1787827027.191005:0:2378:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff985615779600. 00000400:00000200:3.0:1787827027.191007:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1787827027.191011:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000400:00000200:3.0:1787827027.191013:0:2378:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985610afaee0 00000400:00000010:3.0:1787827027.191015:0:2378:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985610afaee0. 00000100:00000001:3.0:1787827027.191017:0:2378:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1787827027.191020:0:2378:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff985610f82940 x1874669228031360/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827043 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827027.191033:0:2378:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1787827027.191036:0:2378:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff985610f82940 x1874669228031360/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827043 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827027.191041:0:2378:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827027.191042:0:2378:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1787827027.191048:0:2378:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1787827027.191050:0:2378:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff985607d95f00. 00000400:00000200:3.0:1787827027.191052:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1787827027.191054:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000400:00000200:3.0:1787827027.191055:0:2378:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9856066186e8 00000400:00000010:3.0:1787827027.191057:0:2378:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9856066186e8. 00000100:00000001:3.0:1787827027.191058:0:2378:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1787827027.191060:0:2378:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff985610f81680 x1874669228031616/t0(0) o8->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 520/544 e 0 to 0 dl 1787827042 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827027.191064:0:2378:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1787827027.191066:0:2378:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff985610f81680 x1874669228031616/t0(0) o8->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 520/544 e 0 to 0 dl 1787827042 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827027.191069:0:2378:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827027.191070:0:2378:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1787827027.191113:0:2378:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1787827027.191115:0:2378:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff985609025400. 00000400:00000200:3.0:1787827027.191117:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1787827027.191119:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000400:00000200:3.0:1787827027.191120:0:2378:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985609573bb0 00000400:00000010:3.0:1787827027.191121:0:2378:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985609573bb0. 00000100:00000001:3.0:1787827027.191123:0:2378:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1787827027.191125:0:2378:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff985610f82580 x1874669228031744/t0(0) o400->lustre-OST0001-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827043 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827027.191129:0:2378:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1787827027.191131:0:2378:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff985610f82580 x1874669228031744/t0(0) o400->lustre-OST0001-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827043 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827027.191138:0:2378:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827027.191139:0:2378:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1787827027.192626:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827027.192638:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1787827027.192641:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827027.192644:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827027.192651:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827027.192656:0:2377:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp) <- 192.168.203.114@tcp : PUT - for me 00000400:00000200:2.0:1787827027.192663:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827027.192667:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827027.192672:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827027.192676:0:2377:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.203.114@tcp of length 224 into portal 25 MB=0x6a90071687d00 00000400:00000200:2.0:1787827027.192680:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827027.192685:0:2377:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.203.114@tcp of length 224/224 into md 0x3228d [1] + 224 00000800:00000001:2.0:1787827027.192711:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827027.192716:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827027.192719:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1787827027.192722:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000100:00000001:2.0:1787827027.192728:0:2377:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1787827027.192734:0:2377:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff985610f83c00 x1874669228031232/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827043 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1787827027.192744:0:2377:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff985610f83c00 x1874669228031232/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827043 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827027.192760:0:2377:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1787827027.192767:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827027.192771:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1787827027.192773:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827027.192775:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827027.192778:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827027.192781:0:2377:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp) <- 192.168.203.114@tcp : PUT - for me 00000400:00000200:2.0:1787827027.192785:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827027.192788:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827027.192790:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827027.192793:0:2377:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.203.114@tcp of length 224 into portal 10 MB=0x6a90071687e00 00000400:00000200:2.0:1787827027.192808:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827027.192813:0:2377:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.203.114@tcp of length 224/224 into md 0x3229d [1] + 224 00000800:00000001:2.0:1787827027.192817:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827027.192820:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827027.192822:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1787827027.192824:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000100:00000001:2.0:1787827027.192826:0:2377:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1787827027.192829:0:2377:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff985610f821c0 x1874669228031488/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827043 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1787827027.192835:0:2377:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff985610f821c0 x1874669228031488/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827043 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827027.192839:0:2377:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1787827027.192842:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827027.192844:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1787827027.192845:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827027.192847:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827027.192849:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827027.192852:0:2377:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp) <- 192.168.203.114@tcp : PUT - for me 00000400:00000200:2.0:1787827027.192855:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827027.192856:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827027.192858:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827027.192860:0:2377:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.203.114@tcp of length 224 into portal 10 MB=0x6a90071687d80 00000400:00000200:2.0:1787827027.192863:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827027.192865:0:2377:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.203.114@tcp of length 224/224 into md 0x322b5 [1] + 224 00000100:00000001:3.0:1787827027.192867:0:2385:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000001:2.0:1787827027.192868:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:3.0:1787827027.192872:0:2385:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000800:00000001:2.0:1787827027.192873:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827027.192876:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1787827027.192879:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000400:00000001:3.0:1787827027.192881:0:2385:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1787827027.192884:0:2385:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1787827027.192887:0:2385:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff98561080ee70. 00000400:00000200:3.0:1787827027.192892:0:2385:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985610afa110 00000400:00000010:3.0:1787827027.192894:0:2385:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985610afa110. 00000100:00000001:2.0:1787827027.192897:0:2377:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:3.0:1787827027.192897:0:2385:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1787827027.192903:0:2377:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff985610f82940 x1874669228031360/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827043 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1787827027.192905:0:2385:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff985610f821c0 x1874669228031488/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827043 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1787827027.192924:0:2377:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff985610f82940 x1874669228031360/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827043 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1787827027.192930:0:2385:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff985610f821c0 x1874669228031488/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827043 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827027.192931:0:2377:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1787827027.192935:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827027.192938:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1787827027.192942:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827027.192945:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1787827027.192945:0:2385:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1787827027.192950:0:2385:0:(client.c:3001:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1787827027.192954:0:2385:0:(client.c:1593:after_reply()) Process entered 00000400:00000200:2.0:1787827027.192957:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 02000000:00000001:3.0:1787827027.192958:0:2385:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1787827027.192962:0:2385:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000400:00000200:2.0:1787827027.192966:0:2377:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp) <- 192.168.203.114@tcp : PUT - for me 00000100:00000001:3.0:1787827027.192966:0:2385:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1787827027.192971:0:2385:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1787827027.192975:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827027.192979:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00001000:3.0:1787827027.192981:0:2385:0:(import.c:2149:obd_at_measure()) add 5 to ffff985611185c60 time=27 v=5 (5 5 5 5) 00000400:00000200:2.0:1787827027.192986:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00001000:3.0:1787827027.192989:0:2385:0:(import.c:2149:obd_at_measure()) add 1 to ffff985611185c18 time=27 v=5 (1 1 1 1) 00000400:00000200:2.0:1787827027.192993:0:2377:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.203.114@tcp of length 224 into portal 4 MB=0x6a90071687f00 00000100:00000001:3.0:1787827027.192995:0:2385:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1787827027.192999:0:2385:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1787827027.193000:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00010000:00000001:3.0:1787827027.193004:0:2385:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000400:00000200:2.0:1787827027.193008:0:2377:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 4 from 12345-192.168.203.114@tcp of length 224/224 into md 0x322c5 [1] + 224 00010000:00000001:3.0:1787827027.193011:0:2385:0:(ldlm_request.c:1703:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1787827027.193013:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:3.0:1787827027.193016:0:2385:0:(client.c:3109:ptlrpc_free_committed()) Process entered 00000800:00000001:2.0:1787827027.193019:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1787827027.193020:0:2385:0:(client.c:3121:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff985602bc9800: skip recheck: last_committed 0 00000400:00000200:2.0:1787827027.193024:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:3.0:1787827027.193025:0:2385:0:(client.c:3122:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1787827027.193029:0:2385:0:(client.c:1797:after_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1787827027.193032:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000100:00000001:2.0:1787827027.193038:0:2377:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000040:3.0:1787827027.193039:0:2385:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff985610f821c0 x1874669228031488/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827043 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1787827027.193045:0:2377:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff985610f82580 x1874669228031744/t0(0) o400->lustre-OST0001-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827043 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827027.193075:0:2385:0:(client.c:2354:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000040:2.0:1787827027.193078:0:2377:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff985610f82580 x1874669228031744/t0(0) o400->lustre-OST0001-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827043 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827027.193079:0:2385:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1787827027.193082:0:2385:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1787827027.193089:0:2385:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff985610f821c0 x1874669228031488/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827043 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827027.193096:0:2377:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00100000:3.0:1787827027.193096:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f821c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669228031488:192.168.203.114@tcp:400:kworker.0 00000800:00000001:2.0:1787827027.193103:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000400:00000001:3.0:1787827027.193104:0:2385:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1787827027.193107:0:2385:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1787827027.193108:0:2385:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff98561080ef78. 00000800:00000001:2.0:1787827027.193109:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1787827027.193111:0:2385:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985610afa880 00000800:00000001:2.0:1787827027.193113:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000400:00000010:3.0:1787827027.193114:0:2385:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985610afa880. 00000100:00000001:3.0:1787827027.193117:0:2385:0:(events.c:78:reply_in_callback()) Process entered 00000800:00000001:2.0:1787827027.193119:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:3.0:1787827027.193120:0:2385:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff985610f83c00 x1874669228031232/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827043 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1787827027.193127:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000200:3.0:1787827027.193132:0:2385:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff985610f83c00 x1874669228031232/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827043 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1787827027.193136:0:2377:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp) <- 192.168.203.114@tcp : PUT - for me 00000100:00000001:3.0:1787827027.193148:0:2385:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1787827027.193150:0:2385:0:(client.c:3001:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1787827027.193158:0:2385:0:(client.c:1593:after_reply()) Process entered 00000400:00000200:2.0:1787827027.193160:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 02000000:00000001:3.0:1787827027.193161:0:2385:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000400:00000200:2.0:1787827027.193165:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000001:3.0:1787827027.193165:0:2385:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000400:00000200:2.0:1787827027.193169:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000001:3.0:1787827027.193169:0:2385:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1787827027.193174:0:2377:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.203.114@tcp of length 224 into portal 4 MB=0x6a90071687e80 02000000:00000001:3.0:1787827027.193174:0:2385:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1787827027.193181:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00001000:3.0:1787827027.193181:0:2385:0:(import.c:2149:obd_at_measure()) add 5 to ffff98561118e460 time=27 v=5 (5 5 5 5) 00000100:00001000:3.0:1787827027.193187:0:2385:0:(import.c:2149:obd_at_measure()) add 1 to ffff98561118e418 time=42 v=5 (1 0 0 0) 00000400:00000200:2.0:1787827027.193190:0:2377:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 4 from 12345-192.168.203.114@tcp of length 224/224 into md 0x322ad [1] + 0 00000100:00000001:3.0:1787827027.193191:0:2385:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1787827027.193193:0:2385:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1787827027.193196:0:2385:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1787827027.193199:0:2385:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827027.193201:0:2385:0:(client.c:1797:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1787827027.193204:0:2385:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff985610f83c00 x1874669228031232/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827043 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1787827027.193209:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:3.0:1787827027.193209:0:2385:0:(client.c:2354:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1787827027.193211:0:2385:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1787827027.193213:0:2385:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1787827027.193215:0:2385:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff985610f83c00 x1874669228031232/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827043 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1787827027.193220:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00100000:3.0:1787827027.193221:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2385:1874669228031232:192.168.203.114@tcp:400:kworker.0 00000400:00000200:2.0:1787827027.193226:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:3.0:1787827027.193227:0:2385:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1787827027.193232:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000100:00000001:3.0:1787827027.193232:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000001:2.0:1787827027.193237:0:2377:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000040:3.0:1787827027.193237:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610f821c0 x1874669228031488/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827043 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827027.193243:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 00000100:00000200:2.0:1787827027.193246:0:2377:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff985610f81680 x1874669228031616/t0(0) o8->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 520/544 e 0 to 0 dl 1787827042 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1787827027.193246:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1787827027.193249:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff985611861800. 02000000:00000001:3.0:1787827027.193253:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1787827027.193256:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:3.0:1787827027.193258:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611185800 refcount=3 obd=lustre-MDT0001-mdc-ffff985602bc9800 00000020:00000001:3.0:1787827027.193261:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 00000100:00000040:2.0:1787827027.193264:0:2377:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=0 replen=544 req@ffff985610f81680 x1874669228031616/t0(0) o8->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 520/544 e 0 to 0 dl 1787827042 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 02000000:00000010:3.0:1787827027.193264:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff985609025200. 02000000:00000001:3.0:1787827027.193269:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1787827027.193272:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1787827027.193275:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610f821c0. 00000100:00000001:3.0:1787827027.193280:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1787827027.193281:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1787827027.193283:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000001:2.0:1787827027.193285:0:2377:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000040:3.0:1787827027.193288:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610f83c00 x1874669228031232/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827043 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1787827027.193292:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827027.193296:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1787827027.193297:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1787827027.193301:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1787827027.193303:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff985611860000. 02000000:00000001:3.0:1787827027.193307:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1787827027.193308:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000100:00000001:2.0:1787827027.193309:0:2386:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000020:00000040:3.0:1787827027.193311:0:2385:0:(genops.c:1223:class_import_put()) import ffff98561118e000 refcount=3 obd=MGC192.168.203.114@tcp 00000100:00000001:2.0:1787827027.193312:0:2386:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1787827027.193314:0:2382:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000020:00000001:3.0:1787827027.193317:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 00000100:00000001:1.0:1787827027.193318:0:2382:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000400:00000001:2.0:1787827027.193318:0:2386:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1787827027.193321:0:2386:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 02000000:00000010:3.0:1787827027.193321:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff985609025a00. 00000400:00000010:2.0:1787827027.193324:0:2386:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff985602ad9c08. 02000000:00000001:3.0:1787827027.193326:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1787827027.193329:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000400:00000200:2.0:1787827027.193330:0:2386:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985609573ee0 00000100:00000010:3.0:1787827027.193331:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610f83c00. 00000400:00000010:2.0:1787827027.193333:0:2386:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985609573ee0. 00000100:00000001:3.0:1787827027.193337:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000400:00000001:1.0:1787827027.193338:0:2382:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000001:2.0:1787827027.193338:0:2386:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:3.0:1787827027.193341:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:1.0:1787827027.193342:0:2382:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1787827027.193345:0:2382:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff985602ecf898. 00000100:00000200:2.0:1787827027.193345:0:2386:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff985610f82580 x1874669228031744/t0(0) o400->lustre-OST0001-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827043 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827027.193347:0:2385:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1787827027.193349:0:2382:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985606618aa0 00000100:00000001:3.0:1787827027.193353:0:2385:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000010:1.0:1787827027.193354:0:2382:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985606618aa0. 00000100:00000001:3.0:1787827027.193357:0:2385:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1787827027.193358:0:2382:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1787827027.193358:0:2386:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff985610f82580 x1874669228031744/t0(0) o400->lustre-OST0001-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827043 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000400:00000001:3.0:1787827027.193364:0:2385:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000200:1.0:1787827027.193365:0:2382:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff985610f81680 x1874669228031616/t0(0) o8->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 520/544 e 0 to 0 dl 1787827042 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827027.193365:0:2386:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1787827027.193369:0:2386:0:(client.c:3001:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:3.0:1787827027.193369:0:2385:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000001:2.0:1787827027.193373:0:2386:0:(client.c:1593:after_reply()) Process entered 00000400:00000010:3.0:1787827027.193374:0:2385:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff98561080e790. 02000000:00000001:2.0:1787827027.193375:0:2386:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000200:1.0:1787827027.193377:0:2382:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff985610f81680 x1874669228031616/t0(0) o8->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 520/544 e 0 to 0 dl 1787827042 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827027.193377:0:2386:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1787827027.193379:0:2386:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1787827027.193380:0:2385:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985610afad48 02000000:00000001:2.0:1787827027.193382:0:2386:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1787827027.193386:0:2385:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985610afad48. 00000100:00000001:1.0:1787827027.193387:0:2382:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00001000:2.0:1787827027.193388:0:2386:0:(import.c:2149:obd_at_measure()) add 5 to ffff9856110b1460 time=105 v=5 (5 5 5 5) 00000100:00000001:1.0:1787827027.193390:0:2382:0:(client.c:3001:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1787827027.193392:0:2385:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:1.0:1787827027.193393:0:2382:0:(client.c:1593:after_reply()) Process entered 00000100:00001000:2.0:1787827027.193393:0:2386:0:(import.c:2149:obd_at_measure()) add 1 to ffff9856110b1418 time=6 v=5 (1 1 1 1) 02000000:00000001:1.0:1787827027.193394:0:2382:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1787827027.193396:0:2382:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1787827027.193397:0:2386:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1787827027.193398:0:2382:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1787827027.193398:0:2386:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:3.0:1787827027.193401:0:2385:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff985610f82940 x1874669228031360/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827043 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1787827027.193402:0:2382:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1787827027.193402:0:2386:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:2.0:1787827027.193405:0:2386:0:(ldlm_request.c:1703:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1787827027.193407:0:2382:0:(import.c:2149:obd_at_measure()) add 0 to ffff9856110b0460 time=105 v=5 (5 5 5 5) 00000100:00000001:2.0:1787827027.193409:0:2386:0:(client.c:3109:ptlrpc_free_committed()) Process entered 00000100:00000040:2.0:1787827027.193411:0:2386:0:(client.c:3121:ptlrpc_free_committed()) lustre-OST0001-osc-ffff985602bc9800: skip recheck: last_committed 4294967636 00000100:00001000:1.0:1787827027.193414:0:2382:0:(import.c:2149:obd_at_measure()) add 1 to ffff9856110b0418 time=6 v=10 (10 1 1 1) 00000100:00000001:2.0:1787827027.193414:0:2386:0:(client.c:3122:ptlrpc_free_committed()) Process leaving 00000100:00000001:2.0:1787827027.193417:0:2386:0:(client.c:1797:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827027.193419:0:2382:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1787827027.193423:0:2382:0:(client.c:1506:ptlrpc_check_status()) Process leaving (rc=18446744073709551597 : -19 : ffffffffffffffed) 00000100:00000040:2.0:1787827027.193425:0:2386:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff985610f82580 x1874669228031744/t0(0) o400->lustre-OST0001-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827043 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827027.193426:0:2382:0:(client.c:1711:after_reply()) Process leaving (rc=18446744073709551597 : -19 : ffffffffffffffed) 00000100:00000040:1.0:1787827027.193430:0:2382:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff985610f81680 x1874669228031616/t0(0) o8->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 520/224 e 0 to 0 dl 1787827042 ref 1 fl Rpc:RNQU/200/0 rc -19/-19 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827027.193432:0:2386:0:(client.c:2354:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1787827027.193436:0:2386:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1787827027.193437:0:2386:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787827027.193439:0:2382:0:(client.c:2354:ptlrpc_check_set()) Process leaving via interpret (rc=18446744073709551597 : -19 : 0xffffffffffffffed) 00000100:00000001:1.0:1787827027.193441:0:2382:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1787827027.193442:0:2382:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827027.193442:0:2386:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff985610f82580 x1874669228031744/t0(0) o400->lustre-OST0001-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827043 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1787827027.193444:0:2385:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff985610f82940 x1874669228031360/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827043 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827027.193445:0:2382:0:(import.c:1057:ptlrpc_connect_interpret()) Process entered 00000100:00000001:1.0:1787827027.193448:0:2382:0:(import.c:1092:ptlrpc_connect_interpret()) Process leaving via out (rc=18446744073709551597 : -19 : 0xffffffffffffffed) 00000100:00080000:1.0:1787827027.193452:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9856110b0000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00100000:2.0:1787827027.193455:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228031744:192.168.203.114@tcp:400:kworker.0 00000100:00080000:1.0:1787827027.193459:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.114@tcp failed (-19) 00000100:00000001:1.0:1787827027.193463:0:2382:0:(import.c:1517:ptlrpc_connect_interpret()) Process leaving (rc=18446744073709551597 : -19 : ffffffffffffffed) 00000100:00000001:2.0:1787827027.193463:0:2386:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1787827027.193468:0:2386:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1787827027.193470:0:2382:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff985610f81680 x1874669228031616/t0(0) o8->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 520/224 e 0 to 0 dl 1787827042 ref 1 fl Interpret:RNQU/200/0 rc -19/-19 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827027.193470:0:2385:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000040:2.0:1787827027.193471:0:2386:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610f82580 x1874669228031744/t0(0) o400->lustre-OST0001-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827043 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827027.193475:0:2385:0:(client.c:3001:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1787827027.193480:0:2385:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:3.0:1787827027.193482:0:2385:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1787827027.193484:0:2386:0:(client.c:2793:__ptlrpc_free_req()) Process entered 00000100:00100000:1.0:1787827027.193486:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228031616:192.168.203.114@tcp:8:kworker.0 02000000:00000001:2.0:1787827027.193487:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:3.0:1787827027.193487:0:2385:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 02000000:00000010:2.0:1787827027.193489:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff98561288dc00. 00000100:00000001:3.0:1787827027.193490:0:2385:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1787827027.193492:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000001:1.0:1787827027.193493:0:2382:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:3.0:1787827027.193494:0:2385:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1787827027.193495:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000100:00000001:1.0:1787827027.193496:0:2382:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000020:00000040:2.0:1787827027.193498:0:2386:0:(genops.c:1223:class_import_put()) import ffff9856110b1000 refcount=2 obd=lustre-OST0001-osc-ffff985602bc9800 00000100:00000040:1.0:1787827027.193500:0:2382:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610f81680 x1874669228031616/t0(0) o8->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 520/224 e 0 to 0 dl 1787827042 ref 1 fl Complete:RNQU/200/0 rc -19/-19 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000020:00000001:2.0:1787827027.193500:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 00000100:00001000:3.0:1787827027.193500:0:2385:0:(import.c:2149:obd_at_measure()) add 5 to ffff985611182460 time=105 v=5 (5 5 5 5) 02000000:00000010:2.0:1787827027.193503:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff985609025b00. 00000100:00001000:3.0:1787827027.193506:0:2385:0:(import.c:2149:obd_at_measure()) add 1 to ffff985611182418 time=41 v=5 (1 0 0 0) 02000000:00000001:2.0:1787827027.193508:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827027.193510:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827027.193513:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610f82580. 00000100:00000001:3.0:1787827027.193514:0:2385:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1787827027.193518:0:2385:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1787827027.193519:0:2386:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827027.193521:0:2386:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787827027.193523:0:2382:0:(client.c:2793:__ptlrpc_free_req()) Process entered 00010000:00000001:3.0:1787827027.193524:0:2385:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 02000000:00000001:1.0:1787827027.193525:0:2382:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:2.0:1787827027.193525:0:2386:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 02000000:00000010:1.0:1787827027.193529:0:2382:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985607d00800. 00010000:00000001:3.0:1787827027.193530:0:2385:0:(ldlm_request.c:1703:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1787827027.193532:0:2382:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000001:2.0:1787827027.193532:0:2386:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000020:00000001:1.0:1787827027.193535:0:2382:0:(genops.c:1217:class_import_put()) Process entered 00000100:00000001:2.0:1787827027.193536:0:2386:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827027.193536:0:2385:0:(client.c:3109:ptlrpc_free_committed()) Process entered 00000020:00000040:1.0:1787827027.193537:0:2382:0:(genops.c:1223:class_import_put()) import ffff9856110b0000 refcount=2 obd=lustre-OST0000-osc-ffff985602bc9800 00000020:00000001:1.0:1787827027.193540:0:2382:0:(genops.c:1230:class_import_put()) Process leaving 00000100:00000001:2.0:1787827027.193541:0:2386:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:3.0:1787827027.193541:0:2385:0:(client.c:3121:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: skip recheck: last_committed 219043334508 02000000:00000010:1.0:1787827027.193542:0:2382:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 1024 at ffff985608492400. 02000000:00000001:1.0:1787827027.193544:0:2382:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:2.0:1787827027.193544:0:2386:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827027.193547:0:2385:0:(client.c:3122:ptlrpc_free_committed()) Process leaving 02000000:00000001:1.0:1787827027.193548:0:2382:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1787827027.193551:0:2382:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610f81680. 00000100:00000001:3.0:1787827027.193552:0:2385:0:(client.c:1797:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827027.193555:0:2382:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1787827027.193557:0:2382:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1787827027.193560:0:2385:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff985610f82940 x1874669228031360/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827043 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827027.193561:0:2382:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787827027.193567:0:2382:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1787827027.193570:0:2382:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827027.193571:0:2385:0:(client.c:2354:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1787827027.193573:0:2382:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1787827027.193575:0:2385:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1787827027.193576:0:2382:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827027.193578:0:2385:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1787827027.193584:0:2385:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff985610f82940 x1874669228031360/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827043 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827027.193591:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669228031360:192.168.203.114@tcp:400:kworker.0 00000100:00000001:3.0:1787827027.193595:0:2385:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1787827027.193597:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1787827027.193599:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610f82940 x1874669228031360/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827043 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827027.193604:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1787827027.193606:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1787827027.193608:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff985611860e00. 02000000:00000001:3.0:1787827027.193610:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1787827027.193612:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:3.0:1787827027.193613:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=3 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:3.0:1787827027.193615:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:3.0:1787827027.193616:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff985609025f00. 02000000:00000001:3.0:1787827027.193618:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1787827027.193619:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1787827027.193621:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610f82940. 00000100:00000001:3.0:1787827027.193623:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1787827027.193624:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1787827027.193625:0:2385:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1787827027.193629:0:2385:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1787827027.193631:0:2385:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827027.193633:0:2385:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1787827027.193634:0:2385:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827027.764812:0:74885:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827027.764818:0:74885:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827027.764825:0:74885:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827027.764826:0:74885:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827027.764830:0:74885:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827027.764831:0:74885:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827027.764834:0:74885:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827027.764835:0:74885:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1787827027.764898:0:79887:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1787827027.764902:0:79887:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1787827027.764905:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:0.0:1787827028.788865:0:74885:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827028.788911:0:74885:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827028.788918:0:74885:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827028.788920:0:74885:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827028.788924:0:74885:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827028.788927:0:74885:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:0.0:1787827028.788930:0:74885:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:0.0:1787827028.788939:0:74885:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:0.0:1787827028.788942:0:74885:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:0.0:1787827028.788944:0:74885:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827028.788948:0:74885:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:0.0:1787827028.788950:0:74885:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827028.788953:0:74885:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827028.788956:0:74885:0:(ldlm_pool.c:496:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827028.788962:0:74885:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827028.788964:0:74885:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1787827028.789043:0:15163:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1787827028.789050:0:15163:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1787827028.789054:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:0.0:1787827029.812841:0:74885:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827029.812847:0:74885:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827029.812852:0:74885:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827029.812853:0:74885:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827029.812856:0:74885:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827029.812856:0:74885:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827029.812858:0:74885:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827029.812859:0:74885:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1787827029.812912:0:33341:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1787827029.812917:0:33341:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1787827029.812921:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:0.0:1787827030.836836:0:74885:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827030.836845:0:74885:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:0.0:1787827030.836847:0:74885:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:0.0:1787827030.836850:0:74885:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827030.836855:0:74885:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:0.0:1787827030.836857:0:74885:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827030.836859:0:74885:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827030.836861:0:74885:0:(ldlm_pool.c:496:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827030.836866:0:74885:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827030.836868:0:74885:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827030.836871:0:74885:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827030.836873:0:74885:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827030.836876:0:74885:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827030.836877:0:74885:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1787827030.836902:0:15164:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1787827030.836905:0:15164:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1787827030.836907:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:0.0:1787827031.860834:0:74885:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827031.860842:0:74885:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827031.860850:0:74885:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827031.860852:0:74885:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827031.860856:0:74885:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827031.860857:0:74885:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827031.860861:0:74885:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827031.860862:0:74885:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1787827031.860890:0:33342:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1787827031.860893:0:33342:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1787827031.860896:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:3.0:1787827032.308828:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1787827032.308838:0:37:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1787827032.308848:0:37:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 904 at ffff985610f82940. 00000020:00000040:3.0:1787827032.308853:0:37:0:(genops.c:1210:class_import_get()) import ffff98561118e000 refcount=4 obd=MGC192.168.203.114@tcp 00000100:00000001:3.0:1787827032.308857:0:37:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1787827032.308860:0:37:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1787827032.308865:0:37:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1787827032.308872:0:37:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff985609025f00. 00000100:00000001:3.0:1787827032.308878:0:37:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1787827032.308891:0:37:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS req@ffff985610f82940 x1874669228031872/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1787827032.308901:0:37:0:(jobid.c:939:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1787827032.308905:0:37:0:(jobid.c:979:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1787827032.308911:0:37:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000abc5c067] to pc [ptlrpcd_01_00+0] req@ffff985610f82940 x1874669228031872/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827032.308953:0:37:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1787827032.308956:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1787827032.308959:0:37:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1787827032.308962:0:37:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 904 at ffff985610f83c00. 00000020:00000040:3.0:1787827032.308965:0:37:0:(genops.c:1210:class_import_get()) import ffff985611182000 refcount=4 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000100:00000001:3.0:1787827032.308967:0:37:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1787827032.308969:0:37:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1787827032.308971:0:37:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1787827032.308974:0:37:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff985609025a00. 00000100:00000001:3.0:1787827032.308978:0:37:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1787827032.308983:0:37:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID req@ffff985610f83c00 x1874669228032000/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1787827032.308987:0:2385:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1787827032.308992:0:2385:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1787827032.308996:0:2385:0:(client.c:1813:ptlrpc_send_new_req()) Process entered 00000100:00000001:3.0:1787827032.309001:0:37:0:(jobid.c:939:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1787827032.309005:0:37:0:(jobid.c:979:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1787827032.309006:0:2385:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff985610f82940 x1874669228031872/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1787827032.309011:0:37:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000759c64fa] to pc [ptlrpcd_01_01+1] req@ffff985610f83c00 x1874669228032000/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827032.309022:0:2385:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1787827032.309026:0:2385:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827032.309029:0:37:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1787827032.309032:0:2385:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000040:3.0:1787827032.309032:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000001:2.0:1787827032.309036:0:2385:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1787827032.309038:0:37:0:(pinger.c:119:ptlrpc_ping()) Process entered 02000000:00000001:2.0:1787827032.309042:0:2385:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:3.0:1787827032.309043:0:37:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 904 at ffff985610f821c0. 00000100:00100000:2.0:1787827032.309049:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2385:1874669228031872:192.168.203.114@tcp:400:kworker.0 00000020:00000040:3.0:1787827032.309049:0:37:0:(genops.c:1210:class_import_get()) import ffff985611185800 refcount=4 obd=lustre-MDT0001-mdc-ffff985602bc9800 00000100:00000001:3.0:1787827032.309053:0:37:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1787827032.309055:0:37:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1787827032.309056:0:37:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1787827032.309063:0:2385:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000010:3.0:1787827032.309063:0:37:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff985609025200. 00000100:00000001:3.0:1787827032.309068:0:37:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1787827032.309070:0:2385:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1787827032.309074:0:2385:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1787827032.309074:0:37:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID req@ffff985610f821c0 x1874669228032128/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 02000000:00000001:2.0:1787827032.309078:0:2385:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:2.0:1787827032.309085:0:2385:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff985607c38600. 00000100:00000001:3.0:1787827032.309086:0:37:0:(jobid.c:939:lustre_get_jobid()) Process entered 02000000:00000001:2.0:1787827032.309090:0:2385:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827032.309091:0:37:0:(jobid.c:979:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1787827032.309099:0:2385:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000010:2.0:1787827032.309106:0:2385:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff98561080ec08. 00000100:00000040:3.0:1787827032.309108:0:37:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000fa701676] to pc [ptlrpcd_01_00+0] req@ffff985610f821c0 x1874669228032128/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000400:00000010:2.0:1787827032.309114:0:2385:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985610afa2a8. 00000100:00000200:2.0:1787827032.309123:0:2385:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1874669228031872, portal 25 00000100:00000001:3.0:1787827032.309125:0:37:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1787827032.309127:0:2385:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00080000:3.0:1787827032.309129:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1787827032.309130:0:2385:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446630094152149312 : -113979557402304 : ffff985610f82940) 00000100:00080000:3.0:1787827032.309137:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00000040:2.0:1787827032.309141:0:2385:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff985610f82940 x1874669228031872/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827048 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827032.309141:0:37:0:(import.c:701:ptlrpc_connect_import_locked()) Process entered 00000100:00080000:3.0:1787827032.309145:0:37:0:(import.c:65:import_set_state_nolock()) ffff9856110b0000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00000001:3.0:1787827032.309152:0:37:0:(import.c:485:import_select_connection()) Process entered 00000100:00000001:2.0:1787827032.309157:0:2385:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00080000:3.0:1787827032.309158:0:37:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3177 00000100:00000040:2.0:1787827032.309164:0:2385:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.203.114@tcp 00000400:00000010:2.0:1787827032.309168:0:2385:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985610afac38. 00000100:00000200:2.0:1787827032.309172:0:2385:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1874669228031872, offset 0 00000400:00000200:2.0:1787827032.309178:0:2385:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.203.114@tcp 00000400:00000200:3.0:1787827032.309283:0:37:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:3.0:1787827032.309292:0:37:0:(peer.c:1685:LNetPeerDiscovered()) Peer NID 192.168.203.114@tcp is discovered: rc = 1 00000400:00000200:2.0:1787827032.309295:0:2385:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00001000:3.0:1787827032.309298:0:37:0:(import.c:2149:obd_at_measure()) add 15 to ffff9856110b0418 time=11 v=10 (10 1 1 1) 00000100:00001000:3.0:1787827032.309303:0:37:0:(import.c:2215:obd_at_measure()) AT ffff9856110b0418 change: old=10 new=15 delta=5 (val=15) hist 15 1 1 1 00000400:00000200:2.0:1787827032.309304:0:2385:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.203.114@tcp local destination 00000100:00080000:3.0:1787827032.309309:0:37:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff985602bc9800: tried all connections, increasing latency to 10s 00000400:00000200:2.0:1787827032.309312:0:2385:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000040:3.0:1787827032.309316:0:37:0:(lustre_net.h:1965:ptlrpc_connection_put()) PUT conn=ffff98561201f640 refcount 14 to 192.168.203.114@tcp 00000100:00000001:3.0:1787827032.309319:0:37:0:(connection.c:156:ptlrpc_connection_addref()) Process entered 00000100:00000040:3.0:1787827032.309322:0:37:0:(connection.c:161:ptlrpc_connection_addref()) conn=ffff98561201f640 refcount 15 to 192.168.203.114@tcp 00000100:00000001:3.0:1787827032.309325:0:37:0:(connection.c:163:ptlrpc_connection_addref()) Process leaving (rc=18446630094169568832 : -113979539982784 : ffff98561201f640) 00000400:00000200:2.0:1787827032.309329:0:2385:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.203.14@tcp 00000020:00000001:3.0:1787827032.309330:0:37:0:(genops.c:792:class_conn2export()) Process entered 00000400:00000200:2.0:1787827032.309335:0:2385:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.203.14@tcp lpni_is_preferred = 0 00000400:00000200:2.0:1787827032.309338:0:2385:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.203.114@tcp 00000400:00000200:2.0:1787827032.309342:0:2385:0:(lib-move.c:1745:lnet_handle_send()) 192.168.203.14@tcp NI seq info: [21885:21885:256:4294967295] 192.168.203.114@tcp LPNI seq info [21885:21885:8:4294967295] 00000020:00000040:3.0:1787827032.309344:0:37:0:(genops.c:804:class_conn2export()) looking for export cookie 0xb3f83dad1b7dcdd6 00000400:00000200:2.0:1787827032.309348:0:2385:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000020:00000001:3.0:1787827032.309349:0:37:0:(lustre_handles.c:130:class_handle2object()) Process entered 00000020:00000040:3.0:1787827032.309354:0:37:0:(lustre_handles.c:148:class_handle2object()) GET export ffff985602cdf000 refcount=4 00000020:00000001:3.0:1787827032.309359:0:37:0:(lustre_handles.c:155:class_handle2object()) Process leaving (rc=18446630093914501120 : -113979795050496 : ffff985602cdf000) 00000400:00000200:2.0:1787827032.309363:0:2385:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp:) -> 192.168.203.114@tcp(192.168.203.114@tcp:192.168.203.114@tcp) : PUT try# 0 00000020:00000001:3.0:1787827032.309364:0:37:0:(genops.c:806:class_conn2export()) Process leaving (rc=18446630093914501120 : -113979795050496 : ffff985602cdf000) 00000100:00000040:3.0:1787827032.309371:0:37:0:(lustre_net.h:1965:ptlrpc_connection_put()) PUT conn=ffff98561201f640 refcount 14 to 192.168.203.114@tcp 00000100:00000001:3.0:1787827032.309372:0:37:0:(connection.c:156:ptlrpc_connection_addref()) Process entered 00000100:00000040:3.0:1787827032.309374:0:37:0:(connection.c:161:ptlrpc_connection_addref()) conn=ffff98561201f640 refcount 15 to 192.168.203.114@tcp 00000100:00000001:3.0:1787827032.309376:0:37:0:(connection.c:163:ptlrpc_connection_addref()) Process leaving (rc=18446630094169568832 : -113979539982784 : ffff98561201f640) 00000020:00000040:3.0:1787827032.309379:0:37:0:(genops.c:879:class_export_put()) PUTting export ffff985602cdf000 : new refcount 3 00000100:00080000:3.0:1787827032.309382:0:37:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff985602bc9800: import ffff9856110b0000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00000001:3.0:1787827032.309385:0:37:0:(import.c:628:import_select_connection()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1787827032.309387:0:37:0:(sec.c:1579:sptlrpc_import_sec_adapt()) Process entered 02000000:00000001:3.0:1787827032.309392:0:37:0:(sec.c:1619:sptlrpc_import_sec_adapt()) Process leaving via out (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1787827032.309394:0:37:0:(sec.c:1647:sptlrpc_import_sec_adapt()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827032.309396:0:37:0:(obd_class.h:889:obd_reconnect()) Process entered 00000008:00100000:3.0:1787827032.309400:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683216 ocd_grant: 8388608, lost: 0. 00000008:00000001:3.0:1787827032.309403:0:37:0:(osc_request.c:4022:osc_reconnect()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827032.309405:0:37:0:(obd_class.h:903:obd_reconnect()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:3.0:1787827032.309410:0:37:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 904 at ffff985610f81680. 00000020:00000040:3.0:1787827032.309412:0:37:0:(genops.c:1210:class_import_get()) import ffff9856110b0000 refcount=3 obd=lustre-OST0000-osc-ffff985602bc9800 02000000:00000001:3.0:1787827032.309415:0:37:0:(sec.c:2038:sptlrpc_sepol_get()) Process entered 02000000:00000001:3.0:1787827032.309417:0:37:0:(sec.c:2048:sptlrpc_sepol_get()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827032.309419:0:37:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1787827032.309421:0:37:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1787827032.309423:0:37:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1787827032.309427:0:37:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 1024 at ffff985608493400. 00000100:00000001:3.0:1787827032.309430:0:37:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1787827032.309437:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81680 x1874669228032256/t0(0) o8->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000200:2.0:1787827032.309442:0:2385:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.203.114@tcp 00000800:00000010:2.0:1787827032.309451:0:2385:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff985615779600. 00000100:00000001:3.0:1787827032.309451:0:37:0:(jobid.c:939:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1787827032.309454:0:37:0:(jobid.c:979:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1787827032.309460:0:37:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000e6e8a1e6] to pc [ptlrpcd_rcv+-1] req@ffff985610f81680 x1874669228032256/t0(0) o8->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000800:00000200:2.0:1787827032.309461:0:2385:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff985606751480] -> 12345-192.168.203.114@tcp (5) 00000800:00000200:2.0:1787827032.309473:0:2385:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.203.114@tcp ip 192.168.203.114:988 00000800:00000200:2.0:1787827032.309478:0:2385:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff985615779600 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1787827032.309484:0:37:0:(import.c:833:ptlrpc_connect_import_locked()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1787827032.309488:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-OST0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1787827032.309491:0:2385:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827032.309492:0:37:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1787827032.309498:0:37:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 904 at ffff985610f82580. 00000400:00000200:2.0:1787827032.309499:0:2385:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000020:00000040:3.0:1787827032.309503:0:37:0:(genops.c:1210:class_import_get()) import ffff9856110b1000 refcount=3 obd=lustre-OST0001-osc-ffff985602bc9800 00000100:00000001:2.0:1787827032.309504:0:2385:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.203.114@tcp: PM QoS new deadline estimation for cpu 2 is 4297860348 00000100:00000001:3.0:1787827032.309508:0:37:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1787827032.309510:0:37:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1787827032.309512:0:37:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1787827032.309513:0:2385:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.203.114@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4297855228 02000000:00000010:3.0:1787827032.309517:0:37:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff985609025400. 00000100:00000001:2.0:1787827032.309520:0:2385:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.203.114@tcp: PM QoS new deadline estimation for cpu 3 is 4297860348 00000100:00000001:3.0:1787827032.309522:0:37:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827032.309525:0:2382:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1787827032.309526:0:2385:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.203.114@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4297855228 00000100:00000001:1.0:1787827032.309530:0:2382:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000040:3.0:1787827032.309530:0:37:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-OST0001_UUID req@ffff985610f82580 x1874669228032384/t0(0) o400->lustre-OST0001-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1787827032.309533:0:2385:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1787827032.309534:0:2382:0:(client.c:1813:ptlrpc_send_new_req()) Process entered 00000100:00000001:2.0:1787827032.309540:0:2385:0:(client.c:1941:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1787827032.309545:0:2385:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827032.309546:0:37:0:(jobid.c:939:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1787827032.309550:0:2385:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1787827032.309551:0:2382:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff985610f81680 x1874669228032256/t0(0) o8->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827032.309551:0:37:0:(jobid.c:979:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1787827032.309561:0:2385:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000040:3.0:1787827032.309561:0:37:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000db1fd97d] to pc [ptlrpcd_01_01+1] req@ffff985610f82580 x1874669228032384/t0(0) o400->lustre-OST0001-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827032.309565:0:2382:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1787827032.309566:0:2385:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1787827032.309569:0:2382:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1787827032.309572:0:2385:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:1.0:1787827032.309575:0:2382:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:2.0:1787827032.309576:0:2385:0:(client.c:1990:ptlrpc_check_set()) Process entered 02000000:00000001:1.0:1787827032.309579:0:2382:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1787827032.309580:0:2385:0:(client.c:1813:ptlrpc_send_new_req()) Process entered 00000100:00000001:3.0:1787827032.309581:0:37:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1787827032.309584:0:2382:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1787827032.309588:0:37:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (3188) 00000100:00000040:2.0:1787827032.309589:0:2385:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff985610f821c0 x1874669228032128/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827032.309592:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228032256:192.168.203.114@tcp:8:kworker.0 00000100:00000001:1.0:1787827032.309600:0:2382:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:3.0:1787827032.309602:0:2386:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:1.0:1787827032.309604:0:2382:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:3.0:1787827032.309606:0:2386:0:(client.c:1990:ptlrpc_check_set()) Process entered 02000000:00000001:1.0:1787827032.309608:0:2382:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827032.309609:0:2386:0:(client.c:1813:ptlrpc_send_new_req()) Process entered 02000000:00000001:1.0:1787827032.309613:0:2382:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000040:3.0:1787827032.309617:0:2386:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff985610f83c00 x1874669228032000/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 02000000:00000010:1.0:1787827032.309619:0:2382:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 1024 at ffff985607d00c00. 00000100:00000001:2.0:1787827032.309621:0:2385:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1787827032.309624:0:2385:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1787827032.309625:0:2382:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1787827032.309631:0:2385:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:3.0:1787827032.309634:0:2386:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000400:00000200:1.0:1787827032.309635:0:2382:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 02000000:00000001:2.0:1787827032.309636:0:2385:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1787827032.309636:0:2386:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1787827032.309640:0:2382:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff985602ecf898. 02000000:00000001:2.0:1787827032.309642:0:2385:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1787827032.309643:0:2386:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 00000400:00000010:1.0:1787827032.309646:0:2382:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985606618aa0. 02000000:00000001:3.0:1787827032.309647:0:2386:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00100000:2.0:1787827032.309650:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f821c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669228032128:192.168.203.114@tcp:400:kworker.0 00000100:00000200:1.0:1787827032.309653:0:2382:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 1024 bytes, xid 1874669228032256, portal 4 02000000:00000001:3.0:1787827032.309653:0:2386:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1787827032.309657:0:2385:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:1.0:1787827032.309659:0:2382:0:(client.c:3275:ptlrpc_request_addref()) Process entered 02000000:00000001:2.0:1787827032.309662:0:2385:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:1.0:1787827032.309663:0:2382:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446630094152144512 : -113979557407104 : ffff985610f81680) 00000100:00100000:3.0:1787827032.309663:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228032000:192.168.203.114@tcp:400:kworker.0 02000000:00000001:2.0:1787827032.309666:0:2385:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1787827032.309671:0:2385:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:3.0:1787827032.309672:0:2386:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000040:1.0:1787827032.309675:0:2382:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff985610f81680 x1874669228032256/t0(0) o8->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 520/544 e 0 to 0 dl 1787827052 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1787827032.309676:0:2386:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000010:2.0:1787827032.309677:0:2385:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff985607c39400. 02000000:00000001:3.0:1787827032.309680:0:2386:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1787827032.309682:0:2385:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1787827032.309686:0:2386:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000200:2.0:1787827032.309711:0:2385:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 02000000:00000010:3.0:1787827032.309714:0:2386:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff98561288c200. 00000400:00000010:2.0:1787827032.309716:0:2385:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff98561080e0b0. 02000000:00000001:3.0:1787827032.309719:0:2386:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1787827032.309722:0:2385:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985610afa7f8. 00000100:00000001:1.0:1787827032.309724:0:2382:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000200:2.0:1787827032.309727:0:2385:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1874669228032128, portal 10 00000400:00000200:3.0:1787827032.309728:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000040:1.0:1787827032.309732:0:2382:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.203.114@tcp 00000100:00000001:2.0:1787827032.309732:0:2385:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000400:00000010:3.0:1787827032.309735:0:2386:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff985602ad9790. 00000100:00000001:2.0:1787827032.309736:0:2385:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446630094152147392 : -113979557404224 : ffff985610f821c0) 00000400:00000010:1.0:1787827032.309739:0:2382:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985606618e58. 00000400:00000010:3.0:1787827032.309742:0:2386:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985609573bb0. 00000100:00000200:1.0:1787827032.309745:0:2382:0:(niobuf.c:94:ptl_send_buf()) Sending 520 bytes to portal 28, xid 1874669228032256, offset 0 00000100:00000040:2.0:1787827032.309746:0:2385:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff985610f821c0 x1874669228032128/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827048 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1787827032.309748:0:2386:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1874669228032000, portal 10 00000100:00000001:3.0:1787827032.309754:0:2386:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000400:00000200:1.0:1787827032.309757:0:2382:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.203.114@tcp 00000100:00000001:3.0:1787827032.309758:0:2386:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446630094152154112 : -113979557397504 : ffff985610f83c00) 00000100:00000001:2.0:1787827032.309778:0:2385:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000400:00000200:1.0:1787827032.309780:0:2382:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000040:3.0:1787827032.309782:0:2386:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff985610f83c00 x1874669228032000/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827048 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1787827032.309785:0:2385:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.203.114@tcp 00000400:00000200:1.0:1787827032.309790:0:2382:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.203.114@tcp local destination 00000400:00000010:2.0:1787827032.309791:0:2385:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985610afabb0. 00000100:00000200:2.0:1787827032.309797:0:2385:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1874669228032128, offset 0 00000100:00000001:3.0:1787827032.309797:0:2386:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000400:00000200:1.0:1787827032.309798:0:2382:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000040:3.0:1787827032.309803:0:2386:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.203.114@tcp 00000400:00000200:1.0:1787827032.309806:0:2382:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.203.14@tcp 00000400:00000200:2.0:1787827032.309808:0:2385:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.203.114@tcp 00000400:00000010:3.0:1787827032.309810:0:2386:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985609573088. 00000400:00000200:1.0:1787827032.309812:0:2382:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.203.14@tcp lpni_is_preferred = 0 00000100:00000200:3.0:1787827032.309815:0:2386:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1874669228032000, offset 0 00000400:00000200:2.0:1787827032.309816:0:2385:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:1.0:1787827032.309819:0:2382:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.203.114@tcp 00000400:00000200:1.0:1787827032.309827:0:2382:0:(lib-move.c:1745:lnet_handle_send()) 192.168.203.14@tcp NI seq info: [21886:21886:255:4294967295] 192.168.203.114@tcp LPNI seq info [21886:21886:7:4294967295] 00000400:00000200:2.0:1787827032.309827:0:2385:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.203.114@tcp local destination 00000400:00000200:3.0:1787827032.309828:0:2386:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.203.114@tcp 00000400:00000200:2.0:1787827032.309835:0:2385:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1787827032.309838:0:2382:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827032.309842:0:2385:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.203.14@tcp 00000400:00000200:2.0:1787827032.309845:0:2385:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.203.14@tcp lpni_is_preferred = 0 00000400:00000200:2.0:1787827032.309848:0:2385:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.203.114@tcp 00000400:00000200:2.0:1787827032.309851:0:2385:0:(lib-move.c:1745:lnet_handle_send()) 192.168.203.14@tcp NI seq info: [21887:21887:255:4294967295] 192.168.203.114@tcp LPNI seq info [21887:21887:7:4294967295] 00000400:00000200:2.0:1787827032.309856:0:2385:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827032.309862:0:2385:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp:) -> 192.168.203.114@tcp(192.168.203.114@tcp:192.168.203.114@tcp) : PUT try# 0 00000800:00000200:2.0:1787827032.309867:0:2385:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.203.114@tcp 00000400:00000200:3.0:1787827032.309870:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000800:00000010:2.0:1787827032.309871:0:2385:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff985615779800. 00000400:00000200:3.0:1787827032.309879:0:2386:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.203.114@tcp local destination 00000800:00000200:2.0:1787827032.309880:0:2385:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff985606751480] -> 12345-192.168.203.114@tcp (5) 00000400:00000200:3.0:1787827032.309888:0:2386:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000800:00000200:2.0:1787827032.309889:0:2385:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.203.114@tcp ip 192.168.203.114:988 00000800:00000200:2.0:1787827032.309894:0:2385:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff985615779800 type 1, nob 320 niov 1 nkiov 1 00000400:00000200:3.0:1787827032.309895:0:2386:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.203.14@tcp 00000100:00000001:2.0:1787827032.309900:0:2385:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1787827032.309901:0:2386:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.203.14@tcp lpni_is_preferred = 0 00000100:00000040:2.0:1787827032.309906:0:2385:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff985602bc9800: using a timeout of 11963 usecs (12 jiffies) 00000400:00000200:3.0:1787827032.309907:0:2386:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.203.114@tcp 00000400:00000200:2.0:1787827032.309914:0:2385:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:3.0:1787827032.309915:0:2386:0:(lib-move.c:1745:lnet_handle_send()) 192.168.203.14@tcp NI seq info: [21888:21888:254:4294967295] 192.168.203.114@tcp LPNI seq info [21888:21888:6:4294967295] 00000100:00000001:2.0:1787827032.309919:0:2385:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff985602bc9800: PM QoS new deadline estimation for cpu 2 is 4297850381 00000400:00000200:3.0:1787827032.309923:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000001:2.0:1787827032.309924:0:2385:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff985602bc9800: PM QoS request active for cpu 2, keep current deadline 4297860348 00000100:00000001:2.0:1787827032.309928:0:2385:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff985602bc9800: PM QoS new deadline estimation for cpu 3 is 4297850381 00000100:00000001:2.0:1787827032.309934:0:2385:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff985602bc9800: PM QoS request active for cpu 3, keep current deadline 4297860348 00000400:00000200:3.0:1787827032.309936:0:2386:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp:) -> 192.168.203.114@tcp(192.168.203.114@tcp:192.168.203.114@tcp) : PUT try# 0 00000800:00000200:3.0:1787827032.309942:0:2386:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.203.114@tcp 00000400:00000200:1.0:1787827032.309947:0:2382:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp:) -> 192.168.203.114@tcp(192.168.203.114@tcp:192.168.203.114@tcp) : PUT try# 0 00000800:00000010:3.0:1787827032.309947:0:2386:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff985609025b00. 00000100:00000001:2.0:1787827032.309949:0:2385:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1787827032.309956:0:2385:0:(client.c:1941:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1787827032.309956:0:2386:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff985606751480] -> 12345-192.168.203.114@tcp (5) 00000800:00000200:1.0:1787827032.309959:0:2382:0:(socklnd_cb.c:931:ksocknal_send()) sending 520 bytes in 1 frags to 12345-192.168.203.114@tcp 00000100:00000001:2.0:1787827032.309962:0:2385:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:1.0:1787827032.309966:0:2382:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff985607d95e00. 00000100:00000001:2.0:1787827032.309966:0:2385:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000800:00000200:3.0:1787827032.309967:0:2386:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.203.114@tcp ip 192.168.203.114:988 00000800:00000200:3.0:1787827032.309973:0:2386:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff985609025b00 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:2.0:1787827032.309974:0:2385:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000800:00000200:1.0:1787827032.309976:0:2382:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff985606751480] -> 12345-192.168.203.114@tcp (5) 00000100:00000001:2.0:1787827032.309977:0:2385:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1787827032.309980:0:2386:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1787827032.309983:0:2385:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:3.0:1787827032.309985:0:2386:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff985602bc9800: using a timeout of 38051 usecs (39 jiffies) 00000800:00000200:1.0:1787827032.309987:0:2382:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.203.114@tcp ip 192.168.203.114:988 00000100:00000001:2.0:1787827032.309987:0:2385:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1787827032.309991:0:2385:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1787827032.309992:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000800:00000200:1.0:1787827032.309994:0:2382:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff985607d95e00 type 1, nob 616 niov 1 nkiov 1 00000100:00000001:2.0:1787827032.309996:0:2385:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827032.310000:0:2386:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff985602bc9800: PM QoS new deadline estimation for cpu 2 is 4297850408 00000100:00000001:1.0:1787827032.310001:0:2382:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827032.310004:0:2386:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff985602bc9800: PM QoS request active for cpu 2, keep current deadline 4297860348 00000100:00000040:1.0:1787827032.310006:0:2382:0:(niobuf.c:635:kick_cpu_latency()) lustre-OST0000-osc-ffff985602bc9800: using a timeout of 14610 usecs (15 jiffies) 00000100:00000001:3.0:1787827032.310008:0:2386:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff985602bc9800: PM QoS new deadline estimation for cpu 3 is 4297850408 00000100:00000001:3.0:1787827032.310013:0:2386:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff985602bc9800: PM QoS request active for cpu 3, keep current deadline 4297860348 00000400:00000200:1.0:1787827032.310014:0:2382:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000001:1.0:1787827032.310017:0:2382:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff985602bc9800: PM QoS new deadline estimation for cpu 2 is 4297850384 00000100:00000001:3.0:1787827032.310018:0:2386:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1787827032.310021:0:2382:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff985602bc9800: PM QoS request active for cpu 2, keep current deadline 4297860348 00000100:00000001:3.0:1787827032.310023:0:2386:0:(client.c:1941:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827032.310024:0:2382:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff985602bc9800: PM QoS new deadline estimation for cpu 3 is 4297850384 00000100:00000001:1.0:1787827032.310028:0:2382:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff985602bc9800: PM QoS request active for cpu 3, keep current deadline 4297860348 00000100:00000001:3.0:1787827032.310029:0:2386:0:(client.c:1813:ptlrpc_send_new_req()) Process entered 00000100:00000001:1.0:1787827032.310030:0:2382:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1787827032.310033:0:2382:0:(client.c:1941:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827032.310036:0:2382:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1787827032.310037:0:2386:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff985610f82580 x1874669228032384/t0(0) o400->lustre-OST0001-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827032.310038:0:2382:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787827032.310042:0:2382:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1787827032.310044:0:2382:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=5 : 5 : 5) 00000100:00000001:1.0:1787827032.310047:0:2382:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1787827032.310049:0:2382:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1787827032.310051:0:2382:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827032.310052:0:2386:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1787827032.310053:0:2382:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827032.310056:0:2386:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1787827032.310060:0:2386:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1787827032.310062:0:2386:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1787827032.310064:0:2386:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1787827032.310067:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228032384:192.168.203.114@tcp:400:kworker.0 00000100:00000001:3.0:1787827032.310070:0:2386:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:3.0:1787827032.310074:0:2386:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1787827032.310075:0:2386:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1787827032.310077:0:2386:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:3.0:1787827032.310081:0:2386:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff98561288ce00. 02000000:00000001:3.0:1787827032.310082:0:2386:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1787827032.310085:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000010:3.0:1787827032.310088:0:2386:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff985602ad93c8. 00000400:00000010:3.0:1787827032.310091:0:2386:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985609573aa0. 00000100:00000200:3.0:1787827032.310094:0:2386:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1874669228032384, portal 4 00000100:00000001:3.0:1787827032.310096:0:2386:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1787827032.310097:0:2386:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446630094152148352 : -113979557403264 : ffff985610f82580) 00000100:00000040:3.0:1787827032.310102:0:2386:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff985610f82580 x1874669228032384/t0(0) o400->lustre-OST0001-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827048 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827032.310108:0:2386:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1787827032.310111:0:2386:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.203.114@tcp 00000400:00000010:3.0:1787827032.310113:0:2386:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985609573770. 00000100:00000200:3.0:1787827032.310115:0:2386:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 28, xid 1874669228032384, offset 0 00000400:00000200:3.0:1787827032.310120:0:2386:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.203.114@tcp 00000400:00000200:3.0:1787827032.310123:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:3.0:1787827032.310127:0:2386:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.203.114@tcp local destination 00000400:00000200:3.0:1787827032.310129:0:2386:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:3.0:1787827032.310132:0:2386:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.203.14@tcp 00000400:00000200:3.0:1787827032.310134:0:2386:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.203.14@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1787827032.310136:0:2386:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.203.114@tcp 00000400:00000200:3.0:1787827032.310139:0:2386:0:(lib-move.c:1745:lnet_handle_send()) 192.168.203.14@tcp NI seq info: [21889:21889:252:4294967295] 192.168.203.114@tcp LPNI seq info [21889:21889:4:4294967295] 00000400:00000200:3.0:1787827032.310142:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:3.0:1787827032.310148:0:2386:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp:) -> 192.168.203.114@tcp(192.168.203.114@tcp:192.168.203.114@tcp) : PUT try# 0 00000800:00000200:3.0:1787827032.310153:0:2386:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.203.114@tcp 00000800:00000010:3.0:1787827032.310156:0:2386:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff985609025d00. 00000800:00000200:3.0:1787827032.310160:0:2386:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff985606751480] -> 12345-192.168.203.114@tcp (5) 00000800:00000200:3.0:1787827032.310164:0:2386:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.203.114@tcp ip 192.168.203.114:988 00000800:00000200:3.0:1787827032.310166:0:2386:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff985609025d00 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1787827032.310177:0:2386:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1787827032.310180:0:2386:0:(niobuf.c:635:kick_cpu_latency()) lustre-OST0001-osc-ffff985602bc9800: using a timeout of 13843 usecs (14 jiffies) 00000400:00000200:3.0:1787827032.310184:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000001:3.0:1787827032.310185:0:2386:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0001-osc-ffff985602bc9800: PM QoS new deadline estimation for cpu 2 is 4297850383 00000100:00000001:3.0:1787827032.310187:0:2386:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0001-osc-ffff985602bc9800: PM QoS request active for cpu 2, keep current deadline 4297860348 00000100:00000001:3.0:1787827032.310189:0:2386:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0001-osc-ffff985602bc9800: PM QoS new deadline estimation for cpu 3 is 4297850383 00000100:00000001:3.0:1787827032.310191:0:2386:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0001-osc-ffff985602bc9800: PM QoS request active for cpu 3, keep current deadline 4297860348 00000100:00000001:3.0:1787827032.310193:0:2386:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1787827032.310195:0:2386:0:(client.c:1941:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827032.310197:0:2386:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827032.310199:0:2386:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1787827032.310202:0:2386:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1787827032.310204:0:2386:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1787827032.310206:0:2386:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1787827032.310207:0:2386:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1787827032.310209:0:2386:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827032.310211:0:2386:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1787827032.310262:0:2378:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1787827032.310266:0:2378:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff985615779600. 00000400:00000200:3.0:1787827032.310273:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1787827032.310278:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000400:00000200:3.0:1787827032.310283:0:2378:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985610afac38 00000400:00000010:3.0:1787827032.310285:0:2378:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985610afac38. 00000100:00000001:3.0:1787827032.310288:0:2378:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1787827032.310292:0:2378:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff985610f82940 x1874669228031872/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827048 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827032.310300:0:2378:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1787827032.310304:0:2378:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff985610f82940 x1874669228031872/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827048 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827032.310311:0:2378:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827032.310312:0:2378:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1787827032.310322:0:2378:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1787827032.310325:0:2378:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff985615779800. 00000400:00000200:3.0:1787827032.310327:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1787827032.310330:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000400:00000200:3.0:1787827032.310332:0:2378:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985610afabb0 00000400:00000010:3.0:1787827032.310334:0:2378:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985610afabb0. 00000100:00000001:3.0:1787827032.310336:0:2378:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1787827032.310339:0:2378:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff985610f821c0 x1874669228032128/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827048 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827032.310345:0:2378:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1787827032.310348:0:2378:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff985610f821c0 x1874669228032128/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827048 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827032.310354:0:2378:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827032.310355:0:2378:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1787827032.310361:0:2378:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1787827032.310362:0:2378:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff985609025b00. 00000400:00000200:3.0:1787827032.310365:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1787827032.310368:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000400:00000200:3.0:1787827032.310370:0:2378:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985609573088 00000400:00000010:3.0:1787827032.310372:0:2378:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985609573088. 00000100:00000001:3.0:1787827032.310374:0:2378:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1787827032.310377:0:2378:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff985610f83c00 x1874669228032000/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827048 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827032.310384:0:2378:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1787827032.310387:0:2378:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff985610f83c00 x1874669228032000/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827048 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827032.310393:0:2378:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827032.310394:0:2378:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1787827032.310401:0:2378:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1787827032.310403:0:2378:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff985607d95e00. 00000400:00000200:3.0:1787827032.310406:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1787827032.310409:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000400:00000200:3.0:1787827032.310411:0:2378:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985606618e58 00000400:00000010:3.0:1787827032.310413:0:2378:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985606618e58. 00000100:00000001:3.0:1787827032.310415:0:2378:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1787827032.310419:0:2378:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff985610f81680 x1874669228032256/t0(0) o8->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 520/544 e 0 to 0 dl 1787827052 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827032.310425:0:2378:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1787827032.310428:0:2378:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff985610f81680 x1874669228032256/t0(0) o8->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 520/544 e 0 to 0 dl 1787827052 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827032.310434:0:2378:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827032.310436:0:2378:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1787827032.310486:0:2378:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1787827032.310488:0:2378:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff985609025d00. 00000400:00000200:3.0:1787827032.310491:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1787827032.310494:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000400:00000200:3.0:1787827032.310496:0:2378:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985609573770 00000400:00000010:3.0:1787827032.310497:0:2378:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985609573770. 00000100:00000001:3.0:1787827032.310500:0:2378:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1787827032.310503:0:2378:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff985610f82580 x1874669228032384/t0(0) o400->lustre-OST0001-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827048 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827032.310509:0:2378:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1787827032.310512:0:2378:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff985610f82580 x1874669228032384/t0(0) o400->lustre-OST0001-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827048 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827032.310517:0:2378:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827032.310519:0:2378:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1787827032.311988:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827032.312001:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1787827032.312004:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827032.312007:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827032.312013:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827032.312020:0:2377:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp) <- 192.168.203.114@tcp : PUT - for me 00000400:00000200:2.0:1787827032.312027:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827032.312031:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827032.312035:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827032.312039:0:2377:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.203.114@tcp of length 224 into portal 25 MB=0x6a90071687f80 00000400:00000200:2.0:1787827032.312043:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827032.312048:0:2377:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.203.114@tcp of length 224/224 into md 0x322d5 [1] + 224 00000800:00000001:2.0:1787827032.312054:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827032.312057:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827032.312060:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1787827032.312063:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000100:00000001:2.0:1787827032.312069:0:2377:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1787827032.312075:0:2377:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff985610f82940 x1874669228031872/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827048 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1787827032.312085:0:2377:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff985610f82940 x1874669228031872/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827048 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827032.312100:0:2377:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1787827032.312106:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827032.312109:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1787827032.312111:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827032.312113:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827032.312115:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827032.312119:0:2377:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp) <- 192.168.203.114@tcp : PUT - for me 00000400:00000200:2.0:1787827032.312122:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827032.312125:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827032.312127:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827032.312130:0:2377:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.203.114@tcp of length 224 into portal 10 MB=0x6a90071688080 00000400:00000200:2.0:1787827032.312133:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827032.312136:0:2377:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.203.114@tcp of length 224/224 into md 0x322ed [1] + 224 00000800:00000001:2.0:1787827032.312140:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827032.312142:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827032.312144:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1787827032.312147:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000100:00000001:2.0:1787827032.312150:0:2377:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1787827032.312154:0:2377:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff985610f821c0 x1874669228032128/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827048 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1787827032.312162:0:2377:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff985610f821c0 x1874669228032128/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827048 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827032.312169:0:2377:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1787827032.312171:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827032.312173:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1787827032.312175:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827032.312177:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827032.312180:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827032.312183:0:2377:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp) <- 192.168.203.114@tcp : PUT - for me 00000400:00000200:2.0:1787827032.312186:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827032.312189:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827032.312191:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827032.312194:0:2377:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.203.114@tcp of length 224 into portal 10 MB=0x6a90071688000 00000400:00000200:2.0:1787827032.312196:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827032.312200:0:2377:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.203.114@tcp of length 224/224 into md 0x322f5 [1] + 224 00000800:00000001:2.0:1787827032.312203:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827032.312207:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827032.312209:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1787827032.312212:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000100:00000001:2.0:1787827032.312214:0:2377:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1787827032.312218:0:2377:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff985610f83c00 x1874669228032000/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827048 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1787827032.312226:0:2377:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff985610f83c00 x1874669228032000/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827048 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827032.312236:0:2377:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1787827032.312239:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827032.312242:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1787827032.312385:0:2386:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1787827032.312388:0:2386:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000400:00000001:3.0:1787827032.312393:0:2386:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1787827032.312396:0:2386:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1787827032.312398:0:2386:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff985602ad9790. 00000400:00000200:3.0:1787827032.312402:0:2386:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985609573bb0 00000400:00000010:3.0:1787827032.312404:0:2386:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985609573bb0. 00000100:00000001:3.0:1787827032.312407:0:2386:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1787827032.312444:0:2386:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff985610f83c00 x1874669228032000/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827048 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1787827032.312454:0:2386:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff985610f83c00 x1874669228032000/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827048 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827032.312461:0:2386:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1787827032.312463:0:2386:0:(client.c:3001:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1787827032.312465:0:2386:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:3.0:1787827032.312468:0:2386:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1787827032.312470:0:2386:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1787827032.312472:0:2386:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1787827032.312475:0:2386:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1787827032.312482:0:2386:0:(import.c:2149:obd_at_measure()) add 5 to ffff985611182460 time=110 v=5 (5 5 5 5) 00000100:00001000:3.0:1787827032.312486:0:2386:0:(import.c:2149:obd_at_measure()) add 1 to ffff985611182418 time=46 v=5 (1 0 0 0) 00000100:00000001:3.0:1787827032.312489:0:2386:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1787827032.312490:0:2386:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1787827032.312493:0:2386:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1787827032.312497:0:2386:0:(ldlm_request.c:1703:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827032.312501:0:2386:0:(client.c:3109:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1787827032.312503:0:2386:0:(client.c:3121:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: skip recheck: last_committed 219043334508 00000100:00000001:3.0:1787827032.312505:0:2386:0:(client.c:3122:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1787827032.312506:0:2386:0:(client.c:1797:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1787827032.312511:0:2386:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff985610f83c00 x1874669228032000/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827048 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827032.312518:0:2386:0:(client.c:2354:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1787827032.312520:0:2386:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1787827032.312522:0:2386:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1787827032.312526:0:2386:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff985610f83c00 x1874669228032000/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827048 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827032.312533:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228032000:192.168.203.114@tcp:400:kworker.0 00000100:00000001:3.0:1787827032.312538:0:2386:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827032.312540:0:2386:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1787827032.312544:0:2386:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610f83c00 x1874669228032000/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827048 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827032.312550:0:2386:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1787827032.312553:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1787827032.312555:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff98561288c200. 02000000:00000001:3.0:1787827032.312558:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1787827032.312560:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:3.0:1787827032.312562:0:2386:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=3 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:3.0:1787827032.312564:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:3.0:1787827032.312567:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff985609025a00. 02000000:00000001:3.0:1787827032.312570:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1787827032.312572:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1787827032.312574:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610f83c00. 00000100:00000001:3.0:1787827032.312576:0:2386:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1787827032.312578:0:2386:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1787827032.312580:0:2386:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827032.312582:0:2386:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1787827032.312583:0:2386:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1787827032.312585:0:2386:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827032.312586:0:2386:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827032.312596:0:2385:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1787827032.312598:0:2385:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000400:00000001:3.0:1787827032.312601:0:2385:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1787827032.312602:0:2385:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1787827032.312604:0:2385:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff98561080e0b0. 00000400:00000200:3.0:1787827032.312607:0:2385:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985610afa7f8 00000400:00000010:3.0:1787827032.312609:0:2385:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985610afa7f8. 00000100:00000001:3.0:1787827032.312611:0:2385:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1787827032.312615:0:2385:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff985610f821c0 x1874669228032128/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827048 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1787827032.312623:0:2385:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff985610f821c0 x1874669228032128/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827048 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827032.312629:0:2385:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1787827032.312631:0:2385:0:(client.c:3001:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1787827032.312632:0:2385:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:3.0:1787827032.312634:0:2385:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1787827032.312635:0:2385:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1787827032.312636:0:2385:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1787827032.312638:0:2385:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1787827032.312642:0:2385:0:(import.c:2149:obd_at_measure()) add 5 to ffff985611185c60 time=32 v=5 (5 5 5 5) 00000100:00001000:3.0:1787827032.312645:0:2385:0:(import.c:2149:obd_at_measure()) add 1 to ffff985611185c18 time=32 v=5 (1 1 1 1) 00000100:00000001:3.0:1787827032.312647:0:2385:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1787827032.312649:0:2385:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1787827032.312650:0:2385:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1787827032.312653:0:2385:0:(ldlm_request.c:1703:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827032.312655:0:2385:0:(client.c:3109:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1787827032.312656:0:2385:0:(client.c:3121:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff985602bc9800: skip recheck: last_committed 0 00000100:00000001:3.0:1787827032.312658:0:2385:0:(client.c:3122:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1787827032.312659:0:2385:0:(client.c:1797:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1787827032.312662:0:2385:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff985610f821c0 x1874669228032128/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827048 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827032.312668:0:2385:0:(client.c:2354:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1787827032.312670:0:2385:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1787827032.312671:0:2385:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1787827032.312675:0:2385:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff985610f821c0 x1874669228032128/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827048 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827032.312682:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f821c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669228032128:192.168.203.114@tcp:400:kworker.0 00000400:00000001:3.0:1787827032.312687:0:2385:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1787827032.312688:0:2385:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1787827032.312713:0:2385:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff98561080ec08. 00000400:00000200:3.0:1787827032.312716:0:2385:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985610afa2a8 00000400:00000010:3.0:1787827032.312717:0:2385:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985610afa2a8. 00000100:00000001:3.0:1787827032.312719:0:2385:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1787827032.312722:0:2385:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff985610f82940 x1874669228031872/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827048 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1787827032.312730:0:2385:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff985610f82940 x1874669228031872/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827048 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827032.312737:0:2385:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1787827032.312738:0:2385:0:(client.c:3001:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1787827032.312740:0:2385:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:3.0:1787827032.312741:0:2385:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1787827032.312743:0:2385:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1787827032.312744:0:2385:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1787827032.312746:0:2385:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1787827032.312748:0:2385:0:(import.c:2149:obd_at_measure()) add 5 to ffff98561118e460 time=32 v=5 (5 5 5 5) 00000100:00001000:3.0:1787827032.312751:0:2385:0:(import.c:2149:obd_at_measure()) add 1 to ffff98561118e418 time=47 v=5 (1 0 0 0) 00000100:00000001:3.0:1787827032.312753:0:2385:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1787827032.312755:0:2385:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1787827032.312756:0:2385:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1787827032.312758:0:2385:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827032.312759:0:2385:0:(client.c:1797:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1787827032.312775:0:2385:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff985610f82940 x1874669228031872/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827048 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827032.312781:0:2385:0:(client.c:2354:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1787827032.312783:0:2385:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1787827032.312784:0:2385:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1787827032.312788:0:2385:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff985610f82940 x1874669228031872/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827048 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827032.312795:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2385:1874669228031872:192.168.203.114@tcp:400:kworker.0 00000100:00000001:3.0:1787827032.312799:0:2385:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1787827032.312800:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1787827032.312803:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610f821c0 x1874669228032128/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827048 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827032.312809:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1787827032.312810:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1787827032.312812:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff985607c39400. 02000000:00000001:3.0:1787827032.312814:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1787827032.312816:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:3.0:1787827032.312817:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611185800 refcount=3 obd=lustre-MDT0001-mdc-ffff985602bc9800 00000020:00000001:3.0:1787827032.312819:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:3.0:1787827032.312820:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff985609025200. 02000000:00000001:3.0:1787827032.312822:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1787827032.312823:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1787827032.312824:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610f821c0. 00000100:00000001:3.0:1787827032.312827:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1787827032.312828:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1787827032.312830:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1787827032.312834:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610f82940 x1874669228031872/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827048 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827032.312839:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1787827032.312841:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1787827032.312842:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff985607c38600. 02000000:00000001:3.0:1787827032.312844:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1787827032.312845:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:3.0:1787827032.312846:0:2385:0:(genops.c:1223:class_import_put()) import ffff98561118e000 refcount=3 obd=MGC192.168.203.114@tcp 00000020:00000001:3.0:1787827032.312848:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:3.0:1787827032.312849:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff985609025f00. 02000000:00000001:3.0:1787827032.312851:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1787827032.312852:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1787827032.312854:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610f82940. 00000100:00000001:3.0:1787827032.312855:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1787827032.312856:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1787827032.312858:0:2385:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1787827032.312864:0:2385:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1787827032.312866:0:2385:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827032.312868:0:2385:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1787827032.312869:0:2385:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1787827032.312920:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827032.312925:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1787827032.312927:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827032.312929:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827032.312934:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827032.312937:0:2378:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp) <- 192.168.203.114@tcp : PUT - for me 00000400:00000200:2.0:1787827032.312942:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827032.312945:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827032.312949:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827032.312952:0:2378:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.203.114@tcp of length 224 into portal 4 MB=0x6a90071688180 00000400:00000200:2.0:1787827032.312955:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827032.312959:0:2378:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 4 from 12345-192.168.203.114@tcp of length 224/224 into md 0x3230d [1] + 224 00000800:00000001:2.0:1787827032.312963:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827032.312966:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827032.312968:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1787827032.312971:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000100:00000001:2.0:1787827032.312975:0:2378:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1787827032.312979:0:2378:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff985610f82580 x1874669228032384/t0(0) o400->lustre-OST0001-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827048 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1787827032.312988:0:2378:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff985610f82580 x1874669228032384/t0(0) o400->lustre-OST0001-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827048 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827032.313003:0:2378:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1787827032.313007:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827032.313010:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1787827032.313012:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827032.313014:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827032.313016:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827032.313020:0:2378:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp) <- 192.168.203.114@tcp : PUT - for me 00000400:00000200:2.0:1787827032.313024:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827032.313026:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827032.313029:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827032.313031:0:2378:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.203.114@tcp of length 224 into portal 4 MB=0x6a90071688100 00000400:00000200:2.0:1787827032.313035:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827032.313038:0:2378:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 4 from 12345-192.168.203.114@tcp of length 224/224 into md 0x322e5 [1] + 0 00000800:00000001:2.0:1787827032.313041:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827032.313045:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827032.313047:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1787827032.313050:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000100:00000001:2.0:1787827032.313053:0:2378:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1787827032.313056:0:2378:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff985610f81680 x1874669228032256/t0(0) o8->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 520/544 e 0 to 0 dl 1787827052 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1787827032.313064:0:2378:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=0 replen=544 req@ffff985610f81680 x1874669228032256/t0(0) o8->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 520/544 e 0 to 0 dl 1787827052 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827032.313079:0:2378:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1787827032.313083:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827032.313085:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1787827032.313125:0:2386:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1787827032.313127:0:2386:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000400:00000001:3.0:1787827032.313130:0:2386:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1787827032.313131:0:2386:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1787827032.313132:0:2386:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff985602ad93c8. 00000400:00000200:3.0:1787827032.313135:0:2386:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985609573aa0 00000400:00000010:3.0:1787827032.313137:0:2386:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985609573aa0. 00000100:00000001:3.0:1787827032.313139:0:2386:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1787827032.313143:0:2386:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff985610f82580 x1874669228032384/t0(0) o400->lustre-OST0001-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827048 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1787827032.313152:0:2386:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff985610f82580 x1874669228032384/t0(0) o400->lustre-OST0001-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827048 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827032.313159:0:2386:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1787827032.313161:0:2386:0:(client.c:3001:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1787827032.313163:0:2386:0:(client.c:1593:after_reply()) Process entered 00000100:00000001:1.0:1787827032.313164:0:2382:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:3.0:1787827032.313165:0:2386:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1787827032.313169:0:2382:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1787827032.313169:0:2386:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1787827032.313173:0:2386:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000400:00000001:1.0:1787827032.313175:0:2382:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 02000000:00000001:3.0:1787827032.313176:0:2386:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000001:1.0:1787827032.313180:0:2382:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1787827032.313184:0:2382:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff985602ecf898. 00000100:00001000:3.0:1787827032.313184:0:2386:0:(import.c:2149:obd_at_measure()) add 5 to ffff9856110b1460 time=110 v=5 (5 5 5 5) 00000400:00000200:1.0:1787827032.313190:0:2382:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985606618aa0 00000100:00001000:3.0:1787827032.313191:0:2386:0:(import.c:2149:obd_at_measure()) add 1 to ffff9856110b1418 time=11 v=5 (1 1 1 1) 00000400:00000010:1.0:1787827032.313194:0:2382:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985606618aa0. 00000100:00000001:3.0:1787827032.313196:0:2386:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1787827032.313199:0:2382:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:3.0:1787827032.313200:0:2386:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1787827032.313204:0:2386:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000200:1.0:1787827032.313206:0:2382:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff985610f81680 x1874669228032256/t0(0) o8->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 520/544 e 0 to 0 dl 1787827052 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00000001:3.0:1787827032.313208:0:2386:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827032.313213:0:2386:0:(client.c:3109:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1787827032.313217:0:2386:0:(client.c:3121:ptlrpc_free_committed()) lustre-OST0001-osc-ffff985602bc9800: skip recheck: last_committed 4294967636 00000100:00000001:3.0:1787827032.313221:0:2386:0:(client.c:3122:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1787827032.313224:0:2386:0:(client.c:1797:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:1.0:1787827032.313227:0:2382:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff985610f81680 x1874669228032256/t0(0) o8->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 520/544 e 0 to 0 dl 1787827052 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1787827032.313231:0:2386:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff985610f82580 x1874669228032384/t0(0) o400->lustre-OST0001-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827048 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827032.313240:0:2386:0:(client.c:2354:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1787827032.313242:0:2386:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1787827032.313243:0:2386:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1787827032.313246:0:2386:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff985610f82580 x1874669228032384/t0(0) o400->lustre-OST0001-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827048 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827032.313256:0:2382:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1787827032.313260:0:2382:0:(client.c:3001:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00100000:3.0:1787827032.313263:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228032384:192.168.203.114@tcp:400:kworker.0 00000100:00000001:1.0:1787827032.313267:0:2382:0:(client.c:1593:after_reply()) Process entered 00000100:00000001:3.0:1787827032.313272:0:2386:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:1.0:1787827032.313274:0:2382:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1787827032.313275:0:2386:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000001:1.0:1787827032.313277:0:2382:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1787827032.313281:0:2382:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1787827032.313281:0:2386:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610f82580 x1874669228032384/t0(0) o400->lustre-OST0001-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827048 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1787827032.313285:0:2382:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1787827032.313293:0:2382:0:(import.c:2149:obd_at_measure()) add 0 to ffff9856110b0460 time=110 v=5 (5 5 5 5) 00000100:00000001:3.0:1787827032.313293:0:2386:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1787827032.313296:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 00000100:00001000:1.0:1787827032.313299:0:2382:0:(import.c:2149:obd_at_measure()) add 1 to ffff9856110b0418 time=11 v=15 (15 1 1 1) 02000000:00000010:3.0:1787827032.313300:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff98561288ce00. 00000100:00000001:1.0:1787827032.313305:0:2382:0:(client.c:1495:ptlrpc_check_status()) Process entered 02000000:00000001:3.0:1787827032.313306:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000001:1.0:1787827032.313308:0:2382:0:(client.c:1506:ptlrpc_check_status()) Process leaving (rc=18446744073709551597 : -19 : ffffffffffffffed) 00000020:00000001:3.0:1787827032.313309:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000100:00000001:1.0:1787827032.313313:0:2382:0:(client.c:1711:after_reply()) Process leaving (rc=18446744073709551597 : -19 : ffffffffffffffed) 00000020:00000040:3.0:1787827032.313313:0:2386:0:(genops.c:1223:class_import_put()) import ffff9856110b1000 refcount=2 obd=lustre-OST0001-osc-ffff985602bc9800 00000020:00000001:3.0:1787827032.313318:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:3.0:1787827032.313320:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff985609025400. 00000100:00000040:1.0:1787827032.313322:0:2382:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff985610f81680 x1874669228032256/t0(0) o8->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 520/224 e 0 to 0 dl 1787827052 ref 1 fl Rpc:RNQU/200/0 rc -19/-19 job:'kworker.0' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1787827032.313324:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1787827032.313326:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1787827032.313328:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610f82580. 00000100:00000001:3.0:1787827032.313333:0:2386:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1787827032.313335:0:2386:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787827032.313337:0:2382:0:(client.c:2354:ptlrpc_check_set()) Process leaving via interpret (rc=18446744073709551597 : -19 : 0xffffffffffffffed) 00000100:00000001:3.0:1787827032.313338:0:2386:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787827032.313344:0:2382:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1787827032.313345:0:2386:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1787827032.313347:0:2382:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1787827032.313349:0:2386:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827032.313351:0:2382:0:(import.c:1057:ptlrpc_connect_interpret()) Process entered 00000100:00000001:3.0:1787827032.313353:0:2386:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1787827032.313355:0:2382:0:(import.c:1092:ptlrpc_connect_interpret()) Process leaving via out (rc=18446744073709551597 : -19 : 0xffffffffffffffed) 00000100:00000001:3.0:1787827032.313356:0:2386:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00080000:1.0:1787827032.313361:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9856110b0000 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827032.313380:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.114@tcp failed (-19) 00000100:00000001:1.0:1787827032.313383:0:2382:0:(import.c:1517:ptlrpc_connect_interpret()) Process leaving (rc=18446744073709551597 : -19 : ffffffffffffffed) 00000100:00000040:1.0:1787827032.313387:0:2382:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff985610f81680 x1874669228032256/t0(0) o8->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 520/224 e 0 to 0 dl 1787827052 ref 1 fl Interpret:RNQU/200/0 rc -19/-19 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827032.313395:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228032256:192.168.203.114@tcp:8:kworker.0 00000100:00000001:1.0:1787827032.313399:0:2382:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787827032.313401:0:2382:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1787827032.313405:0:2382:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985610f81680 x1874669228032256/t0(0) o8->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 520/224 e 0 to 0 dl 1787827052 ref 1 fl Complete:RNQU/200/0 rc -19/-19 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827032.313410:0:2382:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1787827032.313411:0:2382:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1787827032.313413:0:2382:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985607d00c00. 02000000:00000001:1.0:1787827032.313416:0:2382:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1787827032.313418:0:2382:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:1.0:1787827032.313419:0:2382:0:(genops.c:1223:class_import_put()) import ffff9856110b0000 refcount=2 obd=lustre-OST0000-osc-ffff985602bc9800 00000020:00000001:1.0:1787827032.313421:0:2382:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:1.0:1787827032.313423:0:2382:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 1024 at ffff985608493400. 02000000:00000001:1.0:1787827032.313426:0:2382:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1787827032.313428:0:2382:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1787827032.313429:0:2382:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985610f81680. 00000100:00000001:1.0:1787827032.313431:0:2382:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1787827032.313433:0:2382:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787827032.313434:0:2382:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787827032.313438:0:2382:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1787827032.313439:0:2382:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827032.313441:0:2382:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1787827032.313443:0:2382:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827032.884811:0:74885:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827032.884817:0:74885:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827032.884821:0:74885:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827032.884822:0:74885:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827032.884824:0:74885:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827032.884825:0:74885:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827032.884827:0:74885:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827032.884827:0:74885:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1787827032.884889:0:79887:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1787827032.884893:0:79887:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1787827032.884896:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:0.0:1787827033.908884:0:74885:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827033.908891:0:74885:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827033.908897:0:74885:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827033.908898:0:74885:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827033.908902:0:74885:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827033.908903:0:74885:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827033.908906:0:74885:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827033.908907:0:74885:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1787827033.908979:0:15163:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1787827033.908986:0:15163:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1787827033.908991:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000800:00000001:2.0:1787827034.671816:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827034.671848:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1787827034.671856:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827034.671863:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827034.671876:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827034.671886:0:2377:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp) <- 192.168.203.114@tcp : PUT - for me 00000400:00000200:2.0:1787827034.671903:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827034.671909:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827034.671916:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827034.671920:0:2377:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.203.114@tcp of length 328 into portal 15 MB=0x6a90071e13a00 00000400:00000200:2.0:1787827034.671932:0:2377:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index f from 12345-192.168.203.114@tcp of length 328/328 into md 0x17e3d [2] + 1312 00000800:00000001:2.0:1787827034.671943:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827034.671961:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827034.671966:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1787827034.671970:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000100:00000001:2.0:1787827034.671979:0:2377:0:(events.c:286:request_in_callback()) Process entered 00000100:00000200:2.0:1787827034.671982:0:2377:0:(events.c:296:request_in_callback()) event type 2, status 0, service ldlm_cbd 00000100:00000010:2.0:1787827034.671989:0:2377:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 904 at ffff9856140d8000. 00000100:00000040:2.0:1787827034.671993:0:2377:0:(events.c:337:request_in_callback()) incoming req@ffff9856140d8000 x1874669235943936 msgsize 328 00000100:00100000:2.0:1787827034.671997:0:2377:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.114@tcp (source: 12345-192.168.203.114@tcp) 00000100:00000001:2.0:1787827034.672014:0:2377:0:(events.c:373:request_in_callback()) Process leaving 00000800:00000001:2.0:1787827034.672022:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827034.672026:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1787827034.672111:0:15155:0:(service.c:2282:ptlrpc_server_handle_req_in()) Process entered 00000100:00100000:3.0:1787827034.672118:0:15155:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874669235943936 02000000:00000001:3.0:1787827034.672124:0:15155:0:(sec.c:2469:sptlrpc_svc_unwrap_request()) Process entered 00000100:00000001:3.0:1787827034.672137:0:15155:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1787827034.672140:0:15155:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1787827034.672145:0:15155:0:(sec.c:2518:sptlrpc_svc_unwrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1787827034.672149:0:15155:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874669235943936 00000020:00000001:3.0:1787827034.672151:0:15155:0:(genops.c:792:class_conn2export()) Process entered 00000020:00000040:3.0:1787827034.672153:0:15155:0:(genops.c:804:class_conn2export()) looking for export cookie 0xb3f83dad1b7dcd6d 00000020:00000001:3.0:1787827034.672155:0:15155:0:(lustre_handles.c:130:class_handle2object()) Process entered 00000020:00000040:3.0:1787827034.672158:0:15155:0:(lustre_handles.c:148:class_handle2object()) GET export ffff98561118c800 refcount=3 00000020:00000001:3.0:1787827034.672161:0:15155:0:(lustre_handles.c:155:class_handle2object()) Process leaving (rc=18446630094154287104 : -113979555264512 : ffff98561118c800) 00000020:00000001:3.0:1787827034.672162:0:15155:0:(genops.c:806:class_conn2export()) Process leaving (rc=18446630094154287104 : -113979555264512 : ffff98561118c800) 00000100:00000001:3.0:1787827034.672166:0:15155:0:(service.c:1267:ptlrpc_update_export_timer()) Process entered 00000020:00000010:3.0:1787827034.672170:0:15155:0:(genops.c:1055:obd_export_timed_init()) kmalloced '(*data)': 48 at ffff98560919b880. 00000100:00000001:3.0:1787827034.672172:0:15155:0:(service.c:1289:ptlrpc_update_export_timer()) Process leaving via err (rc=0 : 0 : 0x0) 00000020:00000010:3.0:1787827034.672174:0:15155:0:(genops.c:1063:obd_export_timed_fini()) kfreed '*data': 48 at ffff98560919b880. 00000020:00000010:3.0:1787827034.672180:0:15155:0:(lu_object.c:1758:keys_init()) kmalloced '(ctx->lc_value)': 320 at ffff98561288d000. 00000020:00000010:3.0:1787827034.672185:0:15155:0:(lu_ucred.c:27:lu_ucred_key_init()) kmalloced '(value)': 128 at ffff985611033100. 00000020:00000010:3.0:1787827034.672189:0:15155:0:(tgt_main.c:758:tgt_ses_key_init()) slab-alloced 'session': 200 at ffff985609d7ebb8. 00000100:00000040:3.0:1787827034.672195:0:15155:0:(service.c:1432:ptlrpc_at_set_timer()) armed ldlm_cbd at +6s 00000100:00000001:3.0:1787827034.672198:0:15155:0:(service.c:2003:ptlrpc_server_request_add()) Process entered 00000100:00000001:3.0:1787827034.672199:0:15155:0:(service.c:1919:ptlrpc_server_hpreq_init()) Process entered 00000100:00000001:3.0:1787827034.672201:0:15155:0:(service.c:1949:ptlrpc_server_hpreq_init()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827034.672207:0:15155:0:(service.c:2101:ptlrpc_server_request_add()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827034.672221:0:15155:0:(service.c:2445:ptlrpc_server_handle_req_in()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1787827034.672229:0:15155:0:(service.c:2479:ptlrpc_server_handle_request()) Process entered 00000100:00000001:3.0:1787827034.672230:0:15155:0:(service.c:2223:ptlrpc_server_request_get()) Process entered 00000100:00100000:3.0:1787827034.672237:0:15155:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.114@tcp, seq: 256 00000100:00000040:3.0:1787827034.672239:0:15155:0:(service.c:2255:ptlrpc_server_request_get()) RPC GETting export ffff98561118c800 : new rpc_count 1 00000100:00000001:3.0:1787827034.672241:0:15155:0:(service.c:2257:ptlrpc_server_request_get()) Process leaving (rc=18446630094203879424 : -113979505672192 : ffff9856140d8000) 00000100:00000040:3.0:1787827034.672252:0:15155:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from New to Interpret req@ffff9856140d8000 x1874669235943936/t0(0) o104->3c946890-15ca-497a-b838-fe1cba61f4dd@192.168.203.114@tcp:635/0 lens 328/0 e 0 to 0 dl 1787827045 ref 1 fl New:/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1787827034.672263:0:15155:0:(service.c:1267:ptlrpc_update_export_timer()) Process entered 00000020:00000010:3.0:1787827034.672266:0:15155:0:(genops.c:1055:obd_export_timed_init()) kmalloced '(*data)': 48 at ffff98560919b880. 00000100:00000001:3.0:1787827034.672268:0:15155:0:(service.c:1289:ptlrpc_update_export_timer()) Process leaving via err (rc=0 : 0 : 0x0) 00000020:00000010:3.0:1787827034.672271:0:15155:0:(genops.c:1063:obd_export_timed_fini()) kfreed '*data': 48 at ffff98560919b880. 00000100:00100000:3.0:1787827034.672275:0:15155:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9856140d8000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb01_000:3c946890-15ca-497a-b838-fe1cba61f4dd+3:111705:x1874669235943936:12345-192.168.203.114@tcp:104: 00000100:00000200:3.0:1787827034.672280:0:15155:0:(service.c:2563:ptlrpc_server_handle_request()) got req 1874669235943936 00010000:00000001:3.0:1787827034.672283:0:15155:0:(ldlm_lockd.c:2360:ldlm_callback_handler()) Process entered 00010000:00000001:3.0:1787827034.672290:0:15155:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00000020:00000001:3.0:1787827034.672292:0:15155:0:(lustre_handles.c:130:class_handle2object()) Process entered 00000020:00000040:3.0:1787827034.672294:0:15155:0:(lustre_handles.c:148:class_handle2object()) GET ldlm ffff985607f56e00 refcount=2 00000020:00000001:3.0:1787827034.672296:0:15155:0:(lustre_handles.c:155:class_handle2object()) Process leaving (rc=18446630094000975360 : -113979708576256 : ffff985607f56e00) 00010000:00000001:3.0:1787827034.672299:0:15155:0:(ldlm_lock.c:634:__ldlm_handle2lock()) Process leaving (rc=18446630094000975360 : -113979708576256 : ffff985607f56e00) 00010000:00000001:3.0:1787827034.672303:0:15155:0:(ldlm_lock.c:261:ldlm_lock_remove_from_lru_check()) Process entered 00010000:00000001:3.0:1787827034.672306:0:15155:0:(ldlm_lock.c:280:ldlm_lock_remove_from_lru_check()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1787827034.672311:0:15155:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.203.114@tcp lock: ffff985607f56e00/0xb3f83dad1b7e8d4c lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1400000000000 nid: local remote: 0x5aa6d504d91c6f21 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00000001:3.0:1787827034.672338:0:15155:0:(pack_generic.c:296:lustre_pack_reply_v2()) Process entered 02000000:00000001:3.0:1787827034.672341:0:15155:0:(sec.c:2540:sptlrpc_svc_alloc_rs()) Process entered 02000000:00000010:3.0:1787827034.672345:0:15155:0:(sec_null.c:305:null_alloc_rs()) kmalloced '(rs)': 520 at ffff985608493400. 02000000:00000001:3.0:1787827034.672348:0:15155:0:(sec.c:2577:sptlrpc_svc_alloc_rs()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827034.672351:0:15155:0:(pack_generic.c:331:lustre_pack_reply_v2()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1787827034.672355:0:15155:0:(ldlm_lib.c:3355:target_pack_pool_reply()) Process entered 00010000:00000001:3.0:1787827034.672358:0:15155:0:(ldlm_lib.c:3365:target_pack_pool_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1787827034.672362:0:15155:0:(import.c:2149:obd_at_measure()) add 1 to ffff985606a709e8 time=62 v=5 (1 1 1 1) 00000100:00000001:3.0:1787827034.672367:0:15155:0:(connection.c:156:ptlrpc_connection_addref()) Process entered 00000100:00000040:3.0:1787827034.672370:0:15155:0:(connection.c:161:ptlrpc_connection_addref()) conn=ffff98561201f640 refcount 16 to 192.168.203.114@tcp 00000100:00000001:3.0:1787827034.672372:0:15155:0:(connection.c:163:ptlrpc_connection_addref()) Process leaving (rc=18446630094169568832 : -113979539982784 : ffff98561201f640) 02000000:00000001:3.0:1787827034.672376:0:15155:0:(sec.c:2591:sptlrpc_svc_wrap_reply()) Process entered 02000000:00000001:3.0:1787827034.672378:0:15155:0:(sec.c:2602:sptlrpc_svc_wrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827034.672381:0:15155:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1787827034.672383:0:15155:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.203.114@tcp 00000400:00000010:3.0:1787827034.672388:0:15155:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985609573aa0. 00000100:00000200:3.0:1787827034.672394:0:15155:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 16, xid 1874669235943936, offset 224 00000400:00000200:3.0:1787827034.672402:0:15155:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.203.114@tcp 00000400:00000200:3.0:1787827034.672411:0:15155:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:3.0:1787827034.672416:0:15155:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source Specified: 192.168.203.14@tcp to MR: 192.168.203.114@tcp local destination 00000400:00000200:3.0:1787827034.672421:0:15155:0:(lib-move.c:1745:lnet_handle_send()) 192.168.203.14@tcp NI seq info: [21890:21890:256:4294967295] 192.168.203.114@tcp LPNI seq info [21890:21890:8:4294967295] 00000400:00000200:3.0:1787827034.672424:0:15155:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:3.0:1787827034.672437:0:15155:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp:192.168.203.14@tcp) -> 192.168.203.114@tcp(192.168.203.114@tcp:192.168.203.114@tcp) : PUT try# 0 00000800:00000200:3.0:1787827034.672441:0:15155:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.203.114@tcp 00000800:00000010:3.0:1787827034.672446:0:15155:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff985609025400. 00000800:00000200:3.0:1787827034.672453:0:15155:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff985606751480] -> 12345-192.168.203.114@tcp (5) 00000800:00000200:3.0:1787827034.672460:0:15155:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.203.114@tcp ip 192.168.203.114:988 00000800:00000200:3.0:1787827034.672465:0:15155:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff985609025400 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1787827034.672479:0:15155:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1787827034.672482:0:15155:0:(lustre_net.h:1965:ptlrpc_connection_put()) PUT conn=ffff98561201f640 refcount 15 to 192.168.203.114@tcp 00010000:00000001:3.0:1787827034.672486:0:15155:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000010:3.0:1787827034.672490:0:15155:0:(ldlm_lockd.c:2249:ldlm_bl_to_thread()) kmalloced '(blwi)': 192 at ffff985610f13a80. 00010000:00000001:3.0:1787827034.672493:0:15155:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) Process entered 00010000:00010000:3.0:1787827034.672503:0:15155:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00010000:00000001:3.0:1787827034.672505:0:15155:0:(ldlm_lockd.c:2198:__ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1787827034.672506:0:15155:0:(ldlm_lockd.c:2254:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1787827034.672508:0:15155:0:(ldlm_lockd.c:2533:ldlm_callback_handler()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1787827034.672512:0:15155:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9856140d8000 x1874669235943936/t0(0) o104->3c946890-15ca-497a-b838-fe1cba61f4dd@192.168.203.114@tcp:635/0 lens 328/224 e 0 to 0 dl 1787827045 ref 1 fl Interpret:/0/0 rc 0/0 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827034.672520:0:15155:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9856140d8000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb01_000:3c946890-15ca-497a-b838-fe1cba61f4dd+3:111705:x1874669235943936:12345-192.168.203.114@tcp:104: Request processed in 256us (524us total) trans 0 rc 0/0 00000100:00100000:3.0:1787827034.672528:0:15155:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.114@tcp, seq: 256 00000100:00000040:3.0:1787827034.672531:0:15155:0:(service.c:1113:ptlrpc_server_finish_active_request()) RPC PUTting export ffff98561118c800 : new rpc_count 0 00000100:00000001:3.0:1787827034.672534:0:15155:0:(service.c:1955:ptlrpc_server_hpreq_fini()) Process entered 00000100:00000001:3.0:1787827034.672542:0:15155:0:(service.c:1966:ptlrpc_server_hpreq_fini()) Process leaving 00000020:00000010:3.0:1787827034.672546:0:15155:0:(lu_ucred.c:27:lu_ucred_key_fini()) kfreed 'info': 128 at ffff985611033100. 00000020:00000001:1.0:1787827034.672547:0:33341:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1787827034.672552:0:33341:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00000020:00000010:3.0:1787827034.672552:0:15155:0:(tgt_main.c:770:tgt_ses_key_fini()) slab-freed '(session)': 200 at ffff985609d7ebb8. 00010000:00010000:1.0:1787827034.672558:0:33341:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00000020:00000010:3.0:1787827034.672559:0:15155:0:(lu_object.c:1686:keys_fini()) kfreed 'ctx->lc_value': 320 at ffff98561288d000. 00000020:00000040:3.0:1787827034.672565:0:15155:0:(genops.c:879:class_export_put()) PUTting export ffff98561118c800 : new refcount 2 00000100:00000001:3.0:1787827034.672568:0:15155:0:(service.c:2635:ptlrpc_server_handle_request()) Process leaving (rc=1 : 1 : 1) 00010000:00000001:1.0:1787827034.672574:0:33341:0:(ldlm_lockd.c:2881:ldlm_bl_thread_blwi()) Process entered 00010000:00000001:1.0:1787827034.672577:0:33341:0:(ldlm_lockd.c:1905:ldlm_handle_bl_callback()) Process entered 00010000:00010000:1.0:1787827034.672581:0:33341:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.114@tcp lock: ffff985607f56e00/0xb3f83dad1b7e8d4c lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1400000000000 nid: local remote: 0x5aa6d504d91c6f21 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827034.672591:0:33341:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985607f56e00 already unused, calling callback (ffffffffc1543c20) 10000000:00000001:1.0:1787827034.672594:0:33341:0:(mgc_request.c:917:mgc_blocking_ast()) Process entered 10000000:00010000:1.0:1787827034.672598:0:33341:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.114@tcp lock: ffff985607f56e00/0xb3f83dad1b7e8d4c lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1400400000000 nid: local remote: 0x5aa6d504d91c6f21 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00000001:1.0:1787827034.672605:0:33341:0:(ldlm_request.c:1770:ldlm_cli_cancel()) Process entered 00010000:00000001:1.0:1787827034.672607:0:33341:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00000020:00000001:1.0:1787827034.672609:0:33341:0:(lustre_handles.c:130:class_handle2object()) Process entered 00000020:00000040:1.0:1787827034.672610:0:33341:0:(lustre_handles.c:148:class_handle2object()) GET ldlm ffff985607f56e00 refcount=3 00000020:00000001:1.0:1787827034.672612:0:33341:0:(lustre_handles.c:155:class_handle2object()) Process leaving (rc=18446630094000975360 : -113979708576256 : ffff985607f56e00) 00010000:00000001:1.0:1787827034.672615:0:33341:0:(ldlm_lock.c:634:__ldlm_handle2lock()) Process leaving (rc=18446630094000975360 : -113979708576256 : ffff985607f56e00) 00010000:00000001:1.0:1787827034.672618:0:33341:0:(ldlm_request.c:1386:ldlm_cli_cancel_local()) Process entered 00010000:00010000:1.0:1787827034.672621:0:33341:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.114@tcp lock: ffff985607f56e00/0xb3f83dad1b7e8d4c lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1408400000000 nid: local remote: 0x5aa6d504d91c6f21 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000800:00000200:2.0:1787827034.672624:0:2378:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 10000000:00000001:1.0:1787827034.672631:0:33341:0:(mgc_request.c:917:mgc_blocking_ast()) Process entered 00000800:00000010:2.0:1787827034.672631:0:2378:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff985609025400. 10000000:00010000:1.0:1787827034.672637:0:33341:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.114@tcp lock: ffff985607f56e00/0xb3f83dad1b7e8d4c lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1409400000000 nid: local remote: 0x5aa6d504d91c6f21 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000400:00000200:2.0:1787827034.672640:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 10000000:01000000:1.0:1787827034.672647:0:33341:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x2:0x0].0x0 (lustre) 00000400:00000200:2.0:1787827034.672649:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 10000000:00000001:1.0:1787827034.672654:0:33341:0:(mgc_request.c:719:mgc_requeue_add()) Process entered 00000400:00000200:2.0:1787827034.672657:0:2378:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985609573aa0 10000000:00000040:1.0:1787827034.672658:0:33341:0:(mgc_request.c:723:mgc_requeue_add()) log lustre-cliir: requeue (r=2 sp=0 st=1) 10000000:00000040:1.0:1787827034.672661:0:33341:0:(mgc_request.c:726:mgc_requeue_add()) lockh 0x0 00010000:00000001:1.0:1787827034.672663:0:33341:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00010000:00000001:1.0:1787827034.672665:0:33341:0:(ldlm_lock.c:617:__ldlm_handle2lock()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1787827034.672677:0:2378:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985609573aa0. 00000100:00000001:2.0:1787827034.672682:0:2378:0:(events.c:386:reply_out_callback()) Process entered 10000000:00000001:1.0:1787827034.672684:0:33341:0:(mgc_request.c:742:mgc_requeue_add()) Process leaving 00000100:00000001:2.0:1787827034.672685:0:2378:0:(events.c:398:reply_out_callback()) Process leaving 10000000:00000001:1.0:1787827034.672688:0:33341:0:(mgc_request.c:117:config_log_put()) Process entered 10000000:00000040:1.0:1787827034.672730:0:33341:0:(mgc_request.c:123:config_log_put()) log lustre-cliir(ffff985602e53600) refs 2 10000000:00000001:1.0:1787827034.672733:0:33341:0:(mgc_request.c:147:config_log_put()) Process leaving 10000000:00000001:1.0:1787827034.672735:0:33341:0:(mgc_request.c:967:mgc_blocking_ast()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1787827034.672738:0:33341:0:(ldlm_lock.c:2501:ldlm_lock_cancel()) Process entered 00010000:00000001:1.0:1787827034.672740:0:33341:0:(ldlm_lock.c:410:ldlm_lock_destroy_nolock()) Process entered 00010000:00000001:1.0:1787827034.672742:0:33341:0:(ldlm_lock.c:354:ldlm_lock_destroy_internal()) Process entered 00010000:00000001:1.0:1787827034.672756:0:33341:0:(ldlm_lock.c:261:ldlm_lock_remove_from_lru_check()) Process entered 00010000:00000001:1.0:1787827034.672758:0:33341:0:(ldlm_lock.c:280:ldlm_lock_remove_from_lru_check()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:1.0:1787827034.672761:0:33341:0:(lustre_handles.c:107:class_handle_unhash_nolock()) removing object ffff985607f56e00 with handle 0xb3f83dad1b7e8d4c from hash 00010000:00000001:1.0:1787827034.672764:0:33341:0:(ldlm_lock.c:384:ldlm_lock_destroy_internal()) Process leaving 00010000:00000001:1.0:1787827034.672765:0:33341:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:1.0:1787827034.672767:0:33341:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:1.0:1787827034.672768:0:33341:0:(ldlm_lock.c:416:ldlm_lock_destroy_nolock()) Process leaving 00010000:00000001:1.0:1787827034.672770:0:33341:0:(ldlm_lock.c:2537:ldlm_lock_cancel()) Process leaving 00010000:00000001:1.0:1787827034.672772:0:33341:0:(ldlm_request.c:1422:ldlm_cli_cancel_local()) Process leaving (rc=70368744177664 : 70368744177664 : 400000000000) 00010000:00000001:1.0:1787827034.672775:0:33341:0:(ldlm_request.c:2456:ldlm_cli_cancel_list()) Process entered 00010000:00000001:1.0:1787827034.672777:0:33341:0:(ldlm_request.c:1527:ldlm_cli_cancel_req()) Process entered 00000100:00000010:1.0:1787827034.672784:0:33341:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 904 at ffff985607e14f00. 00000020:00000040:1.0:1787827034.672787:0:33341:0:(genops.c:1210:class_import_get()) import ffff98561118e000 refcount=4 obd=MGC192.168.203.114@tcp 00000100:00000001:1.0:1787827034.672792:0:33341:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1787827034.672793:0:33341:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1787827034.672798:0:33341:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1787827034.672803:0:33341:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 512 at ffff985610db8400. 00000100:00000001:1.0:1787827034.672808:0:33341:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1787827034.672811:0:33341:0:(ldlm_request.c:1461:ldlm_cancel_pack()) Process entered 00010000:00010000:1.0:1787827034.672815:0:33341:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.203.114@tcp lock: ffff985607f56e00/0xb3f83dad1b7e8d4c lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5c09400000020 nid: local remote: 0x5aa6d504d91c6f21 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827034.672821:0:33341:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00000001:1.0:1787827034.672823:0:33341:0:(ldlm_request.c:1500:ldlm_cancel_pack()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787827034.672827:0:33341:0:(jobid.c:939:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1787827034.672832:0:33341:0:(jobid.c:979:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1787827034.672839:0:33341:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000073e61c8e] to pc [ptlrpcd_00_00+0] req@ffff985607e14f00 x1874669228032512/t0(0) o103->MGC192.168.203.114@tcp@192.168.203.114@tcp:17/18 lens 328/224 e 0 to 0 dl 0 ref 1 fl New:QU/200/ffffffff rc 0/-1 job:'ldlm_bl.0' uid:0 gid:0 projid:4294967295 00010000:00000001:1.0:1787827034.672856:0:33341:0:(ldlm_request.c:1595:ldlm_cli_cancel_req()) Process leaving via out (rc=0 : 0 : 0x0) 00010000:00000001:1.0:1787827034.672859:0:33341:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:1.0:1787827034.672860:0:33341:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:1.0:1787827034.672862:0:33341:0:(ldlm_request.c:2504:ldlm_cli_cancel_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1787827034.672864:0:33341:0:(ldlm_request.c:1834:ldlm_cli_cancel()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:1.0:1787827034.672866:0:33341:0:(mgc_request.c:967:mgc_blocking_ast()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1787827034.672870:0:33341:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.114@tcp lock: ffff985607f56e00/0xb3f83dad1b7e8d4c lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5c09400000020 nid: local remote: 0x5aa6d504d91c6f21 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00000001:1.0:1787827034.672877:0:33341:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00010000:1.0:1787827034.672880:0:33341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.114@tcp lock: ffff985607f56e00/0xb3f83dad1b7e8d4c lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5c09400000020 nid: local remote: 0x5aa6d504d91c6f21 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00000040:1.0:1787827034.672888:0:33341:0:(ldlm_resource.c:1704:ldlm_resource_putref()) putref res: ffff98560568f600 count: 0 00010000:00000001:1.0:1787827034.672901:0:33341:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:1.0:1787827034.672904:0:33341:0:(ldlm_lockd.c:1935:ldlm_handle_bl_callback()) Process leaving 00010000:00000010:1.0:1787827034.672906:0:33341:0:(ldlm_lockd.c:2919:ldlm_bl_thread_blwi()) kfreed 'blwi': 192 at ffff985610f13a80. 00010000:00000001:1.0:1787827034.672910:0:33341:0:(ldlm_lockd.c:2923:ldlm_bl_thread_blwi()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1787827034.672913:0:33341:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1787827034.672915:0:33341:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1787827034.672917:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000001:1.0:1787827034.672919:0:33341:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1787827034.672920:0:33341:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1787827034.672924:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000001:1.0:1787827034.672927:0:33341:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1787827034.672930:0:33341:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1787827034.672932:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000001:0.0:1787827034.672935:0:2383:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1787827034.672940:0:2383:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1787827034.672943:0:2383:0:(client.c:1813:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1787827034.672951:0:2383:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff985607e14f00 x1874669228032512/t0(0) o103->MGC192.168.203.114@tcp@192.168.203.114@tcp:17/18 lens 328/224 e 0 to 0 dl 0 ref 1 fl New:QU/200/ffffffff rc 0/-1 job:'ldlm_bl.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827034.672956:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1787827034.672964:0:2383:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1787827034.672967:0:2383:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1787827034.672970:0:2383:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1787827034.672973:0:2383:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1787827034.672975:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1787827034.672976:0:2383:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1787827034.672982:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607e14f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2383:1874669228032512:192.168.203.114@tcp:103:ldlm_bl.0 00000100:00000001:0.0:1787827034.672988:0:2383:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1787827034.672993:0:2383:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1787827034.672995:0:2383:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1787827034.672998:0:2383:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1787827034.673002:0:2383:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff985607f54e00. 02000000:00000001:0.0:1787827034.673004:0:2383:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1787827034.673009:0:2383:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000010:0.0:1787827034.673013:0:2383:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9856094acd10. 00000400:00000010:0.0:1787827034.673018:0:2383:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985610d30880. 00000100:00000200:0.0:1787827034.673024:0:2383:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1874669228032512, portal 18 00000100:00000001:0.0:1787827034.673026:0:2383:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1787827034.673028:0:2383:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446630093999656704 : -113979709894912 : ffff985607e14f00) 00000100:00000040:0.0:1787827034.673034:0:2383:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff985607e14f00 x1874669228032512/t0(0) o103->MGC192.168.203.114@tcp@192.168.203.114@tcp:17/18 lens 328/224 e 0 to 0 dl 1787827050 ref 2 fl Rpc:r/200/ffffffff rc 0/-1 job:'ldlm_bl.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787827034.673041:0:2383:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1787827034.673043:0:2383:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.203.114@tcp 00000400:00000010:0.0:1787827034.673045:0:2383:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985610d30d48. 00000100:00000200:0.0:1787827034.673049:0:2383:0:(niobuf.c:94:ptl_send_buf()) Sending 328 bytes to portal 17, xid 1874669228032512, offset 0 00000400:00000200:0.0:1787827034.673060:0:2383:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.203.114@tcp 00000400:00000200:0.0:1787827034.673065:0:2383:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:0.0:1787827034.673069:0:2383:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.203.114@tcp local destination 00000400:00000200:0.0:1787827034.673073:0:2383:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1787827034.673077:0:2383:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.203.14@tcp 00000400:00000200:0.0:1787827034.673081:0:2383:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.203.14@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1787827034.673084:0:2383:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.203.114@tcp 00000400:00000200:0.0:1787827034.673087:0:2383:0:(lib-move.c:1745:lnet_handle_send()) 192.168.203.14@tcp NI seq info: [21891:21891:256:4294967295] 192.168.203.114@tcp LPNI seq info [21891:21891:8:4294967295] 00000400:00000200:0.0:1787827034.673091:0:2383:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:0.0:1787827034.673098:0:2383:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp:) -> 192.168.203.114@tcp(192.168.203.114@tcp:192.168.203.114@tcp) : PUT try# 0 00000800:00000200:0.0:1787827034.673102:0:2383:0:(socklnd_cb.c:931:ksocknal_send()) sending 328 bytes in 1 frags to 12345-192.168.203.114@tcp 00000800:00000010:0.0:1787827034.673106:0:2383:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9856030d2900. 00000800:00000200:0.0:1787827034.673110:0:2383:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff985606751480] -> 12345-192.168.203.114@tcp (5) 00000800:00000200:0.0:1787827034.673116:0:2383:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.203.114@tcp ip 192.168.203.114:988 00000800:00000200:0.0:1787827034.673119:0:2383:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9856030d2900 type 1, nob 424 niov 1 nkiov 1 00000100:00000001:0.0:1787827034.673136:0:2383:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1787827034.673141:0:2383:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000001:0.0:1787827034.673143:0:2383:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.203.114@tcp: PM QoS new deadline estimation for cpu 2 is 4297862712 00000100:00000001:0.0:1787827034.673149:0:2383:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.203.114@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4297860348 00000100:00000001:0.0:1787827034.673153:0:2383:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.203.114@tcp: PM QoS new deadline estimation for cpu 3 is 4297862712 00000100:00000001:0.0:1787827034.673156:0:2383:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.203.114@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4297860348 00000100:00000001:0.0:1787827034.673159:0:2383:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1787827034.673162:0:2383:0:(client.c:1941:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787827034.673165:0:2383:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787827034.673167:0:2383:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1787827034.673172:0:2383:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1787827034.673174:0:2383:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1787827034.673177:0:2383:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1787827034.673178:0:2383:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1787827034.673180:0:2383:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787827034.673182:0:2383:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1787827034.673233:0:2377:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1787827034.673237:0:2377:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9856030d2900. 00000400:00000200:2.0:1787827034.673240:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1787827034.673244:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000400:00000200:2.0:1787827034.673248:0:2377:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985610d30d48 00000400:00000010:2.0:1787827034.673250:0:2377:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985610d30d48. 00000100:00000001:2.0:1787827034.673263:0:2377:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1787827034.673269:0:2377:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff985607e14f00 x1874669228032512/t0(0) o103->MGC192.168.203.114@tcp@192.168.203.114@tcp:17/18 lens 328/224 e 0 to 0 dl 1787827050 ref 2 fl Rpc:r/200/ffffffff rc 0/-1 job:'ldlm_bl.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827034.673279:0:2377:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827034.673282:0:2377:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff985607e14f00 x1874669228032512/t0(0) o103->MGC192.168.203.114@tcp@192.168.203.114@tcp:17/18 lens 328/224 e 0 to 0 dl 1787827050 ref 2 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'ldlm_bl.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827034.673289:0:2377:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1787827034.673290:0:2377:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1787827034.674714:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827034.674727:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1787827034.674730:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827034.674732:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827034.674737:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827034.674743:0:2378:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp) <- 192.168.203.114@tcp : PUT - for me 00000400:00000200:2.0:1787827034.674757:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827034.674761:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827034.674764:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827034.674768:0:2378:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.203.114@tcp of length 224 into portal 18 MB=0x6a90071688200 00000400:00000200:2.0:1787827034.674772:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827034.674776:0:2378:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 12 from 12345-192.168.203.114@tcp of length 224/224 into md 0x32325 [1] + 224 00000800:00000001:2.0:1787827034.674781:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827034.674788:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827034.674790:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1787827034.674792:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000100:00000001:2.0:1787827034.674798:0:2378:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1787827034.674803:0:2378:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff985607e14f00 x1874669228032512/t0(0) o103->MGC192.168.203.114@tcp@192.168.203.114@tcp:17/18 lens 328/224 e 0 to 0 dl 1787827050 ref 1 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'ldlm_bl.0' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1787827034.674811:0:2378:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff985607e14f00 x1874669228032512/t0(0) o103->MGC192.168.203.114@tcp@192.168.203.114@tcp:17/18 lens 328/224 e 0 to 0 dl 1787827050 ref 1 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'ldlm_bl.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827034.674826:0:2378:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1787827034.674831:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827034.674834:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1787827034.674886:0:2383:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1787827034.674890:0:2383:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1787827034.674895:0:2383:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1787827034.674897:0:2383:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1787827034.674900:0:2383:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9856094acd10. 00000400:00000200:0.0:1787827034.674903:0:2383:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985610d30880 00000400:00000010:0.0:1787827034.674906:0:2383:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985610d30880. 00000100:00000001:0.0:1787827034.674909:0:2383:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1787827034.674914:0:2383:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff985607e14f00 x1874669228032512/t0(0) o103->MGC192.168.203.114@tcp@192.168.203.114@tcp:17/18 lens 328/224 e 0 to 0 dl 1787827050 ref 1 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'ldlm_bl.0' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1787827034.674923:0:2383:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff985607e14f00 x1874669228032512/t0(0) o103->MGC192.168.203.114@tcp@192.168.203.114@tcp:17/18 lens 328/224 e 0 to 0 dl 1787827050 ref 1 fl Rpc:RQU/200/ffffffff rc 0/-1 job:'ldlm_bl.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787827034.674930:0:2383:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1787827034.674931:0:2383:0:(client.c:3001:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1787827034.674934:0:2383:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:0.0:1787827034.674936:0:2383:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1787827034.674938:0:2383:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1787827034.674940:0:2383:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1787827034.674944:0:2383:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1787827034.674949:0:2383:0:(import.c:2149:obd_at_measure()) add 5 to ffff98561118e4a8 time=962 v=5 (5 5 0 0) 00000100:00001000:0.0:1787827034.674953:0:2383:0:(import.c:2149:obd_at_measure()) add 1 to ffff98561118e418 time=49 v=5 (1 0 0 0) 00000100:00000001:0.0:1787827034.674956:0:2383:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1787827034.674958:0:2383:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827034.674960:0:2383:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1787827034.674962:0:2383:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787827034.674965:0:2383:0:(client.c:1797:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787827034.674970:0:2383:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff985607e14f00 x1874669228032512/t0(0) o103->MGC192.168.203.114@tcp@192.168.203.114@tcp:17/18 lens 328/224 e 0 to 0 dl 1787827050 ref 1 fl Rpc:RQU/200/0 rc 0/0 job:'ldlm_bl.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787827034.674977:0:2383:0:(client.c:2354:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1787827034.674980:0:2383:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1787827034.674982:0:2383:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1787827034.674986:0:2383:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff985607e14f00 x1874669228032512/t0(0) o103->MGC192.168.203.114@tcp@192.168.203.114@tcp:17/18 lens 328/224 e 0 to 0 dl 1787827050 ref 1 fl Interpret:RQU/200/0 rc 0/0 job:'ldlm_bl.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827034.674993:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607e14f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2383:1874669228032512:192.168.203.114@tcp:103:ldlm_bl.0 00000100:00000001:0.0:1787827034.674998:0:2383:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1787827034.675001:0:2383:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1787827034.675004:0:2383:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607e14f00 x1874669228032512/t0(0) o103->MGC192.168.203.114@tcp@192.168.203.114@tcp:17/18 lens 328/224 e 0 to 0 dl 1787827050 ref 1 fl Complete:RQU/200/0 rc 0/0 job:'ldlm_bl.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787827034.675010:0:2383:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1787827034.675013:0:2383:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1787827034.675016:0:2383:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff985607f54e00. 02000000:00000001:0.0:1787827034.675019:0:2383:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1787827034.675021:0:2383:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:0.0:1787827034.675023:0:2383:0:(genops.c:1223:class_import_put()) import ffff98561118e000 refcount=3 obd=MGC192.168.203.114@tcp 00000020:00000001:0.0:1787827034.675025:0:2383:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:0.0:1787827034.675028:0:2383:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985610db8400. 02000000:00000001:0.0:1787827034.675031:0:2383:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1787827034.675032:0:2383:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1787827034.675034:0:2383:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607e14f00. 00000100:00000001:0.0:1787827034.675037:0:2383:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1787827034.675038:0:2383:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1787827034.675040:0:2383:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1787827034.675045:0:2383:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1787827034.675048:0:2383:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787827034.675050:0:2383:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1787827034.675052:0:2383:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000010:1.1F:1787827034.678739:0:0:0:(ldlm_resource.c:1555:__ldlm_resource_free()) slab-freed '(res)': 256 at ffff98560568f600. 00010000:00000010:1.1:1787827034.678781:0:0:0:(ldlm_lock.c:174:lock_handle_free()) slab-freed 'lock': 512 at ffff985607f56e00. 00010000:00000001:0.0:1787827034.932873:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827034.932882:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827034.932891:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827034.932892:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827034.932896:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827034.932897:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827034.932900:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827034.932901:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1787827034.932942:0:15164:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1787827034.932946:0:15164:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1787827034.932950:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:0.0:1787827035.956833:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827035.956839:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827035.956843:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827035.956844:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827035.956847:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827035.956848:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827035.956851:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827035.956854:0:49842:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:0.0:1787827035.956856:0:49842:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:0.0:1787827035.956860:0:49842:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:0.0:1787827035.956863:0:49842:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:0.0:1787827035.956864:0:49842:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827035.956866:0:49842:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:0.0:1787827035.956868:0:49842:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827035.956870:0:49842:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827035.956871:0:49842:0:(ldlm_pool.c:496:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1787827035.956898:0:33342:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1787827035.956900:0:33342:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1787827035.956902:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:00000001:0.0:1787827036.404905:0:15166:0:(mgc_request.c:105:config_log_get()) Process entered 10000000:00000040:0.0:1787827036.404911:0:15166:0:(mgc_request.c:108:config_log_get()) log lustre-cliir (ffff985602e53600) refs 2 10000000:00000001:0.0:1787827036.404915:0:15166:0:(mgc_request.c:109:config_log_get()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:0.0:1787827036.404917:0:15166:0:(mgc_request.c:117:config_log_put()) Process entered 10000000:00000001:0.0:1787827036.404918:0:15166:0:(mgc_request.c:120:config_log_put()) Process leaving 10000000:00000001:0.0:1787827036.404919:0:15166:0:(mgc_request.c:601:do_requeue()) Process entered 10000000:01000000:0.0:1787827036.404922:0:15166:0:(mgc_request.c:611:do_requeue()) updating log lustre-cliir 10000000:00000001:0.0:1787827036.404923:0:15166:0:(mgc_request.c:1783:mgc_process_log()) Process entered 10000000:01000000:0.0:1787827036.404925:0:15166:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-cliir-ffff985602bc9800 from 208 10000000:00000001:0.0:1787827036.404927:0:15166:0:(mgc_request.c:990:mgc_enqueue()) Process entered 10000000:01000000:0.0:1787827036.404928:0:15166:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-cliir (res 0x65727473756c) 00000100:00000010:0.0:1787827036.404937:0:15166:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 904 at ffff985607f08000. 00000020:00000040:0.0:1787827036.404940:0:15166:0:(genops.c:1210:class_import_get()) import ffff98561118e000 refcount=4 obd=MGC192.168.203.114@tcp 00000100:00000001:0.0:1787827036.404944:0:15166:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1787827036.404946:0:15166:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1787827036.404951:0:15166:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1787827036.404958:0:15166:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 512 at ffff985607f55000. 00000100:00000001:0.0:1787827036.404965:0:15166:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827036.404969:0:15166:0:(ldlm_request.c:1064:ldlm_cli_enqueue()) Process entered 00010000:00000001:0.0:1787827036.404972:0:15166:0:(ldlm_lock.c:1700:ldlm_lock_create()) Process entered 00010000:00000010:0.0:1787827036.404979:0:15166:0:(ldlm_resource.c:1510:ldlm_resource_new()) slab-alloced 'res': 256 at ffff9856057bbd00. 00010000:00000001:0.0:1787827036.404984:0:15166:0:(ldlm_lock.c:433:ldlm_lock_new()) Process entered 00010000:00000010:0.0:1787827036.404987:0:15166:0:(ldlm_lock.c:437:ldlm_lock_new()) slab-alloced 'lock': 512 at ffff985610e07c00. 00000020:00000001:0.0:1787827036.404991:0:15166:0:(lustre_handles.c:61:class_handle_hash()) Process entered 00000020:00000040:0.0:1787827036.404995:0:15166:0:(lustre_handles.c:93:class_handle_hash()) added object ffff985610e07c00 with handle 0xb3f83dad1b7eda5a to hash 00000020:00000001:0.0:1787827036.404997:0:15166:0:(lustre_handles.c:94:class_handle_hash()) Process leaving 00010000:00000001:0.0:1787827036.404999:0:15166:0:(ldlm_lock.c:488:ldlm_lock_new()) Process leaving (rc=18446630094150597632 : -113979558953984 : ffff985610e07c00) 00010000:00000001:0.0:1787827036.405002:0:15166:0:(ldlm_lock.c:1741:ldlm_lock_create()) Process leaving (rc=18446630094150597632 : -113979558953984 : ffff985610e07c00) 00010000:00000001:0.0:1787827036.405007:0:15166:0:(ldlm_lock.c:261:ldlm_lock_remove_from_lru_check()) Process entered 00010000:00000001:0.0:1787827036.405009:0:15166:0:(ldlm_lock.c:280:ldlm_lock_remove_from_lru_check()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1787827036.405013:0:15166:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.114@tcp lock: ffff985610e07c00/0xb3f83dad1b7eda5a lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827036.405020:0:15166:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.114@tcp lock: ffff985610e07c00/0xb3f83dad1b7eda5a lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827036.405029:0:15166:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.114@tcp lock: ffff985610e07c00/0xb3f83dad1b7eda5a lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00000001:0.0:1787827036.405033:0:15166:0:(client.c:3352:ptlrpc_queue_wait()) Process entered 00000100:00000001:0.0:1787827036.405034:0:15166:0:(client.c:1169:ptlrpc_prep_set()) Process entered 00000100:00000010:0.0:1787827036.405040:0:15166:0:(client.c:1171:ptlrpc_prep_set()) kmalloced '(set)': 144 at ffff98560869d9c0. 00000100:00000001:0.0:1787827036.405041:0:15166:0:(client.c:1186:ptlrpc_prep_set()) Process leaving (rc=18446630094008605120 : -113979700946496 : ffff98560869d9c0) 00000100:00000001:0.0:1787827036.405042:0:15166:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1787827036.405043:0:15166:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446630094000652288 : -113979708899328 : ffff985607f08000) 00000100:00000001:0.0:1787827036.405047:0:15166:0:(jobid.c:939:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1787827036.405050:0:15166:0:(jobid.c:979:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787827036.405052:0:15166:0:(client.c:2688:ptlrpc_set_wait()) Process entered 00000100:00000001:0.0:1787827036.405053:0:15166:0:(client.c:1813:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1787827036.405061:0:15166:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff985607f08000 x1874669228032640/t0(0) o101->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 328/344 e 0 to 0 dl 0 ref 2 fl New:QU/200/ffffffff rc 0/-1 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787827036.405067:0:15166:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1787827036.405069:0:15166:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1787827036.405071:0:15166:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1787827036.405073:0:15166:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1787827036.405074:0:15166:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1787827036.405077:0:15166:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607f08000 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669228032640:192.168.203.114@tcp:101:ll_cfg_requeue.0 00000100:00000001:0.0:1787827036.405080:0:15166:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1787827036.405084:0:15166:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1787827036.405086:0:15166:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1787827036.405087:0:15166:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1787827036.405091:0:15166:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 1024 at ffff985608544000. 02000000:00000001:0.0:1787827036.405092:0:15166:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1787827036.405097:0:15166:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000010:0.0:1787827036.405101:0:15166:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9856094acd10. 00000400:00000010:0.0:1787827036.405106:0:15166:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985610d30880. 00000100:00000200:0.0:1787827036.405112:0:15166:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 1024 bytes, xid 1874669228032640, portal 25 00000100:00000001:0.0:1787827036.405113:0:15166:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1787827036.405114:0:15166:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446630094000652288 : -113979708899328 : ffff985607f08000) 00000100:00000040:0.0:1787827036.405117:0:15166:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff985607f08000 x1874669228032640/t0(0) o101->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 328/344 e 0 to 0 dl 1787827052 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787827036.405123:0:15166:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1787827036.405125:0:15166:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.203.114@tcp 00000400:00000010:0.0:1787827036.405127:0:15166:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985610d30000. 00000100:00000200:0.0:1787827036.405129:0:15166:0:(niobuf.c:94:ptl_send_buf()) Sending 328 bytes to portal 26, xid 1874669228032640, offset 0 00000400:00000200:0.0:1787827036.405134:0:15166:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.203.114@tcp 00000400:00000200:0.0:1787827036.405141:0:15166:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:0.0:1787827036.405146:0:15166:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.203.114@tcp local destination 00000400:00000200:0.0:1787827036.405151:0:15166:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1787827036.405154:0:15166:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.203.14@tcp 00000400:00000200:0.0:1787827036.405158:0:15166:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.203.14@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1787827036.405161:0:15166:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.203.114@tcp 00000400:00000200:0.0:1787827036.405165:0:15166:0:(lib-move.c:1745:lnet_handle_send()) 192.168.203.14@tcp NI seq info: [21892:21892:256:4294967295] 192.168.203.114@tcp LPNI seq info [21892:21892:8:4294967295] 00000400:00000200:0.0:1787827036.405169:0:15166:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:0.0:1787827036.405179:0:15166:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp:) -> 192.168.203.114@tcp(192.168.203.114@tcp:192.168.203.114@tcp) : PUT try# 0 00000800:00000200:0.0:1787827036.405184:0:15166:0:(socklnd_cb.c:931:ksocknal_send()) sending 328 bytes in 1 frags to 12345-192.168.203.114@tcp 00000800:00000010:0.0:1787827036.405190:0:15166:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9856030d2900. 00000800:00000200:0.0:1787827036.405197:0:15166:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff985606751480] -> 12345-192.168.203.114@tcp (5) 00000800:00000200:0.0:1787827036.405205:0:15166:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.203.114@tcp ip 192.168.203.114:988 00000800:00000200:0.0:1787827036.405209:0:15166:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9856030d2900 type 1, nob 424 niov 1 nkiov 1 00000100:00000001:0.0:1787827036.405228:0:15166:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1787827036.405232:0:15166:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000001:0.0:1787827036.405235:0:15166:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.203.114@tcp: PM QoS new deadline estimation for cpu 2 is 4297864444 00000100:00000001:0.0:1787827036.405242:0:15166:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.203.114@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4297862712 00000100:00000001:0.0:1787827036.405248:0:15166:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.203.114@tcp: PM QoS new deadline estimation for cpu 3 is 4297864444 00000100:00000001:0.0:1787827036.405252:0:15166:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.203.114@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4297862712 00000100:00000001:0.0:1787827036.405255:0:15166:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1787827036.405258:0:15166:0:(client.c:1941:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787827036.405261:0:15166:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1787827036.405263:0:15166:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00100000:0.0:1787827036.405266:0:15166:0:(client.c:2715:ptlrpc_set_wait()) set ffff98560869d9c0 going to sleep for 11 seconds 00000100:00000001:0.0:1787827036.405269:0:15166:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1787827036.405272:0:15166:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1787827036.405385:0:2377:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1787827036.405390:0:2377:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9856030d2900. 00000400:00000200:2.0:1787827036.405397:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1787827036.405403:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000400:00000200:2.0:1787827036.405409:0:2377:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985610d30000 00000400:00000010:2.0:1787827036.405411:0:2377:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985610d30000. 00000100:00000001:2.0:1787827036.405415:0:2377:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1787827036.405420:0:2377:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff985607f08000 x1874669228032640/t0(0) o101->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 328/344 e 0 to 0 dl 1787827052 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827036.405430:0:2377:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827036.405434:0:2377:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 2 req@ffff985607f08000 x1874669228032640/t0(0) o101->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 328/344 e 0 to 0 dl 1787827052 ref 3 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827036.405441:0:2377:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1787827036.405443:0:2377:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1787827036.406668:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827036.406684:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1787827036.406688:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827036.406802:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827036.406810:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827036.406816:0:2378:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp) <- 192.168.203.114@tcp : PUT - for me 00000400:00000200:2.0:1787827036.406823:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827036.406827:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827036.406831:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827036.406834:0:2378:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.203.114@tcp of length 344 into portal 25 MB=0x6a90071688280 00000400:00000200:2.0:1787827036.406839:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827036.406844:0:2378:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.203.114@tcp of length 344/344 into md 0x3232d [1] + 224 00000800:00000001:2.0:1787827036.406850:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827036.406857:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827036.406860:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1787827036.406863:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000100:00000001:2.0:1787827036.406868:0:2378:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1787827036.406873:0:2378:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff985607f08000 x1874669228032640/t0(0) o101->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 328/344 e 0 to 0 dl 1787827052 ref 2 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1787827036.406884:0:2378:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=344 offset=224 replen=344 req@ffff985607f08000 x1874669228032640/t0(0) o101->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 328/344 e 0 to 0 dl 1787827052 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827036.406898:0:2378:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1787827036.406905:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827036.406908:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1787827036.406933:0:15166:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1787827036.406939:0:15166:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1787827036.406941:0:15166:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1787827036.406943:0:15166:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9856094acd10. 00000400:00000200:0.0:1787827036.406946:0:15166:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985610d30880 00000400:00000010:0.0:1787827036.406948:0:15166:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985610d30880. 00000100:00000001:0.0:1787827036.406957:0:15166:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1787827036.406969:0:15166:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff985607f08000 x1874669228032640/t0(0) o101->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 328/344 e 0 to 0 dl 1787827052 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1787827036.406982:0:15166:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff985607f08000 x1874669228032640/t0(0) o101->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 328/344 e 0 to 0 dl 1787827052 ref 2 fl Rpc:RQU/200/ffffffff rc 0/-1 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787827036.406988:0:15166:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1787827036.406991:0:15166:0:(client.c:3001:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1787827036.406993:0:15166:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:0.0:1787827036.406996:0:15166:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1787827036.406999:0:15166:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1787827036.407001:0:15166:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1787827036.407003:0:15166:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1787827036.407010:0:15166:0:(import.c:2149:obd_at_measure()) add 5 to ffff98561118e460 time=36 v=5 (5 5 5 5) 00000100:00001000:0.0:1787827036.407014:0:15166:0:(import.c:2149:obd_at_measure()) add 1 to ffff98561118e418 time=51 v=5 (1 0 0 0) 00000100:00000001:0.0:1787827036.407017:0:15166:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1787827036.407018:0:15166:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827036.407021:0:15166:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1787827036.407024:0:15166:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787827036.407027:0:15166:0:(client.c:1797:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787827036.407031:0:15166:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff985607f08000 x1874669228032640/t0(0) o101->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 328/344 e 0 to 0 dl 1787827052 ref 2 fl Rpc:RQU/200/0 rc 0/0 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787827036.407036:0:15166:0:(client.c:2354:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1787827036.407038:0:15166:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1787827036.407040:0:15166:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1787827036.407042:0:15166:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff985607f08000 x1874669228032640/t0(0) o101->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 328/344 e 0 to 0 dl 1787827052 ref 2 fl Interpret:RQU/200/0 rc 0/0 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827036.407047:0:15166:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607f08000 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669228032640:192.168.203.114@tcp:101:ll_cfg_requeue.0 00000100:00000001:0.0:1787827036.407050:0:15166:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1787827036.407052:0:15166:0:(client.c:2779:ptlrpc_set_wait()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787827036.407053:0:15166:0:(client.c:1237:ptlrpc_set_destroy()) Process entered 00000100:00000001:0.0:1787827036.407055:0:15166:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1787827036.407057:0:15166:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff985607f08000 x1874669228032640/t0(0) o101->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 328/344 e 0 to 0 dl 1787827052 ref 2 fl Complete:RQU/200/0 rc 0/0 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787827036.407060:0:15166:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:0.0:1787827036.407062:0:15166:0:(client.c:3800:ptlrpc_reqset_free()) kfreed 'set': 144 at ffff98560869d9c0. 00000100:00000001:0.0:1787827036.407064:0:15166:0:(client.c:1274:ptlrpc_set_destroy()) Process leaving 00000100:00000001:0.0:1787827036.407065:0:15166:0:(client.c:3371:ptlrpc_queue_wait()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827036.407067:0:15166:0:(ldlm_request.c:661:ldlm_cli_enqueue_fini()) Process entered 00010000:00000001:0.0:1787827036.407070:0:15166:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00000020:00000001:0.0:1787827036.407072:0:15166:0:(lustre_handles.c:130:class_handle2object()) Process entered 00000020:00000040:0.0:1787827036.407074:0:15166:0:(lustre_handles.c:148:class_handle2object()) GET ldlm ffff985610e07c00 refcount=4 00000020:00000001:0.0:1787827036.407076:0:15166:0:(lustre_handles.c:155:class_handle2object()) Process leaving (rc=18446630094150597632 : -113979558953984 : ffff985610e07c00) 00010000:00000001:0.0:1787827036.407078:0:15166:0:(ldlm_lock.c:634:__ldlm_handle2lock()) Process leaving (rc=18446630094150597632 : -113979558953984 : ffff985610e07c00) 00010000:00000040:0.0:1787827036.407082:0:15166:0:(ldlm_request.c:746:ldlm_cli_enqueue_fini()) local: ffff985610e07c00, remote cookie: 0x5aa6d504d91df545, flags: 0x0 00010000:00000001:0.0:1787827036.407084:0:15166:0:(ldlm_lock.c:1795:ldlm_lock_enqueue()) Process entered 00010000:00000001:0.0:1787827036.407087:0:15166:0:(ldlm_lock.c:1125:ldlm_grant_lock()) Process entered 00010000:00000001:0.0:1787827036.407089:0:15166:0:(ldlm_lock.c:989:search_granted_lock()) Process entered 00010000:00000001:0.0:1787827036.407090:0:15166:0:(ldlm_lock.c:1055:search_granted_lock()) Process leaving 00010000:00000001:0.0:1787827036.407091:0:15166:0:(ldlm_lock.c:1067:ldlm_granted_list_add_lock()) Process entered 00010000:00000040:0.0:1787827036.407094:0:15166:0:(ldlm_resource.c:1876:ldlm_resource_dump()) --- Resource: [0x65727473756c:0x2:0x0].0x0 (ffff9856057bbd00) refcount = 1 00010000:00010000:0.0:1787827036.407098:0:15166:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.114@tcp lock: ffff985610e07c00/0xb3f83dad1b7eda5a lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x5aa6d504d91df545 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00000001:0.0:1787827036.407103:0:15166:0:(ldlm_lock.c:1094:ldlm_granted_list_add_lock()) Process leaving 00010000:00000001:0.0:1787827036.407104:0:15166:0:(ldlm_lock.c:1157:ldlm_grant_lock()) Process leaving 00010000:00000001:0.0:1787827036.407105:0:15166:0:(ldlm_lock.c:1914:ldlm_lock_enqueue()) Process leaving via out (rc=0 : 0 : 0x0) 00010000:00000001:0.0:1787827036.407108:0:15166:0:(ldlm_request.c:286:ldlm_completion_ast()) Process entered 00010000:00000001:0.0:1787827036.407109:0:15166:0:(ldlm_request.c:295:ldlm_completion_ast()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1787827036.407111:0:15166:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.114@tcp lock: ffff985610e07c00/0xb3f83dad1b7eda5a lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x5aa6d504d91df545 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00000001:0.0:1787827036.407115:0:15166:0:(ldlm_request.c:842:ldlm_cli_enqueue_fini()) Process leaving 00010000:00000001:0.0:1787827036.407116:0:15166:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:0.0:1787827036.407117:0:15166:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:0.0:1787827036.407118:0:15166:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:0.0:1787827036.407119:0:15166:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:0.0:1787827036.407120:0:15166:0:(ldlm_request.c:1218:ldlm_cli_enqueue()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787827036.407122:0:15166:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1787827036.407124:0:15166:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607f08000 x1874669228032640/t0(0) o101->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 328/344 e 0 to 0 dl 1787827052 ref 1 fl Complete:RQU/200/0 rc 0/0 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787827036.407127:0:15166:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1787827036.407129:0:15166:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1787827036.407130:0:15166:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985608544000. 02000000:00000001:0.0:1787827036.407132:0:15166:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1787827036.407134:0:15166:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:0.0:1787827036.407135:0:15166:0:(genops.c:1223:class_import_put()) import ffff98561118e000 refcount=3 obd=MGC192.168.203.114@tcp 00000020:00000001:0.0:1787827036.407136:0:15166:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:0.0:1787827036.407138:0:15166:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607f55000. 02000000:00000001:0.0:1787827036.407141:0:15166:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1787827036.407142:0:15166:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1787827036.407144:0:15166:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607f08000. 00000100:00000001:0.0:1787827036.407148:0:15166:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1787827036.407150:0:15166:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 10000000:00000001:0.0:1787827036.407152:0:15166:0:(mgc_request.c:1023:mgc_enqueue()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:0.0:1787827036.407155:0:15166:0:(mgc_request.c:105:config_log_get()) Process entered 10000000:00000040:0.0:1787827036.407157:0:15166:0:(mgc_request.c:108:config_log_get()) log lustre-cliir (ffff985602e53600) refs 3 10000000:00000001:0.0:1787827036.407160:0:15166:0:(mgc_request.c:109:config_log_get()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827036.407161:0:15166:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00000020:00000001:0.0:1787827036.407163:0:15166:0:(lustre_handles.c:130:class_handle2object()) Process entered 00000020:00000040:0.0:1787827036.407164:0:15166:0:(lustre_handles.c:148:class_handle2object()) GET ldlm ffff985610e07c00 refcount=3 00000020:00000001:0.0:1787827036.407166:0:15166:0:(lustre_handles.c:155:class_handle2object()) Process leaving (rc=18446630094150597632 : -113979558953984 : ffff985610e07c00) 00010000:00000001:0.0:1787827036.407168:0:15166:0:(ldlm_lock.c:634:__ldlm_handle2lock()) Process leaving (rc=18446630094150597632 : -113979558953984 : ffff985610e07c00) 00010000:00000001:0.0:1787827036.407170:0:15166:0:(ldlm_lock.c:2547:ldlm_lock_set_data()) Process entered 00010000:00000001:0.0:1787827036.407171:0:15166:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:0.0:1787827036.407172:0:15166:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:0.0:1787827036.407174:0:15166:0:(ldlm_lock.c:2556:ldlm_lock_set_data()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:0.0:1787827036.407176:0:15166:0:(mgc_request.c:1520:mgc_process_recover_log()) Process entered 10000000:00000010:0.0:1787827036.407180:0:15166:0:(mgc_request.c:1532:mgc_process_recover_log()) kmalloced '(folios)': 32 at ffff9856119542e0. 00000100:00000010:0.0:1787827036.407202:0:15166:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 904 at ffff985607f08000. 00000020:00000040:0.0:1787827036.407205:0:15166:0:(genops.c:1210:class_import_get()) import ffff98561118e000 refcount=4 obd=MGC192.168.203.114@tcp 00000100:00000001:0.0:1787827036.407208:0:15166:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1787827036.407211:0:15166:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1787827036.407214:0:15166:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1787827036.407219:0:15166:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 512 at ffff985607f55000. 00000100:00000001:0.0:1787827036.407223:0:15166:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787827036.407225:0:15166:0:(client.c:227:ptlrpc_prep_bulk_imp()) Process entered 00000100:00000010:0.0:1787827036.407228:0:15166:0:(client.c:169:ptlrpc_new_bulk()) kmalloced '(desc)': 984 at ffff985608544000. 00000100:00000010:0.0:1787827036.407230:0:15166:0:(client.c:174:ptlrpc_new_bulk()) kmalloced '(desc->bd_vec)': 64 at ffff9856111e4880. 00000020:00000040:0.0:1787827036.407231:0:15166:0:(genops.c:1210:class_import_get()) import ffff98561118e000 refcount=5 obd=MGC192.168.203.114@tcp 00000100:00000001:0.0:1787827036.407235:0:15166:0:(client.c:3352:ptlrpc_queue_wait()) Process entered 00000100:00000001:0.0:1787827036.407235:0:15166:0:(client.c:1169:ptlrpc_prep_set()) Process entered 00000100:00000010:0.0:1787827036.407238:0:15166:0:(client.c:1171:ptlrpc_prep_set()) kmalloced '(set)': 144 at ffff98560869d9c0. 00000100:00000001:0.0:1787827036.407240:0:15166:0:(client.c:1186:ptlrpc_prep_set()) Process leaving (rc=18446630094008605120 : -113979700946496 : ffff98560869d9c0) 00000100:00000001:0.0:1787827036.407241:0:15166:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1787827036.407242:0:15166:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446630094000652288 : -113979708899328 : ffff985607f08000) 00000100:00000001:0.0:1787827036.407245:0:15166:0:(jobid.c:939:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1787827036.407248:0:15166:0:(jobid.c:979:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787827036.407249:0:15166:0:(client.c:2688:ptlrpc_set_wait()) Process entered 00000100:00000001:0.0:1787827036.407250:0:15166:0:(client.c:1813:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1787827036.407253:0:15166:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff985607f08000 x1874669228032768/t0(0) o256->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 304/240 e 0 to 0 dl 0 ref 2 fl New:QU/200/ffffffff rc 0/-1 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787827036.407258:0:15166:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1787827036.407259:0:15166:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1787827036.407262:0:15166:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1787827036.407264:0:15166:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1787827036.407266:0:15166:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1787827036.407269:0:15166:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607f08000 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669228032768:192.168.203.114@tcp:256:ll_cfg_requeue.0 00000100:00000001:0.0:1787827036.407273:0:15166:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1787827036.407276:0:15166:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1787827036.407278:0:15166:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787827036.407281:0:15166:0:(niobuf.c:346:ptlrpc_register_bulk()) Process entered 00000400:00000200:0.0:1787827036.407285:0:15166:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000010:0.0:1787827036.407288:0:15166:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9856094acd10. 00000400:00000010:0.0:1787827036.407293:0:15166:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 184 at ffff98560869da80. 00000100:00000200:0.0:1787827036.407297:0:15166:0:(niobuf.c:453:ptlrpc_register_bulk()) Setup 1 bulk put-sink buffers: 4 pages 16384 bytes, mbits x0x6a90071688300-0x6a90071688300, portal 33 00000100:00000001:0.0:1787827036.407300:0:15166:0:(niobuf.c:455:ptlrpc_register_bulk()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1787827036.407303:0:15166:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1787827036.407305:0:15166:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff985607f55c00. 02000000:00000001:0.0:1787827036.407307:0:15166:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1787827036.407310:0:15166:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000010:0.0:1787827036.407312:0:15166:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9856094ac370. 00000400:00000010:0.0:1787827036.407315:0:15166:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985610d30880. 00000100:00000200:0.0:1787827036.407317:0:15166:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1874669228032768, portal 25 00000100:00000001:0.0:1787827036.407318:0:15166:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1787827036.407319:0:15166:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446630094000652288 : -113979708899328 : ffff985607f08000) 00000100:00000040:0.0:1787827036.407323:0:15166:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff985607f08000 x1874669228032768/t0(0) o256->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 304/240 e 0 to 0 dl 1787827052 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787827036.407328:0:15166:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1787827036.407329:0:15166:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.203.114@tcp 00000400:00000010:0.0:1787827036.407331:0:15166:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985610d30ee0. 00000100:00000200:0.0:1787827036.407333:0:15166:0:(niobuf.c:94:ptl_send_buf()) Sending 304 bytes to portal 26, xid 1874669228032768, offset 0 00000400:00000200:0.0:1787827036.407336:0:15166:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.203.114@tcp 00000400:00000200:0.0:1787827036.407341:0:15166:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:0.0:1787827036.407345:0:15166:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.203.114@tcp local destination 00000400:00000200:0.0:1787827036.407349:0:15166:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1787827036.407352:0:15166:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.203.14@tcp 00000400:00000200:0.0:1787827036.407355:0:15166:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.203.14@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1787827036.407357:0:15166:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.203.114@tcp 00000400:00000200:0.0:1787827036.407359:0:15166:0:(lib-move.c:1745:lnet_handle_send()) 192.168.203.14@tcp NI seq info: [21893:21893:256:4294967295] 192.168.203.114@tcp LPNI seq info [21893:21893:8:4294967295] 00000400:00000200:0.0:1787827036.407361:0:15166:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:0.0:1787827036.407367:0:15166:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp:) -> 192.168.203.114@tcp(192.168.203.114@tcp:192.168.203.114@tcp) : PUT try# 0 00000800:00000200:0.0:1787827036.407371:0:15166:0:(socklnd_cb.c:931:ksocknal_send()) sending 304 bytes in 1 frags to 12345-192.168.203.114@tcp 00000800:00000010:0.0:1787827036.407374:0:15166:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9856030d2e00. 00000800:00000200:0.0:1787827036.407379:0:15166:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff985606751480] -> 12345-192.168.203.114@tcp (5) 00000800:00000200:0.0:1787827036.407385:0:15166:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.203.114@tcp ip 192.168.203.114:988 00000800:00000200:0.0:1787827036.407389:0:15166:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9856030d2e00 type 1, nob 400 niov 1 nkiov 1 00000100:00000001:0.0:1787827036.407402:0:15166:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1787827036.407406:0:15166:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000001:0.0:1787827036.407408:0:15166:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.203.114@tcp: PM QoS new deadline estimation for cpu 2 is 4297864446 00000100:00000001:0.0:1787827036.407414:0:15166:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.203.114@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4297864444 00000100:00000001:0.0:1787827036.407418:0:15166:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.203.114@tcp: PM QoS new deadline estimation for cpu 3 is 4297864446 00000100:00000001:0.0:1787827036.407423:0:15166:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.203.114@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4297864444 00000100:00000001:0.0:1787827036.407427:0:15166:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1787827036.407430:0:15166:0:(client.c:1941:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787827036.407431:0:15166:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1787827036.407433:0:15166:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00100000:0.0:1787827036.407436:0:15166:0:(client.c:2715:ptlrpc_set_wait()) set ffff98560869d9c0 going to sleep for 11 seconds 00000100:00000001:0.0:1787827036.407438:0:15166:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1787827036.407440:0:15166:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1787827036.407489:0:2377:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1787827036.407493:0:2377:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9856030d2e00. 00000400:00000200:2.0:1787827036.407497:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1787827036.407502:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000400:00000200:2.0:1787827036.407505:0:2377:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985610d30ee0 00000400:00000010:2.0:1787827036.407507:0:2377:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985610d30ee0. 00000100:00000001:2.0:1787827036.407510:0:2377:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1787827036.407513:0:2377:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff985607f08000 x1874669228032768/t0(0) o256->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 304/240 e 0 to 0 dl 1787827052 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827036.407525:0:2377:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827036.407527:0:2377:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 2 req@ffff985607f08000 x1874669228032768/t0(0) o256->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 304/240 e 0 to 0 dl 1787827052 ref 3 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827036.407531:0:2377:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1787827036.407532:0:2377:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1787827036.408415:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827036.408424:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1787827036.408427:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827036.408429:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827036.408435:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827036.408440:0:2378:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp) <- 192.168.203.114@tcp : PUT - for me 00000400:00000200:2.0:1787827036.408446:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827036.408449:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827036.408453:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827036.408457:0:2378:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.203.114@tcp of length 44 into portal 33 MB=0x6a90071688300 00000400:00000200:2.0:1787827036.408461:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827036.408466:0:2378:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 21 from 12345-192.168.203.114@tcp of length 44/44 into md 0x32335 [4] + 0 00000400:00000010:2.0:1787827036.408470:0:2378:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9856094acd10. 00000400:00000200:2.0:1787827036.408473:0:2378:0:(lib-md.c:42:lnet_md_unlink()) Queueing unlink of md ffff98560869da80 00000800:00000001:2.0:1787827036.408478:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827036.408484:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827036.408487:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1787827036.408491:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000400:00000200:2.0:1787827036.408494:0:2378:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff98560869da80 00000400:00000010:2.0:1787827036.408496:0:2378:0:(lib-lnet.h:286:lnet_md_free()) kfreed 'md': 184 at ffff98560869da80. 00000100:00000001:2.0:1787827036.408501:0:2378:0:(events.c:174:client_bulk_callback()) Process entered 00000100:00000200:2.0:1787827036.408503:0:2378:0:(events.c:192:client_bulk_callback()) event type 2, status 0, req ffff985607f08000 desc ffff985608544000 mbits 1874669228032768 00000100:00000001:2.0:1787827036.408515:0:2378:0:(events.c:217:client_bulk_callback()) Process leaving 00000400:00000200:2.0:1787827036.408520:0:2378:0:(lib-md.c:96:lnet_cpt_of_md()) Couldn't resolve first page of md 0000000000000000 with offset 0 00000400:00000200:2.0:1787827036.408524:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827036.408528:0:2378:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source Specified: 192.168.203.14@tcp to MR: 192.168.203.114@tcp local destination 00000400:00000200:2.0:1787827036.408533:0:2378:0:(lib-move.c:1745:lnet_handle_send()) 192.168.203.14@tcp NI seq info: [21894:21894:256:4294967295] 192.168.203.114@tcp LPNI seq info [21894:21894:8:4294967295] 00000400:00000200:2.0:1787827036.408539:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000001:0.0:1787827036.408549:0:15166:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000400:00000200:2.0:1787827036.408549:0:2378:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp:192.168.203.14@tcp) -> 192.168.203.114@tcp(192.168.203.114@tcp:192.168.203.114@tcp) 192.168.203.114@tcp : ACK try# 0 00000100:00000001:0.0:1787827036.408553:0:15166:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1787827036.408557:0:2378:0:(socklnd_cb.c:931:ksocknal_send()) sending 0 bytes in 4 frags to 12345-192.168.203.114@tcp 00000800:00000010:2.0:1787827036.408562:0:2378:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 296 at ffff985607e32c00. 00000800:00000200:2.0:1787827036.408567:0:2378:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff985606751480] -> 12345-192.168.203.114@tcp (5) 00000800:00000200:2.0:1787827036.408572:0:2378:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.203.114@tcp ip 192.168.203.114:988 00000800:00000200:2.0:1787827036.408576:0:2378:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff985607e32c00 type 0, nob 96 niov 1 nkiov 0 00000800:00000001:3.0:1787827036.408608:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000200:2.0:1787827036.408636:0:2378:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1787827036.408640:0:2378:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 296 at ffff985607e32c00. 00000400:00000200:2.0:1787827036.408644:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1787827036.408648:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: ACK: OK 00000800:00000001:3.0:1787827036.408848:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1787827036.409433:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827036.409440:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1787827036.409443:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827036.409445:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827036.409450:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827036.409455:0:2378:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp) <- 192.168.203.114@tcp : PUT - for me 00000400:00000200:2.0:1787827036.409461:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827036.409464:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827036.409467:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827036.409471:0:2378:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.203.114@tcp of length 240 into portal 25 MB=0x6a90071688300 00000400:00000200:2.0:1787827036.409474:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827036.409478:0:2378:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.203.114@tcp of length 240/240 into md 0x3233d [1] + 224 00000800:00000001:2.0:1787827036.409483:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827036.409504:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827036.409507:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1787827036.409511:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000100:00000001:2.0:1787827036.409515:0:2378:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1787827036.409521:0:2378:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff985607f08000 x1874669228032768/t0(0) o256->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 304/240 e 0 to 0 dl 1787827052 ref 2 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1787827036.409530:0:2378:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=240 offset=224 replen=240 req@ffff985607f08000 x1874669228032768/t0(0) o256->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 304/240 e 0 to 0 dl 1787827052 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827036.409547:0:2378:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1787827036.409553:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827036.409555:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1787827036.409578:0:15166:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1787827036.409585:0:15166:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1787827036.409588:0:15166:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1787827036.409590:0:15166:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9856094ac370. 00000400:00000200:0.0:1787827036.409593:0:15166:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985610d30880 00000400:00000010:0.0:1787827036.409594:0:15166:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985610d30880. 00000100:00000001:0.0:1787827036.409597:0:15166:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1787827036.409601:0:15166:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff985607f08000 x1874669228032768/t0(0) o256->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 304/240 e 0 to 0 dl 1787827052 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1787827036.409607:0:15166:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff985607f08000 x1874669228032768/t0(0) o256->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 304/240 e 0 to 0 dl 1787827052 ref 2 fl Rpc:RQU/200/ffffffff rc 0/-1 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787827036.409613:0:15166:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1787827036.409615:0:15166:0:(client.c:3001:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1787827036.409617:0:15166:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:0.0:1787827036.409620:0:15166:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1787827036.409622:0:15166:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1787827036.409624:0:15166:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1787827036.409627:0:15166:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1787827036.409632:0:15166:0:(import.c:2149:obd_at_measure()) add 5 to ffff98561118e460 time=36 v=5 (5 5 5 5) 00000100:00001000:0.0:1787827036.409636:0:15166:0:(import.c:2149:obd_at_measure()) add 1 to ffff98561118e418 time=51 v=5 (1 0 0 0) 00000100:00000001:0.0:1787827036.409639:0:15166:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1787827036.409640:0:15166:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827036.409643:0:15166:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1787827036.409645:0:15166:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787827036.409647:0:15166:0:(client.c:1797:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787827036.409652:0:15166:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Bulk req@ffff985607f08000 x1874669228032768/t0(0) o256->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 304/240 e 0 to 0 dl 1787827052 ref 2 fl Rpc:RQU/200/0 rc 0/0 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1787827036.409660:0:15166:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Bulk to Interpret req@ffff985607f08000 x1874669228032768/t0(0) o256->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 304/240 e 0 to 0 dl 1787827052 ref 2 fl Bulk:RQU/200/0 rc 0/0 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787827036.409667:0:15166:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1787827036.409669:0:15166:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1787827036.409673:0:15166:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff985607f08000 x1874669228032768/t0(0) o256->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 304/240 e 0 to 0 dl 1787827052 ref 2 fl Interpret:RQU/200/0 rc 0/0 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827036.409680:0:15166:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607f08000 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669228032768:192.168.203.114@tcp:256:ll_cfg_requeue.0 00000100:00000001:0.0:1787827036.409686:0:15166:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1787827036.409689:0:15166:0:(client.c:2779:ptlrpc_set_wait()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787827036.409713:0:15166:0:(client.c:1237:ptlrpc_set_destroy()) Process entered 00000100:00000001:0.0:1787827036.409715:0:15166:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1787827036.409731:0:15166:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff985607f08000 x1874669228032768/t0(0) o256->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 304/240 e 0 to 0 dl 1787827052 ref 2 fl Complete:RQU/200/0 rc 0/0 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787827036.409749:0:15166:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:0.0:1787827036.409752:0:15166:0:(client.c:3800:ptlrpc_reqset_free()) kfreed 'set': 144 at ffff98560869d9c0. 00000100:00000001:0.0:1787827036.409754:0:15166:0:(client.c:1274:ptlrpc_set_destroy()) Process leaving 00000100:00000001:0.0:1787827036.409755:0:15166:0:(client.c:3371:ptlrpc_queue_wait()) Process leaving (rc=0 : 0 : 0) 10000000:00000040:0.0:1787827036.409761:0:15166:0:(mgc_request.c:1600:mgc_process_recover_log()) Latest version 208, more 0. 10000000:00000001:0.0:1787827036.409765:0:15166:0:(mgc_request.c:1308:mgc_apply_recover_logs()) Process entered 10000000:00000010:0.0:1787827036.409771:0:15166:0:(mgc_request.c:1331:mgc_apply_recover_logs()) kmalloced '(nidlist)': 640 at ffff985608547400. 10000000:00000010:0.0:1787827036.409775:0:15166:0:(mgc_request.c:1343:mgc_apply_recover_logs()) kmalloced '(obdname)': 128 at ffff985606643880. 10000000:00000200:0.0:1787827036.409779:0:15166:0:(mgc_request.c:1241:mgc_build_nidlist()) use 1 from 1 NIDs 10000000:00000040:0.0:1787827036.409781:0:15166:0:(mgc_request.c:1440:mgc_apply_recover_logs()) Looking for OBD lustre-OST0000-osc-ffff985602bc9800 00010000:00000001:0.0:1787827036.409788:0:15166:0:(ldlm_lib.c:159:client_import_add_nids_to_conn()) Process entered 00000100:00000001:0.0:1787827036.409791:0:15166:0:(lustre_peer.c:229:class_check_uuid()) Process entered 00000100:00000040:0.0:1787827036.409792:0:15166:0:(lustre_peer.c:232:class_check_uuid()) check if uuid 192.168.203.114@tcp has 192.168.203.114@tcp. 00000100:00000001:0.0:1787827036.409794:0:15166:0:(lustre_peer.c:251:class_check_uuid()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1787827036.409796:0:15166:0:(lustre_peer.c:189:class_add_nids_to_uuid()) Process entered 00000100:00000200:0.0:1787827036.409797:0:15166:0:(lustre_peer.c:194:class_add_nids_to_uuid()) Comparing 192.168.203.114@tcp with 192.168.203.114@tcp 00000100:00000200:0.0:1787827036.409799:0:15166:0:(lustre_peer.c:201:class_add_nids_to_uuid()) Updating UUID '192.168.203.114@tcp' 00000400:00000200:0.0:1787827036.409802:0:15166:0:(peer.c:1448:LNetAddPeer()) Add peer with 1 NIDs 00000400:00001000:0.0:1787827036.409804:0:15166:0:(api-ni.c:3331:LNetNIInit()) refs 1 00000400:00000200:0.0:1787827036.409806:0:15166:0:(peer.c:1462:LNetAddPeer()) Adding NID[0]: 192.168.203.114@tcp 00000400:00000200:0.0:1787827036.409809:0:15166:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:0.0:1787827036.409811:0:15166:0:(peer.c:1907:lnet_peer_add()) peer 192.168.203.114@tcp NID flags 0x100001: -17 00000400:00000200:0.0:1787827036.409813:0:15166:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:0.0:1787827036.409818:0:15166:0:(peer.c:2975:lnet_discover_peer_locked()) non-blocking discovery 00000400:00000200:0.0:1787827036.409820:0:15166:0:(peer.c:2984:lnet_discover_peer_locked()) peer 192.168.203.114@tcp NID 192.168.203.114@tcp: 0. pending discovery 00000400:00000200:0.0:1787827036.409822:0:15166:0:(peer.c:1516:LNetAddPeer()) A peer exists for NID 192.168.203.114@tcp 00000100:00000001:0.0:1787827036.409824:0:15166:0:(lustre_peer.c:219:class_add_nids_to_uuid()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827036.409825:0:15166:0:(ldlm_lib.c:174:client_import_add_nids_to_conn()) Process leaving (rc=0 : 0 : 0) 10000000:00000040:0.0:1787827036.409827:0:15166:0:(mgc_request.c:1470:mgc_apply_recover_logs()) IR: lustre-OST0000-osc-ffff985602bc9800 target is obsoleted(2/3), reconnect 00000100:00000001:0.0:1787827036.409829:0:15166:0:(recover.c:297:ptlrpc_recover_import()) Process entered 00000100:00080000:0.0:1787827036.409832:0:15166:0:(import.c:214:ptlrpc_set_import_discon()) osc: import ffff9856110b0000 already not connected (conn 8, was 0): DISCONN 00010000:00000001:0.0:1787827036.409835:0:15166:0:(ldlm_lib.c:49:import_set_conn()) Process entered 00000100:00000200:0.0:1787827036.409840:0:15166:0:(events.c:559:ptlrpc_uuid_to_peer()) 192.168.203.114@tcp->12345-192.168.203.114@tcp 00000100:00000001:0.0:1787827036.409842:0:15166:0:(connection.c:104:ptlrpc_connection_get()) Process entered 00000400:00000200:0.0:1787827036.409843:0:15166:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:0.0:1787827036.409846:0:15166:0:(peer.c:1635:LNetPrimaryNID()) NID 192.168.203.114@tcp primary NID 192.168.203.114@tcp rc 0 00000100:00000001:0.0:1787827036.409849:0:15166:0:(connection.c:156:ptlrpc_connection_addref()) Process entered 00000100:00000040:0.0:1787827036.409850:0:15166:0:(connection.c:161:ptlrpc_connection_addref()) conn=ffff98561201f640 refcount 16 to 192.168.203.114@tcp 00000100:00000001:0.0:1787827036.409852:0:15166:0:(connection.c:163:ptlrpc_connection_addref()) Process leaving (rc=18446630094169568832 : -113979539982784 : ffff98561201f640) 00000100:00000001:0.0:1787827036.409854:0:15166:0:(connection.c:110:ptlrpc_connection_get()) Process leaving via out (rc=18446630094169568832 : -113979539982784 : 0xffff98561201f640) 00000100:00000040:0.0:1787827036.409855:0:15166:0:(connection.c:149:ptlrpc_connection_get()) conn=ffff98561201f640 refcount 16 to 192.168.203.114@tcp 00000100:00000040:0.0:1787827036.409857:0:15166:0:(client.c:137:ptlrpc_uuid_to_connection()) 192.168.203.114@tcp -> ffff98561201f640 00010000:00080000:0.0:1787827036.409859:0:15166:0:(ldlm_lib.c:85:import_set_conn()) imp ffff9856110b0000@lustre-OST0000-osc-ffff985602bc9800: found existing conn 192.168.203.114@tcp, moved to head 00010000:00000001:0.0:1787827036.409860:0:15166:0:(ldlm_lib.c:87:import_set_conn()) Process leaving via out_free (rc=0 : 0 : 0x0) 00010000:00000040:0.0:1787827036.409862:0:15166:0:(lustre_net.h:1965:ptlrpc_connection_put()) PUT conn=ffff98561201f640 refcount 15 to 192.168.203.114@tcp 00010000:00000001:0.0:1787827036.409864:0:15166:0:(ldlm_lib.c:114:import_set_conn()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787827036.409866:0:15166:0:(import.c:701:ptlrpc_connect_import_locked()) Process entered 00000100:00080000:0.0:1787827036.409867:0:15166:0:(import.c:65:import_set_state_nolock()) ffff9856110b0000 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00000001:0.0:1787827036.409869:0:15166:0:(import.c:485:import_select_connection()) Process entered 00000100:00080000:0.0:1787827036.409871:0:15166:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 0 00000400:00000200:0.0:1787827036.409873:0:15166:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:0.0:1787827036.409875:0:15166:0:(peer.c:1685:LNetPeerDiscovered()) Peer NID 192.168.203.114@tcp is discovered: rc = 1 00000100:00000040:0.0:1787827036.409877:0:15166:0:(lustre_net.h:1965:ptlrpc_connection_put()) PUT conn=ffff98561201f640 refcount 14 to 192.168.203.114@tcp 00000100:00000001:0.0:1787827036.409878:0:15166:0:(connection.c:156:ptlrpc_connection_addref()) Process entered 00000100:00000040:0.0:1787827036.409879:0:15166:0:(connection.c:161:ptlrpc_connection_addref()) conn=ffff98561201f640 refcount 15 to 192.168.203.114@tcp 00000100:00000001:0.0:1787827036.409880:0:15166:0:(connection.c:163:ptlrpc_connection_addref()) Process leaving (rc=18446630094169568832 : -113979539982784 : ffff98561201f640) 00000020:00000001:0.0:1787827036.409881:0:15166:0:(genops.c:792:class_conn2export()) Process entered 00000020:00000040:0.0:1787827036.409883:0:15166:0:(genops.c:804:class_conn2export()) looking for export cookie 0xb3f83dad1b7dcdd6 00000020:00000001:0.0:1787827036.409885:0:15166:0:(lustre_handles.c:130:class_handle2object()) Process entered 00000020:00000040:0.0:1787827036.409887:0:15166:0:(lustre_handles.c:148:class_handle2object()) GET export ffff985602cdf000 refcount=4 00000020:00000001:0.0:1787827036.409889:0:15166:0:(lustre_handles.c:155:class_handle2object()) Process leaving (rc=18446630093914501120 : -113979795050496 : ffff985602cdf000) 00000020:00000001:0.0:1787827036.409890:0:15166:0:(genops.c:806:class_conn2export()) Process leaving (rc=18446630093914501120 : -113979795050496 : ffff985602cdf000) 00000100:00000040:0.0:1787827036.409892:0:15166:0:(lustre_net.h:1965:ptlrpc_connection_put()) PUT conn=ffff98561201f640 refcount 14 to 192.168.203.114@tcp 00000100:00000001:0.0:1787827036.409893:0:15166:0:(connection.c:156:ptlrpc_connection_addref()) Process entered 00000100:00000040:0.0:1787827036.409894:0:15166:0:(connection.c:161:ptlrpc_connection_addref()) conn=ffff98561201f640 refcount 15 to 192.168.203.114@tcp 00000100:00000001:0.0:1787827036.409895:0:15166:0:(connection.c:163:ptlrpc_connection_addref()) Process leaving (rc=18446630094169568832 : -113979539982784 : ffff98561201f640) 00000020:00000040:0.0:1787827036.409898:0:15166:0:(genops.c:879:class_export_put()) PUTting export ffff985602cdf000 : new refcount 3 00000100:00080000:0.0:1787827036.409901:0:15166:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff985602bc9800: import ffff9856110b0000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00000001:0.0:1787827036.409903:0:15166:0:(import.c:628:import_select_connection()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1787827036.409905:0:15166:0:(sec.c:1579:sptlrpc_import_sec_adapt()) Process entered 02000000:00000001:0.0:1787827036.409910:0:15166:0:(sec.c:1619:sptlrpc_import_sec_adapt()) Process leaving via out (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1787827036.409913:0:15166:0:(sec.c:1647:sptlrpc_import_sec_adapt()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787827036.409915:0:15166:0:(obd_class.h:889:obd_reconnect()) Process entered 00000008:00100000:0.0:1787827036.409919:0:15166:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683216 ocd_grant: 8388608, lost: 0. 00000008:00000001:0.0:1787827036.409922:0:15166:0:(osc_request.c:4022:osc_reconnect()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787827036.409924:0:15166:0:(obd_class.h:903:obd_reconnect()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:0.0:1787827036.409928:0:15166:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 904 at ffff985607f0b840. 00000020:00000040:0.0:1787827036.409931:0:15166:0:(genops.c:1210:class_import_get()) import ffff9856110b0000 refcount=3 obd=lustre-OST0000-osc-ffff985602bc9800 02000000:00000001:0.0:1787827036.409934:0:15166:0:(sec.c:2038:sptlrpc_sepol_get()) Process entered 02000000:00000001:0.0:1787827036.409936:0:15166:0:(sec.c:2048:sptlrpc_sepol_get()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787827036.409938:0:15166:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1787827036.409941:0:15166:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1787827036.409944:0:15166:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1787827036.409950:0:15166:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 1024 at ffff985608546c00. 00000100:00000001:0.0:1787827036.409954:0:15166:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1787827036.409960:0:15166:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607f0b840 x1874669228032896/t0(0) o8->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1787827036.409968:0:15166:0:(jobid.c:939:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1787827036.409972:0:15166:0:(jobid.c:979:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787827036.409976:0:15166:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000002bed56f1] to pc [ptlrpcd_rcv+-1] req@ffff985607f0b840 x1874669228032896/t0(0) o8->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787827036.410005:0:15166:0:(import.c:833:ptlrpc_connect_import_locked()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787827036.410007:0:15166:0:(recover.c:350:ptlrpc_recover_import()) Process leaving 10000000:00000040:0.0:1787827036.410008:0:15166:0:(mgc_request.c:1487:mgc_apply_recover_logs()) ir apply logs 208/208 for lustre-OST0000-osc-ffff985602bc9800 10000000:00000010:0.0:1787827036.410010:0:15166:0:(mgc_request.c:1492:mgc_apply_recover_logs()) kfreed 'obdname': 128 at ffff985606643880. 10000000:00000010:0.0:1787827036.410013:0:15166:0:(mgc_request.c:1494:mgc_apply_recover_logs()) kfreed 'nidlist': 640 at ffff985608547400. 10000000:00000001:0.0:1787827036.410014:0:15166:0:(mgc_request.c:1496:mgc_apply_recover_logs()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787827036.410016:0:15166:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1787827036.410019:0:15166:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607f08000 x1874669228032768/t0(0) o256->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 304/240 e 0 to 0 dl 1787827052 ref 1 fl Complete:RQU/200/0 rc 0/0 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787827036.410023:0:15166:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1787827036.410025:0:15166:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1787827036.410026:0:15166:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff985607f55c00. 02000000:00000001:0.0:1787827036.410028:0:15166:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1787827036.410029:0:15166:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:0.0:1787827036.410030:0:15166:0:(genops.c:1223:class_import_put()) import ffff98561118e000 refcount=4 obd=MGC192.168.203.114@tcp 00000020:00000001:0.0:1787827036.410032:0:15166:0:(genops.c:1230:class_import_put()) Process leaving 00000100:00000001:0.0:1787827036.410033:0:15166:0:(client.c:325:ptlrpc_free_bulk()) Process entered 00000020:00000001:0.0:1787827036.410035:0:15166:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:0.0:1787827036.410035:0:15166:0:(genops.c:1223:class_import_put()) import ffff98561118e000 refcount=3 obd=MGC192.168.203.114@tcp 00000020:00000001:0.0:1787827036.410036:0:15166:0:(genops.c:1230:class_import_put()) Process leaving 00000100:00000010:0.0:1787827036.410038:0:15166:0:(client.c:346:ptlrpc_free_bulk()) kvfree 'desc->bd_vec': 64 at ffff9856111e4880. 00000100:00000010:0.0:1787827036.410041:0:15166:0:(client.c:347:ptlrpc_free_bulk()) kfreed 'desc': 984 at ffff985608544000. 00000100:00000001:0.0:1787827036.410042:0:15166:0:(client.c:348:ptlrpc_free_bulk()) Process leaving 02000000:00000010:0.0:1787827036.410044:0:15166:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff985607f55000. 02000000:00000001:0.0:1787827036.410045:0:15166:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1787827036.410046:0:15166:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1787827036.410048:0:15166:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607f08000. 00000100:00000001:0.0:1787827036.410052:0:15166:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1787827036.410053:0:15166:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 10000000:00000010:0.0:1787827036.410067:0:15166:0:(mgc_request.c:1652:mgc_process_recover_log()) kvfree 'folios': 32 at ffff9856119542e0. 10000000:01000000:0.0:1787827036.410071:0:15166:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.114@tcp: configuration from log 'lustre-cliir' succeeded (0). 00000100:00000001:1.0:1787827036.410073:0:2382:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 10000000:00000001:0.0:1787827036.410075:0:15166:0:(mgc_request.c:1029:mgc_cancel()) Process entered 00010000:00000001:0.0:1787827036.410081:0:15166:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00000100:00000001:1.0:1787827036.410083:0:2382:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000020:00000001:0.0:1787827036.410084:0:15166:0:(lustre_handles.c:130:class_handle2object()) Process entered 00000100:00000001:1.0:1787827036.410087:0:2382:0:(client.c:1813:ptlrpc_send_new_req()) Process entered 00000020:00000040:0.0:1787827036.410097:0:15166:0:(lustre_handles.c:148:class_handle2object()) GET ldlm ffff985610e07c00 refcount=3 00000100:00000040:1.0:1787827036.410099:0:2382:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff985607f0b840 x1874669228032896/t0(0) o8->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00000020:00000001:0.0:1787827036.410104:0:15166:0:(lustre_handles.c:155:class_handle2object()) Process leaving (rc=18446630094150597632 : -113979558953984 : ffff985610e07c00) 00010000:00000001:0.0:1787827036.410119:0:15166:0:(ldlm_lock.c:634:__ldlm_handle2lock()) Process leaving (rc=18446630094150597632 : -113979558953984 : ffff985610e07c00) 00000100:00000001:1.0:1787827036.410123:0:2382:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1787827036.410126:0:2382:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827036.410127:0:15166:0:(ldlm_lock.c:854:ldlm_lock_decref_internal()) Process entered 02000000:00000001:1.0:1787827036.410130:0:2382:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 00010000:00010000:0.0:1787827036.410135:0:15166:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.114@tcp lock: ffff985610e07c00/0xb3f83dad1b7eda5a lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x5aa6d504d91df545 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 02000000:00000001:1.0:1787827036.410137:0:2382:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1787827036.410146:0:2382:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1787827036.410155:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607f0b840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228032896:192.168.203.114@tcp:8:ll_cfg_requeue.0 00010000:00000001:0.0:1787827036.410167:0:15166:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00000100:00000001:1.0:1787827036.410168:0:2382:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00010000:00000001:0.0:1787827036.410174:0:15166:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 02000000:00000001:1.0:1787827036.410179:0:2382:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 00010000:00010000:0.0:1787827036.410180:0:15166:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.114@tcp lock: ffff985610e07c00/0xb3f83dad1b7eda5a lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x5aa6d504d91df545 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 02000000:00000001:1.0:1787827036.410185:0:2382:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1787827036.410193:0:2382:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 00010000:00000001:0.0:1787827036.410199:0:15166:0:(ldlm_lock.c:923:ldlm_lock_decref_internal()) Process leaving 00010000:00000001:0.0:1787827036.410203:0:15166:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:0.0:1787827036.410203:0:15166:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 10000000:00000001:0.0:1787827036.410206:0:15166:0:(mgc_request.c:1033:mgc_cancel()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1787827036.410206:0:2382:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 1024 at ffff985607d01400. 02000000:00000001:1.0:1787827036.410217:0:2382:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:0.0:1787827036.410218:0:15166:0:(mgc_request.c:1907:mgc_process_log()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:0.0:1787827036.410229:0:15166:0:(mgc_request.c:621:do_requeue()) Process leaving 10000000:00000001:0.0:1787827036.410233:0:15166:0:(mgc_request.c:117:config_log_put()) Process entered 00000400:00000200:1.0:1787827036.410235:0:2382:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 10000000:00000040:0.0:1787827036.410237:0:15166:0:(mgc_request.c:123:config_log_put()) log lustre-cliir(ffff985602e53600) refs 3 00000400:00000010:1.0:1787827036.410245:0:2382:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff985602ecf898. 10000000:00000001:0.0:1787827036.410246:0:15166:0:(mgc_request.c:147:config_log_put()) Process leaving 00000400:00000010:1.0:1787827036.410262:0:2382:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985606618aa0. 00000100:00000200:1.0:1787827036.410272:0:2382:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 1024 bytes, xid 1874669228032896, portal 4 00000100:00000001:1.0:1787827036.410274:0:2382:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1787827036.410277:0:2382:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446630094000666688 : -113979708884928 : ffff985607f0b840) 00000100:00000040:1.0:1787827036.410289:0:2382:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff985607f0b840 x1874669228032896/t0(0) o8->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 520/544 e 0 to 0 dl 1787827046 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827036.410299:0:2382:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1787827036.410302:0:2382:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.203.114@tcp 00000400:00000010:1.0:1787827036.410306:0:2382:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985606618550. 00000100:00000200:1.0:1787827036.410310:0:2382:0:(niobuf.c:94:ptl_send_buf()) Sending 520 bytes to portal 28, xid 1874669228032896, offset 0 00000400:00000200:1.0:1787827036.410317:0:2382:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.203.114@tcp 00000400:00000200:1.0:1787827036.410327:0:2382:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:1.0:1787827036.410334:0:2382:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.203.114@tcp local destination 00000400:00000200:1.0:1787827036.410339:0:2382:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1787827036.410343:0:2382:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.203.14@tcp 00000400:00000200:1.0:1787827036.410346:0:2382:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.203.14@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1787827036.410348:0:2382:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.203.114@tcp 00000400:00000200:1.0:1787827036.410352:0:2382:0:(lib-move.c:1745:lnet_handle_send()) 192.168.203.14@tcp NI seq info: [21895:21895:256:4294967295] 192.168.203.114@tcp LPNI seq info [21895:21895:8:4294967295] 00000400:00000200:1.0:1787827036.410354:0:2382:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:1.0:1787827036.410363:0:2382:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp:) -> 192.168.203.114@tcp(192.168.203.114@tcp:192.168.203.114@tcp) : PUT try# 0 00000800:00000200:1.0:1787827036.410369:0:2382:0:(socklnd_cb.c:931:ksocknal_send()) sending 520 bytes in 1 frags to 12345-192.168.203.114@tcp 00000800:00000010:1.0:1787827036.410374:0:2382:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff985607d95200. 00000800:00000200:1.0:1787827036.410381:0:2382:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff985606751480] -> 12345-192.168.203.114@tcp (5) 00000800:00000200:1.0:1787827036.410389:0:2382:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.203.114@tcp ip 192.168.203.114:988 00000800:00000200:1.0:1787827036.410394:0:2382:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff985607d95200 type 1, nob 616 niov 1 nkiov 1 00000100:00000001:1.0:1787827036.410417:0:2382:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1787827036.410423:0:2382:0:(niobuf.c:635:kick_cpu_latency()) lustre-OST0000-osc-ffff985602bc9800: using a timeout of 14601 usecs (15 jiffies) 00000400:00000200:1.0:1787827036.410427:0:2382:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000001:1.0:1787827036.410430:0:2382:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff985602bc9800: PM QoS new deadline estimation for cpu 2 is 4297854484 00000100:00000001:1.0:1787827036.410434:0:2382:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff985602bc9800: PM QoS request active for cpu 2, keep current deadline 4297864446 00000100:00000001:1.0:1787827036.410436:0:2382:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff985602bc9800: PM QoS new deadline estimation for cpu 3 is 4297854484 00000100:00000001:1.0:1787827036.410439:0:2382:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff985602bc9800: PM QoS request active for cpu 3, keep current deadline 4297864446 00000100:00000001:1.0:1787827036.410442:0:2382:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1787827036.410446:0:2382:0:(client.c:1941:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827036.410449:0:2382:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827036.410453:0:2382:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787827036.410461:0:2382:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1787827036.410464:0:2382:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=5 : 5 : 5) 00000100:00000001:1.0:1787827036.410467:0:2382:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1787827036.410468:0:2382:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1787827036.410481:0:2382:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827036.410484:0:2382:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1787827036.410546:0:2377:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1787827036.410551:0:2377:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff985607d95200. 00000400:00000200:3.0:1787827036.410557:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1787827036.410563:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000400:00000200:3.0:1787827036.410567:0:2377:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985606618550 00000400:00000010:3.0:1787827036.410570:0:2377:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985606618550. 00000100:00000001:3.0:1787827036.410574:0:2377:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1787827036.410579:0:2377:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff985607f0b840 x1874669228032896/t0(0) o8->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 520/544 e 0 to 0 dl 1787827046 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827036.410588:0:2377:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1787827036.410592:0:2377:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff985607f0b840 x1874669228032896/t0(0) o8->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 520/544 e 0 to 0 dl 1787827046 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827036.410600:0:2377:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827036.410601:0:2377:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1787827036.411749:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827036.411761:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1787827036.411764:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827036.411767:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827036.411775:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827036.411782:0:2378:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp) <- 192.168.203.114@tcp : PUT - for me 00000400:00000200:2.0:1787827036.411788:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827036.411792:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827036.411797:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827036.411801:0:2378:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.203.114@tcp of length 416 into portal 4 MB=0x6a90071688380 00000400:00000200:2.0:1787827036.411805:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827036.411810:0:2378:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 4 from 12345-192.168.203.114@tcp of length 416/416 into md 0x32345 [1] + 0 00000800:00000001:2.0:1787827036.411817:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827036.411825:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827036.411827:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1787827036.411832:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000100:00000001:2.0:1787827036.411838:0:2378:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1787827036.411846:0:2378:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff985607f0b840 x1874669228032896/t0(0) o8->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 520/544 e 0 to 0 dl 1787827046 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1787827036.411857:0:2378:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=416 offset=0 replen=544 req@ffff985607f0b840 x1874669228032896/t0(0) o8->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 520/544 e 0 to 0 dl 1787827046 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827036.411876:0:2378:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1787827036.411883:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827036.411886:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787827036.411944:0:2382:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1787827036.411947:0:2382:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1787827036.411957:0:2382:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1787827036.411959:0:2382:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1787827036.411961:0:2382:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff985602ecf898. 00000400:00000200:1.0:1787827036.411965:0:2382:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985606618aa0 00000400:00000010:1.0:1787827036.411967:0:2382:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985606618aa0. 00000100:00000001:1.0:1787827036.411969:0:2382:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1787827036.411974:0:2382:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff985607f0b840 x1874669228032896/t0(0) o8->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 520/544 e 0 to 0 dl 1787827046 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1787827036.411981:0:2382:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff985607f0b840 x1874669228032896/t0(0) o8->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 520/544 e 0 to 0 dl 1787827046 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827036.411986:0:2382:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1787827036.411988:0:2382:0:(client.c:3001:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787827036.411990:0:2382:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:1.0:1787827036.411994:0:2382:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1787827036.411997:0:2382:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1787827036.411999:0:2382:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1787827036.412002:0:2382:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1787827036.412011:0:2382:0:(import.c:2149:obd_at_measure()) add 5 to ffff9856110b0460 time=114 v=5 (5 5 5 5) 00000100:00001000:1.0:1787827036.412016:0:2382:0:(import.c:2149:obd_at_measure()) add 1 to ffff9856110b0418 time=15 v=5 (15 1 1 1) 00000100:00000001:1.0:1787827036.412018:0:2382:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1787827036.412021:0:2382:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1787827036.412024:0:2382:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:1.0:1787827036.412029:0:2382:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=46162) req@ffff985607f0b840 x1874669228032896/t0(0) o8->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 520/416 e 0 to 0 dl 1787827046 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00010000:00000001:1.0:1787827036.412037:0:2382:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827036.412042:0:2382:0:(client.c:3109:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1787827036.412044:0:2382:0:(client.c:3121:ptlrpc_free_committed()) lustre-OST0000-osc-ffff985602bc9800: skip recheck: last_committed 8589935703 00000100:00000001:1.0:1787827036.412046:0:2382:0:(client.c:3122:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1787827036.412048:0:2382:0:(client.c:1797:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1787827036.412053:0:2382:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff985607f0b840 x1874669228032896/t0(0) o8->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 520/416 e 0 to 0 dl 1787827046 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827036.412060:0:2382:0:(client.c:2354:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1787827036.412065:0:2382:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1787827036.412067:0:2382:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787827036.412070:0:2382:0:(import.c:1057:ptlrpc_connect_interpret()) Process entered 00000100:00080000:1.0:1787827036.412078:0:2382:0:(import.c:1132:ptlrpc_connect_interpret()) lustre-OST0000-osc-ffff985602bc9800: connect to target with instance 3 00000020:00000001:1.0:1787827036.412081:0:2382:0:(genops.c:792:class_conn2export()) Process entered 00000020:00000040:1.0:1787827036.412083:0:2382:0:(genops.c:804:class_conn2export()) looking for export cookie 0xb3f83dad1b7dcdd6 00000020:00000001:1.0:1787827036.412086:0:2382:0:(lustre_handles.c:130:class_handle2object()) Process entered 00000020:00000040:1.0:1787827036.412089:0:2382:0:(lustre_handles.c:148:class_handle2object()) GET export ffff985602cdf000 refcount=4 00000020:00000001:1.0:1787827036.412091:0:2382:0:(lustre_handles.c:155:class_handle2object()) Process leaving (rc=18446630093914501120 : -113979795050496 : ffff985602cdf000) 00000020:00000001:1.0:1787827036.412093:0:2382:0:(genops.c:806:class_conn2export()) Process leaving (rc=18446630093914501120 : -113979795050496 : ffff985602cdf000) 00000100:00001000:1.0:1787827036.412096:0:2382:0:(import.c:2149:obd_at_measure()) add 1 to ffff9856110b0418 time=1787827036 v=0 (0 0 0 0) 00000100:00001000:1.0:1787827036.412098:0:2382:0:(import.c:2215:obd_at_measure()) AT ffff9856110b0418 change: old=0 new=5 delta=5 (val=1) hist 1 0 0 0 00000100:00000040:1.0:1787827036.412102:0:2382:0:(obd_cksum.h:77:obd_cksum_types_supported_client()) Crypto hash speed: crc 1695, crc32c 10359, adler 1383 00000020:00000040:1.0:1787827036.412107:0:2382:0:(genops.c:879:class_export_put()) PUTting export ffff985602cdf000 : new refcount 3 00000100:00000001:1.0:1787827036.412110:0:2382:0:(obd_class.h:1222:obd_import_event()) Process entered 00000008:00000001:1.0:1787827036.412113:0:2382:0:(osc_request.c:4091:osc_import_event()) Process entered 00000008:00000020:1.0:1787827036.412116:0:2382:0:(osc_request.c:1064:osc_init_grant()) lustre-OST0000-osc-ffff985602bc9800, setting cl_avail_grant: 8388608 cl_lost_grant: 0. chunk bits: 12 cl_max_extent_pages: 16384 00000008:00000001:1.0:1787827036.412119:0:2382:0:(obd_class.h:1242:obd_notify()) Process entered 00020000:00000001:1.0:1787827036.412122:0:2382:0:(lov_obd.c:430:lov_notify()) Process entered 00000080:00000001:1.0:1787827036.412126:0:2382:0:(llite_lib.c:386:ll_upcall_notify()) Process entered 00000080:00000001:1.0:1787827036.412129:0:2382:0:(llite_lib.c:411:ll_upcall_notify()) Process leaving (rc=0 : 0 : 0) 00020000:00000001:1.0:1787827036.412131:0:2382:0:(lov_obd.c:467:lov_notify()) Process leaving (rc=0 : 0 : 0) 00000008:00000001:1.0:1787827036.412132:0:2382:0:(obd_class.h:1255:obd_notify()) Process leaving (rc=0 : 0 : 0) 00000008:00000001:1.0:1787827036.412134:0:2382:0:(osc_request.c:4158:osc_import_event()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827036.412135:0:2382:0:(obd_class.h:1233:obd_import_event()) Process leaving 00000100:00080000:1.0:1787827036.412139:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9856110b0000 lustre-OST0000_UUID: changing import state from CONNECTING to REPLAY 00000100:00000001:1.0:1787827036.412143:0:2382:0:(import.c:1648:ptlrpc_import_recovery_state_machine()) Process entered 00000100:00080000:1.0:1787827036.412145:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-OST0000_UUID 00000100:00000001:1.0:1787827036.412147:0:2382:0:(recover.c:40:ptlrpc_replay_next()) Process entered 00000100:00000001:1.0:1787827036.412148:0:2382:0:(client.c:3109:ptlrpc_free_committed()) Process entered 00000100:00100000:1.0:1787827036.412150:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff985602bc9800: committing for last_committed 8589935703 gen 5 00000100:00000040:1.0:1787827036.412153:0:2382:0:(client.c:3235:ptlrpc_free_committed()) lustre-OST0000-osc-ffff985602bc9800: normal: skip=0 replay=0/0 committed=0/0 00000100:00000001:1.0:1787827036.412155:0:2382:0:(client.c:3236:ptlrpc_free_committed()) Process leaving 00000100:00080000:1.0:1787827036.412157:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9856110b0000 from lustre-OST0000_UUID committed 8589935703 last 0 00000100:00000001:1.0:1787827036.412159:0:2382:0:(recover.c:138:ptlrpc_replay_next()) Process leaving (rc=0 : 0 : 0) 00000100:00080000:1.0:1787827036.412162:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9856110b0000 lustre-OST0000_UUID: changing import state from REPLAY to REPLAY_LOCKS 00000020:00000040:1.0:1787827036.412164:0:2382:0:(genops.c:1210:class_import_get()) import ffff9856110b0000 refcount=4 obd=lustre-OST0000-osc-ffff985602bc9800 00000100:00000001:1.0:1787827036.412583:0:2382:0:(import.c:1754:ptlrpc_import_recovery_state_machine()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827036.412588:0:2382:0:(import.c:1517:ptlrpc_connect_interpret()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1787827036.412594:0:2382:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff985607f0b840 x1874669228032896/t0(0) o8->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 520/416 e 0 to 0 dl 1787827046 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827036.412604:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607f0b840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228032896:192.168.203.114@tcp:8:ll_cfg_requeue.0 00000100:00000001:1.0:1787827036.412610:0:2382:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787827036.412614:0:2382:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00010000:00080000:2.0:1787827036.412617:0:88790:0:(ldlm_request.c:2972:ldlm_lock_replay_thread()) lock replay thread lustre-OST0000-osc-ffff985602bc9800 to lustre-OST0000_UUID@192.168.203.114@tcp 00000100:00000040:1.0:1787827036.412621:0:2382:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607f0b840 x1874669228032896/t0(0) o8->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 520/416 e 0 to 0 dl 1787827046 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00010000:00000001:2.0:1787827036.412623:0:88790:0:(ldlm_request.c:2924:__ldlm_replay_locks()) Process entered 00010000:00010000:2.0:1787827036.412628:0:88790:0:(ldlm_request.c:2888:ldlm_cancel_unused_locks_for_replay()) Dropping as many unused locks as possible before replay for namespace lustre-OST0000-osc-ffff985602bc9800 (1) 00000100:00000001:1.0:1787827036.412631:0:2382:0:(client.c:2793:__ptlrpc_free_req()) Process entered 00010000:00000001:2.0:1787827036.412635:0:88790:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 02000000:00000001:1.0:1787827036.412636:0:2382:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1787827036.412641:0:2382:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff985607d01400. 02000000:00000001:1.0:1787827036.412645:0:2382:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1787827036.412647:0:2382:0:(genops.c:1217:class_import_put()) Process entered 00000008:00000001:2.0:1787827036.412648:0:88790:0:(osc_lock.c:735:osc_ldlm_weigh_ast()) Process entered 00000020:00000001:2.0:1787827036.412653:0:88790:0:(cl_object.c:754:cl_env_obtain()) Process entered 00000020:00000040:1.0:1787827036.412656:0:2382:0:(genops.c:1223:class_import_put()) import ffff9856110b0000 refcount=3 obd=lustre-OST0000-osc-ffff985602bc9800 00000020:00000001:2.0:1787827036.412659:0:88790:0:(cl_object.c:785:cl_env_obtain()) Process leaving (rc=18446630094152538088 : -113979557013528 : ffff985610fe17e8) 00000020:00000001:1.0:1787827036.412661:0:2382:0:(genops.c:1230:class_import_put()) Process leaving 00000020:00001000:2.0:1787827036.412665:0:88790:0:(cl_object.c:823:cl_env_get()) 1@ffff985610fe17e0 02000000:00000010:1.0:1787827036.412666:0:2382:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 1024 at ffff985608546c00. 02000000:00000001:1.0:1787827036.412673:0:2382:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 00000020:00000001:2.0:1787827036.412677:0:88790:0:(cl_object.c:143:cl_object_top()) ffff98560856afc0 -> ffff985605435d00 02000000:00000001:1.0:1787827036.412678:0:2382:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1787827036.412681:0:2382:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607f0b840. 00000008:00000001:2.0:1787827036.412682:0:88790:0:(osc_lock.c:699:osc_lock_weight()) Process entered 00000020:00000001:2.0:1787827036.412685:0:88790:0:(cl_object.c:143:cl_object_top()) ffff985605435d00 -> ffff985605435d00 00000100:00000001:1.0:1787827036.412689:0:2382:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1787827036.412722:0:2382:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787827036.412725:0:2382:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000020:00000001:2.0:1787827036.412728:0:88790:0:(cl_io.c:125:__cl_io_init()) Process entered 00000100:00000001:1.0:1787827036.412743:0:2382:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1787827036.412747:0:2382:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:2.0:1787827036.412747:0:88790:0:(vvp_io.c:1880:vvp_io_init()) Process entered 00000100:00000001:1.0:1787827036.412752:0:2382:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000080:00200000:2.0:1787827036.412752:0:88790:0:(vvp_io.c:1885:vvp_io_init()) [0x2000182d1:0x643:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000100:00000001:1.0:1787827036.412756:0:2382:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1787827036.412759:0:88790:0:(cl_io.c:902:cl_io_slice_add()) Process entered 00000020:00000001:2.0:1787827036.412762:0:88790:0:(cl_io.c:908:cl_io_slice_add()) Process leaving 00000080:00000001:2.0:1787827036.412763:0:88790:0:(vvp_io.c:1957:vvp_io_init()) Process leaving (rc=0 : 0 : 0) 00020000:00000002:2.0:1787827036.412769:0:88790:0:(lov_object.c:1600:lov_io_init()) [0x2000182d1:0x643:0x0]io ffff985607ea80a0 type 8 ignore/verify layout 1/0, compressed 0 00020000:00000001:2.0:1787827036.412774:0:88790:0:(lov_io.c:2452:lov_io_init_composite()) Process entered 00020000:00000001:2.0:1787827036.412776:0:88790:0:(lov_io.c:764:lov_io_slice_init()) Process entered 00020000:00000001:2.0:1787827036.412778:0:88790:0:(lov_io.c:868:lov_io_slice_init()) Process leaving via out (rc=0 : 0 : 0x0) 00020000:00000001:2.0:1787827036.412780:0:88790:0:(lov_io.c:206:lov_io_subio_init()) Process entered 00020000:00000001:2.0:1787827036.412782:0:88790:0:(lov_io.c:214:lov_io_subio_init()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1787827036.412783:0:88790:0:(cl_io.c:902:cl_io_slice_add()) Process entered 00000020:00000001:2.0:1787827036.412785:0:88790:0:(cl_io.c:908:cl_io_slice_add()) Process leaving 00020000:00000001:2.0:1787827036.412786:0:88790:0:(lov_io.c:2465:lov_io_init_composite()) Process leaving 00000020:00000001:2.0:1787827036.412788:0:88790:0:(cl_io.c:146:__cl_io_init()) Process leaving (rc=0 : 0 : 0) 00000008:00000001:2.0:1787827036.412791:0:88790:0:(osc_cache.c:3566:osc_page_gang_lookup()) Process entered 00000020:00000001:2.0:1787827036.412796:0:88790:0:(cl_page.c:534:cl_page_get()) Process entered 00000020:00000001:2.0:1787827036.412798:0:88790:0:(cl_page.c:536:cl_page_get()) Process leaving 00000020:00000001:2.0:1787827036.412800:0:88790:0:(cl_page.c:554:cl_batch_put()) Process entered 00000020:00000001:2.0:1787827036.412807:0:88790:0:(cl_page.c:556:cl_batch_put()) page@0000000018ff6f2e[2 000000008e5ad20d 1 1 0000000000000000 0 0 0] 00000020:00000001:2.0:1787827036.412809:0:88790:0:(cl_page.c:556:cl_batch_put()) 2 00000020:00000001:2.0:1787827036.412810:0:88790:0:(cl_page.c:569:cl_batch_put()) Process leaving 00000008:00000001:2.0:1787827036.412812:0:88790:0:(osc_cache.c:3643:osc_page_gang_lookup()) Process leaving (rc=1 : 1 : 1) 00000020:00000001:2.0:1787827036.412814:0:88790:0:(cl_io.c:71:cl_io_fini()) Process entered 00020000:00000001:2.0:1787827036.412817:0:88790:0:(lov_io.c:1054:lov_io_fini()) Process entered 00020000:00000001:2.0:1787827036.412819:0:88790:0:(lov_io.c:1073:lov_io_fini()) Process leaving 00000080:00000001:2.0:1787827036.412821:0:88790:0:(vvp_io.c:296:vvp_io_fini()) Process entered 00000080:00200000:2.0:1787827036.412824:0:88790:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x643:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00000001:2.0:1787827036.412828:0:88790:0:(vvp_io.c:402:vvp_io_fini()) Process leaving 00000020:00000001:2.0:1787827036.412830:0:88790:0:(cl_io.c:112:cl_io_fini()) Process leaving 00000008:00000001:2.0:1787827036.412831:0:88790:0:(osc_lock.c:789:osc_ldlm_weigh_ast()) Process leaving 00000020:00001000:2.0:1787827036.412835:0:88790:0:(cl_object.c:917:cl_env_put()) 1@ffff985610fe17e0 00000008:00000001:2.0:1787827036.412838:0:88790:0:(osc_request.c:4180:osc_cancel_weight()) Process leaving (rc=1 : 1 : 1) 00010000:00000001:2.0:1787827036.412840:0:88790:0:(ldlm_request.c:1930:ldlm_cancel_no_wait_policy()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1787827036.412844:0:88790:0:(ldlm_lock.c:261:ldlm_lock_remove_from_lru_check()) Process entered 00010000:00000001:2.0:1787827036.412848:0:88790:0:(ldlm_lock.c:280:ldlm_lock_remove_from_lru_check()) Process leaving (rc=1 : 1 : 1) 00010000:00000001:2.0:1787827036.412850:0:88790:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=1 : 1 : 1) 00010000:00000001:2.0:1787827036.412854:0:88790:0:(ldlm_lock.c:2501:ldlm_lock_cancel()) Process entered 00000008:00000001:2.0:1787827036.412857:0:88790:0:(osc_lock.c:534:osc_ldlm_blocking_ast()) Process entered 00000020:00000001:2.0:1787827036.412859:0:88790:0:(cl_object.c:754:cl_env_obtain()) Process entered 00000020:00000001:2.0:1787827036.412860:0:88790:0:(cl_object.c:785:cl_env_obtain()) Process leaving (rc=18446630094152538088 : -113979557013528 : ffff985610fe17e8) 00000020:00001000:2.0:1787827036.412863:0:88790:0:(cl_object.c:823:cl_env_get()) 1@ffff985610fe17e0 00000008:00000001:2.0:1787827036.412865:0:88790:0:(osc_lock.c:418:__osc_dlm_blocking_ast()) Process entered 00000008:00000001:2.0:1787827036.412867:0:88790:0:(osc_lock.c:360:osc_lock_flush()) Process entered 00000020:00000001:2.0:1787827036.412868:0:88790:0:(cl_object.c:754:cl_env_obtain()) Process entered 00000020:00000001:2.0:1787827036.412869:0:88790:0:(cl_object.c:785:cl_env_obtain()) Process leaving (rc=18446630094152536360 : -113979557015256 : ffff985610fe1128) 00000020:00001000:2.0:1787827036.412871:0:88790:0:(cl_object.c:823:cl_env_get()) 1@ffff985610fe1120 00000008:00000001:2.0:1787827036.412873:0:88790:0:(osc_cache.c:3810:osc_ldlm_hp_handle()) Process entered 00000008:00000020:2.0:1787827036.412875:0:88790:0:(osc_cache.c:3830:osc_ldlm_hp_handle()) High prio I/O check: start 0 end 4503599627370495 RPC(0):r0/w0/d0 00000008:00000001:2.0:1787827036.412878:0:88790:0:(osc_cache.c:3846:osc_ldlm_hp_handle()) Process leaving via out_unlock (rc=0 : 0 : 0x0) 00000008:00000001:2.0:1787827036.412880:0:88790:0:(osc_cache.c:3894:osc_ldlm_hp_handle()) Process leaving (rc=0 : 0 : 0) 00000008:00000001:2.0:1787827036.412883:0:88790:0:(osc_cache.c:3779:osc_lock_discard_pages()) Process entered 00000020:00000001:2.0:1787827036.412884:0:88790:0:(cl_object.c:143:cl_object_top()) ffff98560856afc0 -> ffff985605435d00 00000020:00000001:2.0:1787827036.412886:0:88790:0:(cl_object.c:143:cl_object_top()) ffff985605435d00 -> ffff985605435d00 00000020:00000001:2.0:1787827036.412887:0:88790:0:(cl_io.c:125:__cl_io_init()) Process entered 00000080:00000001:2.0:1787827036.412888:0:88790:0:(vvp_io.c:1880:vvp_io_init()) Process entered 00000080:00200000:2.0:1787827036.412890:0:88790:0:(vvp_io.c:1885:vvp_io_init()) [0x2000182d1:0x643:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000020:00000001:2.0:1787827036.412893:0:88790:0:(cl_io.c:902:cl_io_slice_add()) Process entered 00000020:00000001:2.0:1787827036.412894:0:88790:0:(cl_io.c:908:cl_io_slice_add()) Process leaving 00000080:00000001:2.0:1787827036.412895:0:88790:0:(vvp_io.c:1957:vvp_io_init()) Process leaving (rc=0 : 0 : 0) 00020000:00000002:2.0:1787827036.412898:0:88790:0:(lov_object.c:1600:lov_io_init()) [0x2000182d1:0x643:0x0]io ffff985605795078 type 8 ignore/verify layout 1/0, compressed 0 00020000:00000001:2.0:1787827036.412901:0:88790:0:(lov_io.c:2452:lov_io_init_composite()) Process entered 00020000:00000001:2.0:1787827036.412902:0:88790:0:(lov_io.c:764:lov_io_slice_init()) Process entered 00020000:00000001:2.0:1787827036.412904:0:88790:0:(lov_io.c:868:lov_io_slice_init()) Process leaving via out (rc=0 : 0 : 0x0) 00020000:00000001:2.0:1787827036.412906:0:88790:0:(lov_io.c:206:lov_io_subio_init()) Process entered 00020000:00000001:2.0:1787827036.412908:0:88790:0:(lov_io.c:214:lov_io_subio_init()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1787827036.412910:0:88790:0:(cl_io.c:902:cl_io_slice_add()) Process entered 00000020:00000001:2.0:1787827036.412911:0:88790:0:(cl_io.c:908:cl_io_slice_add()) Process leaving 00020000:00000001:2.0:1787827036.412912:0:88790:0:(lov_io.c:2465:lov_io_init_composite()) Process leaving 00000020:00000001:2.0:1787827036.412914:0:88790:0:(cl_io.c:146:__cl_io_init()) Process leaving (rc=0 : 0 : 0) 00000008:00000001:2.0:1787827036.412916:0:88790:0:(osc_cache.c:3566:osc_page_gang_lookup()) Process entered 00000020:00000001:2.0:1787827036.412918:0:88790:0:(cl_page.c:534:cl_page_get()) Process entered 00000020:00000001:2.0:1787827036.412919:0:88790:0:(cl_page.c:536:cl_page_get()) Process leaving 00000008:00000001:2.0:1787827036.412922:0:88790:0:(osc_lock.c:1351:osc_obj_dlmlock_at_pgoff()) Process entered 00000008:00000001:2.0:1787827036.412926:0:88790:0:(osc_request.c:3520:osc_match_base()) Process entered 00010000:00000001:2.0:1787827036.412929:0:88790:0:(ldlm_lock.c:1446:ldlm_lock_match_with_skip()) Process entered 00010000:00000040:2.0:1787827036.412938:0:88790:0:(ldlm_resource.c:1656:ldlm_resource_getref()) getref res: ffff9856091e7e00 count: 2 00010000:00000001:2.0:1787827036.412941:0:88790:0:(ldlm_resource.c:1595:ldlm_resource_get()) Process leaving via found (rc=0 : 0 : 0x0) 00000008:00000001:2.0:1787827036.412949:0:88790:0:(osc_request.c:3535:osc_match_base()) Process leaving (rc=0 : 0 : 0) 00000008:00000001:2.0:1787827036.412951:0:88790:0:(osc_lock.c:1385:osc_obj_dlmlock_at_pgoff()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1787827036.412954:0:88790:0:(cl_page.c:700:__cl_page_own()) Process entered 00000020:00000001:2.0:1787827036.412958:0:88790:0:(cl_io.c:1149:cl_io_top()) Process entered 00000020:00000001:2.0:1787827036.412960:0:88790:0:(cl_io.c:1152:cl_io_top()) Process leaving (rc=18446630093959286904 : -113979750264712 : ffff985605795078) 00000020:00000001:2.0:1787827036.412962:0:88790:0:(cl_page.c:634:cl_page_owner_set()) Process entered 00000020:00000001:2.0:1787827036.412963:0:88790:0:(cl_page.c:637:cl_page_owner_set()) Process leaving 00000020:00000001:2.0:1787827036.412965:0:88790:0:(cl_page.c:504:__cl_page_state_set()) Process entered 00000020:00000001:2.0:1787827036.412968:0:88790:0:(cl_page.c:507:__cl_page_state_set()) page@0000000018ff6f2e[2 000000008e5ad20d 1 1 00000000479415d7 0 0 0] 00000020:00000001:2.0:1787827036.412970:0:88790:0:(cl_page.c:507:__cl_page_state_set()) 1 -> 2 00000020:00000001:2.0:1787827036.412972:0:88790:0:(cl_page.c:515:__cl_page_state_set()) Process leaving 00000020:00000040:2.0:1787827036.412973:0:88790:0:(cl_page.c:744:__cl_page_own()) res 0 00000020:00000001:2.0:1787827036.412974:0:88790:0:(cl_page.c:747:__cl_page_own()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1787827036.412977:0:88790:0:(cl_page.c:976:cl_page_delete()) Process entered 00000020:00000001:2.0:1787827036.412978:0:88790:0:(cl_page.c:928:__cl_page_delete()) Process entered 00000020:00000001:2.0:1787827036.412980:0:88790:0:(cl_page.c:623:cl_page_owner_clear()) Process entered 00000020:00000001:2.0:1787827036.412981:0:88790:0:(cl_page.c:629:cl_page_owner_clear()) Process leaving 00000020:00000001:2.0:1787827036.412981:0:88790:0:(cl_page.c:504:__cl_page_state_set()) Process entered 00000020:00000001:2.0:1787827036.412983:0:88790:0:(cl_page.c:507:__cl_page_state_set()) page@0000000018ff6f2e[2 000000008e5ad20d 2 1 0000000000000000 0 0 0] 00000020:00000001:2.0:1787827036.412984:0:88790:0:(cl_page.c:507:__cl_page_state_set()) 2 -> 5 00000020:00000001:2.0:1787827036.412984:0:88790:0:(cl_page.c:515:__cl_page_state_set()) Process leaving 00000008:00000001:2.0:1787827036.412986:0:88790:0:(osc_page.c:137:osc_page_delete()) Process entered 00000008:00000001:2.0:1787827036.412986:0:88790:0:(osc_page.c:138:osc_page_delete()) ffff9856055ccd10 00000008:00000001:2.0:1787827036.412988:0:88790:0:(osc_cache.c:2782:osc_teardown_async_page()) Process entered 00000008:00000040:2.0:1787827036.412989:0:88790:0:(osc_cache.c:2785:osc_teardown_async_page()) teardown oap ffff9856055ccd20 page ffff9856055ccd10 at index 0. 00000008:00000001:2.0:1787827036.412990:0:88790:0:(osc_cache.c:2807:osc_teardown_async_page()) Process leaving (rc=0 : 0 : 0) 00000008:00000001:2.0:1787827036.412995:0:88790:0:(osc_page.c:169:osc_page_delete()) Process leaving 00000080:00000020:2.0:1787827036.412997:0:88790:0:(vvp_page.c:56:vvp_page_delete()) delete page ffffd6d3c4b1cd00 index 0 00000020:00000001:2.0:1787827036.413001:0:88790:0:(cl_page.c:946:__cl_page_delete()) Process leaving 00000020:00000001:2.0:1787827036.413002:0:88790:0:(cl_page.c:978:cl_page_delete()) Process leaving 00000020:00000001:2.0:1787827036.413016:0:88790:0:(cl_page.c:645:__cl_page_disown()) Process entered 00000020:00000001:2.0:1787827036.413017:0:88790:0:(cl_page.c:623:cl_page_owner_clear()) Process entered 00000020:00000001:2.0:1787827036.413018:0:88790:0:(cl_page.c:629:cl_page_owner_clear()) Process leaving 00000020:00000001:2.0:1787827036.413019:0:88790:0:(cl_page.c:660:__cl_page_disown()) Process leaving 00000020:00000001:2.0:1787827036.413020:0:88790:0:(cl_page.c:554:cl_batch_put()) Process entered 00000020:00000001:2.0:1787827036.413022:0:88790:0:(cl_page.c:556:cl_batch_put()) page@0000000018ff6f2e[1 000000008e5ad20d 5 1 0000000000000000 0 0 0] 00000020:00000001:2.0:1787827036.413024:0:88790:0:(cl_page.c:556:cl_batch_put()) 1 00000020:00000001:2.0:1787827036.413026:0:88790:0:(cl_page.c:236:cl_page_free()) Process entered 00000020:00000010:2.0:1787827036.413030:0:88790:0:(cl_page.c:223:__cl_page_free()) slab-freed 'cl_page': 232 at ffff9856055cccb0. 00000020:00000001:2.0:1787827036.413034:0:88790:0:(cl_page.c:274:cl_page_free()) Process leaving 00000020:00000001:2.0:1787827036.413035:0:88790:0:(cl_page.c:569:cl_batch_put()) Process leaving 00000008:00000001:2.0:1787827036.413076:0:88790:0:(osc_cache.c:3643:osc_page_gang_lookup()) Process leaving (rc=1 : 1 : 1) 00000020:00000001:2.0:1787827036.413078:0:88790:0:(cl_io.c:71:cl_io_fini()) Process entered 00020000:00000001:2.0:1787827036.413080:0:88790:0:(lov_io.c:1054:lov_io_fini()) Process entered 00020000:00000001:2.0:1787827036.413082:0:88790:0:(lov_io.c:1073:lov_io_fini()) Process leaving 00000080:00000001:2.0:1787827036.413083:0:88790:0:(vvp_io.c:296:vvp_io_fini()) Process entered 00000080:00200000:2.0:1787827036.413085:0:88790:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x643:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 1 00000080:00000001:2.0:1787827036.413087:0:88790:0:(vvp_io.c:402:vvp_io_fini()) Process leaving 00000020:00000001:2.0:1787827036.413088:0:88790:0:(cl_io.c:112:cl_io_fini()) Process leaving 00000008:00000001:2.0:1787827036.413089:0:88790:0:(osc_cache.c:3798:osc_lock_discard_pages()) Process leaving (rc=0 : 0 : 0) 00000020:00001000:2.0:1787827036.413090:0:88790:0:(cl_object.c:917:cl_env_put()) 1@ffff985610fe1120 00000008:00000001:2.0:1787827036.413092:0:88790:0:(osc_lock.c:392:osc_lock_flush()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1787827036.413094:0:88790:0:(cl_object.c:143:cl_object_top()) ffff98560856afc0 -> ffff985605435d00 00010000:00000001:2.0:1787827036.413096:0:88790:0:(ldlm_extent.c:945:ldlm_extent_shift_kms()) Process entered 00010000:00000001:2.0:1787827036.413099:0:88790:0:(ldlm_extent.c:1018:ldlm_extent_shift_kms()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1787827036.413100:0:88790:0:(cl_object.c:143:cl_object_top()) ffff98560856afc0 -> ffff985605435d00 00000020:00000001:2.0:1787827036.413101:0:88790:0:(cl_object.c:248:cl_object_attr_update()) Process entered 00000008:00000020:2.0:1787827036.413103:0:88790:0:(osc_object.c:161:osc_attr_update()) set kms from 4096to 0 00020000:00000001:2.0:1787827036.413106:0:88790:0:(lovsub_object.c:98:lovsub_attr_update()) Process entered 00020000:00000001:2.0:1787827036.413108:0:88790:0:(lovsub_object.c:100:lovsub_attr_update()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1787827036.413110:0:88790:0:(cl_object.c:261:cl_object_attr_update()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1787827036.413112:0:88790:0:(cl_object.c:143:cl_object_top()) ffff98560856afc0 -> ffff985605435d00 00000020:00000001:2.0:1787827036.413113:0:88790:0:(cl_object.c:143:cl_object_top()) ffff985605435d00 -> ffff985605435d00 00000020:00000001:2.0:1787827036.413115:0:88790:0:(cl_object.c:248:cl_object_attr_update()) Process entered 00000020:00000001:2.0:1787827036.413118:0:88790:0:(cl_object.c:261:cl_object_attr_update()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1787827036.413120:0:88790:0:(cl_object.c:143:cl_object_top()) ffff98560856afc0 -> ffff985605435d00 00000008:00000001:2.0:1787827036.413122:0:88790:0:(osc_lock.c:475:__osc_dlm_blocking_ast()) Process leaving (rc=0 : 0 : 0) 00000020:00001000:2.0:1787827036.413124:0:88790:0:(cl_object.c:917:cl_env_put()) 1@ffff985610fe17e0 00000008:00000001:2.0:1787827036.413126:0:88790:0:(osc_lock.c:574:osc_ldlm_blocking_ast()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1787827036.413145:0:88790:0:(ldlm_lock.c:410:ldlm_lock_destroy_nolock()) Process entered 00010000:00000001:2.0:1787827036.413147:0:88790:0:(ldlm_lock.c:354:ldlm_lock_destroy_internal()) Process entered 00010000:00000001:2.0:1787827036.413149:0:88790:0:(ldlm_lock.c:261:ldlm_lock_remove_from_lru_check()) Process entered 00010000:00000001:2.0:1787827036.413151:0:88790:0:(ldlm_lock.c:280:ldlm_lock_remove_from_lru_check()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:2.0:1787827036.413167:0:88790:0:(lustre_handles.c:107:class_handle_unhash_nolock()) removing object ffff985610e07800 with handle 0xb3f83dad1b7ea395 from hash 00010000:00000001:2.0:1787827036.413170:0:88790:0:(ldlm_lock.c:384:ldlm_lock_destroy_internal()) Process leaving 00010000:00000001:2.0:1787827036.413172:0:88790:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:2.0:1787827036.413175:0:88790:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:2.0:1787827036.413176:0:88790:0:(ldlm_lock.c:416:ldlm_lock_destroy_nolock()) Process leaving 00010000:00000001:2.0:1787827036.413180:0:88790:0:(ldlm_lock.c:2537:ldlm_lock_cancel()) Process leaving 00010000:00000001:2.0:1787827036.413182:0:88790:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00010000:2.0:1787827036.413186:0:88790:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0000-osc-ffff985602bc9800 lock: 0000000021ff82c6/0xb3f83dad1b7ea395 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x9a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x804869400000000 nid: local remote: 0x5aa6d504d91d0e29 expref: -99 pid: 87460 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00000010:2.0:1787827036.413195:0:88790:0:(ldlm_lock.c:209:ldlm_lock_put()) kvfree 'lock->l_lvb_data': 56 at ffff985614dac900. 00010000:00000040:2.0:1787827036.413200:0:88790:0:(ldlm_resource.c:1704:ldlm_resource_putref()) putref res: ffff9856091e7e00 count: 0 00010000:00000010:2.0:1787827036.413206:0:88790:0:(ldlm_resource.c:1563:ldlm_resource_free()) slab-freed 'res->lr_itree': 216 at ffff9856091e7300. 00010000:00000001:2.0:1787827036.413215:0:88790:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:2.0:1787827036.413217:0:88790:0:(ldlm_request.c:1890:ldlm_cli_cancel_list_local()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1787827036.413219:0:88790:0:(ldlm_request.c:2901:ldlm_cancel_unused_locks_for_replay()) Canceled 0 unused locks from namespace lustre-OST0000-osc-ffff985602bc9800 00000001:00000001:2.0:1787827036.413222:0:88790:0:(hash.c:1696:cfs_hash_for_each_nolock()) Process entered 00000001:00000001:2.0:1787827036.413225:0:88790:0:(hash.c:1609:cfs_hash_for_each_relax()) Process entered 00000001:00000001:2.0:1787827036.413262:0:88790:0:(hash.c:1712:cfs_hash_for_each_nolock()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1787827036.413265:0:88790:0:(ldlm_request.c:2954:__ldlm_replay_locks()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1787827036.413267:0:88790:0:(import.c:1648:ptlrpc_import_recovery_state_machine()) Process entered 00000100:00080000:2.0:1787827036.413269:0:88790:0:(import.c:65:import_set_state_nolock()) ffff9856110b0000 lustre-OST0000_UUID: changing import state from REPLAY_LOCKS to REPLAY_WAIT 00000100:00000001:2.0:1787827036.413272:0:88790:0:(import.c:1557:signal_completed_replay()) Process entered 00000100:00000010:2.0:1787827036.413278:0:88790:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 904 at ffff9856140d8f00. 00000020:00000040:2.0:1787827036.413281:0:88790:0:(genops.c:1210:class_import_get()) import ffff9856110b0000 refcount=4 obd=lustre-OST0000-osc-ffff985602bc9800 00000100:00000001:2.0:1787827036.413286:0:88790:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1787827036.413289:0:88790:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1787827036.413294:0:88790:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1787827036.413300:0:88790:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff985615779b00. 00000100:00000001:2.0:1787827036.413305:0:88790:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1787827036.413309:0:88790:0:(jobid.c:939:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1787827036.413313:0:88790:0:(jobid.c:979:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1787827036.413318:0:88790:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000e9d4c698] to pc [ptlrpcd_rcv+-1] req@ffff9856140d8f00 x1874669228033024/t0(0) o400->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:QU/2c0/ffffffff rc 0/-1 job:'ldlm_lock_repla.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827036.413338:0:88790:0:(import.c:1582:signal_completed_replay()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1787827036.413340:0:88790:0:(import.c:1754:ptlrpc_import_recovery_state_machine()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1787827036.413343:0:88790:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827036.413344:0:88790:0:(genops.c:1223:class_import_put()) import ffff9856110b0000 refcount=3 obd=lustre-OST0000-osc-ffff985602bc9800 00000020:00000001:2.0:1787827036.413346:0:88790:0:(genops.c:1230:class_import_put()) Process leaving 00000100:00000001:1.0:1787827036.413370:0:2382:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1787827036.413373:0:2382:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1787827036.413376:0:2382:0:(client.c:1813:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1787827036.413391:0:2382:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9856140d8f00 x1874669228033024/t0(0) o400->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:QU/2c0/ffffffff rc 0/-1 job:'ldlm_lock_repla.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827036.413401:0:2382:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1787827036.413404:0:2382:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1787827036.413409:0:2382:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1787827036.413412:0:2382:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1787827036.413415:0:2382:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1787827036.413419:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228033024:192.168.203.114@tcp:400:ldlm_lock_repla.0 00000100:00000001:1.0:1787827036.413424:0:2382:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1787827036.413430:0:2382:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1787827036.413433:0:2382:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1787827036.413436:0:2382:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1787827036.413441:0:2382:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff985610db9800. 02000000:00000001:1.0:1787827036.413443:0:2382:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1787827036.413450:0:2382:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000010:1.0:1787827036.413453:0:2382:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff985602ecf898. 00000400:00000010:1.0:1787827036.413458:0:2382:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985606618aa0. 00000100:00000200:1.0:1787827036.413463:0:2382:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1874669228033024, portal 4 00000100:00000001:1.0:1787827036.413465:0:2382:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1787827036.413466:0:2382:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446630094203883264 : -113979505668352 : ffff9856140d8f00) 00000100:00000040:1.0:1787827036.413470:0:2382:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=2c0 req@ffff9856140d8f00 x1874669228033024/t0(0) o400->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827052 ref 2 fl Rpc:r/2c0/ffffffff rc 0/-1 job:'ldlm_lock_repla.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827036.413475:0:2382:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1787827036.413477:0:2382:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.203.114@tcp 00000400:00000010:1.0:1787827036.413479:0:2382:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985606618990. 00000100:00000200:1.0:1787827036.413481:0:2382:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 28, xid 1874669228033024, offset 0 00000400:00000200:1.0:1787827036.413486:0:2382:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.203.114@tcp 00000400:00000200:1.0:1787827036.413492:0:2382:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:1.0:1787827036.413497:0:2382:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.203.114@tcp local destination 00000400:00000200:1.0:1787827036.413502:0:2382:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1787827036.413506:0:2382:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.203.14@tcp 00000400:00000200:1.0:1787827036.413510:0:2382:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.203.14@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1787827036.413513:0:2382:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.203.114@tcp 00000400:00000200:1.0:1787827036.413516:0:2382:0:(lib-move.c:1745:lnet_handle_send()) 192.168.203.14@tcp NI seq info: [21896:21896:256:4294967295] 192.168.203.114@tcp LPNI seq info [21896:21896:8:4294967295] 00000400:00000200:1.0:1787827036.413519:0:2382:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:1.0:1787827036.413526:0:2382:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp:) -> 192.168.203.114@tcp(192.168.203.114@tcp:192.168.203.114@tcp) : PUT try# 0 00000800:00000200:1.0:1787827036.413530:0:2382:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.203.114@tcp 00000800:00000010:1.0:1787827036.413535:0:2382:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff985607d95a00. 00000800:00000200:1.0:1787827036.413542:0:2382:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff985606751480] -> 12345-192.168.203.114@tcp (5) 00000800:00000200:1.0:1787827036.413550:0:2382:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.203.114@tcp ip 192.168.203.114:988 00000800:00000200:1.0:1787827036.413554:0:2382:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff985607d95a00 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1787827036.413571:0:2382:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1787827036.413576:0:2382:0:(niobuf.c:635:kick_cpu_latency()) lustre-OST0000-osc-ffff985602bc9800: using a timeout of 14591 usecs (15 jiffies) 00000400:00000200:1.0:1787827036.413580:0:2382:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000001:1.0:1787827036.413582:0:2382:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff985602bc9800: PM QoS new deadline estimation for cpu 2 is 4297854487 00000100:00000001:1.0:1787827036.413585:0:2382:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff985602bc9800: PM QoS request active for cpu 2, keep current deadline 4297864446 00000100:00000001:1.0:1787827036.413586:0:2382:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff985602bc9800: PM QoS new deadline estimation for cpu 3 is 4297854487 00000100:00000001:1.0:1787827036.413588:0:2382:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff985602bc9800: PM QoS request active for cpu 3, keep current deadline 4297864446 00000100:00000001:1.0:1787827036.413590:0:2382:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1787827036.413592:0:2382:0:(client.c:1941:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827036.413594:0:2382:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827036.413595:0:2382:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787827036.413598:0:2382:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1787827036.413599:0:2382:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1787827036.413601:0:2382:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1787827036.413603:0:2382:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1787827036.413604:0:2382:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827036.413605:0:2382:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1787827036.413717:0:2377:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1787827036.413722:0:2377:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff985607d95a00. 00000400:00000200:3.0:1787827036.413728:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1787827036.413746:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000400:00000200:3.0:1787827036.413751:0:2377:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985606618990 00000400:00000010:3.0:1787827036.413753:0:2377:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985606618990. 00000100:00000001:3.0:1787827036.413758:0:2377:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1787827036.413763:0:2377:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff9856140d8f00 x1874669228033024/t0(0) o400->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827052 ref 2 fl Rpc:r/2c0/ffffffff rc 0/-1 job:'ldlm_lock_repla.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827036.413775:0:2377:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1787827036.413778:0:2377:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9856140d8f00 x1874669228033024/t0(0) o400->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827052 ref 2 fl Rpc:Qr/2c0/ffffffff rc 0/-1 job:'ldlm_lock_repla.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827036.413786:0:2377:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827036.413788:0:2377:0:(events.c:68:request_out_callback()) Process leaving 00010000:00000010:2.1F:1787827036.419161:0:0:0:(ldlm_resource.c:1555:__ldlm_resource_free()) slab-freed '(res)': 256 at ffff9856091e7e00. 00010000:00000010:2.1:1787827036.419196:0:0:0:(ldlm_lock.c:174:lock_handle_free()) slab-freed 'lock': 512 at ffff985610e07800. 00000800:00000001:2.0:1787827036.482808:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827036.482840:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1787827036.482846:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827036.482849:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827036.482860:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827036.482870:0:2378:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp) <- 192.168.203.114@tcp : PUT - for me 00000400:00000200:2.0:1787827036.482881:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827036.482888:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827036.482894:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827036.482899:0:2378:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.203.114@tcp of length 224 into portal 4 MB=0x6a90071688400 00000400:00000200:2.0:1787827036.482904:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827036.482911:0:2378:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 4 from 12345-192.168.203.114@tcp of length 224/224 into md 0x3234d [1] + 224 00000800:00000001:2.0:1787827036.482920:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827036.482934:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827036.482938:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1787827036.482944:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000100:00000001:2.0:1787827036.482954:0:2378:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1787827036.482965:0:2378:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff9856140d8f00 x1874669228033024/t0(0) o400->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827052 ref 1 fl Rpc:Qr/2c0/ffffffff rc 0/-1 job:'ldlm_lock_repla.0' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1787827036.482979:0:2378:0:(events.c:149:reply_in_callback()) @@@ reply in flags=2c0 mlen=224 offset=224 replen=224 req@ffff9856140d8f00 x1874669228033024/t0(0) o400->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827052 ref 1 fl Rpc:RQ/2c0/ffffffff rc 0/-1 job:'ldlm_lock_repla.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827036.482999:0:2378:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1787827036.483009:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827036.483013:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787827036.483071:0:2382:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1787827036.483079:0:2382:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1787827036.483088:0:2382:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1787827036.483091:0:2382:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1787827036.483095:0:2382:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff985602ecf898. 00000400:00000200:1.0:1787827036.483104:0:2382:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985606618aa0 00000400:00000010:1.0:1787827036.483108:0:2382:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985606618aa0. 00000100:00000001:1.0:1787827036.483113:0:2382:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1787827036.483125:0:2382:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff9856140d8f00 x1874669228033024/t0(0) o400->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827052 ref 1 fl Rpc:RQ/2c0/ffffffff rc 0/-1 job:'ldlm_lock_repla.0' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1787827036.483137:0:2382:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff9856140d8f00 x1874669228033024/t0(0) o400->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827052 ref 1 fl Rpc:RQU/2c0/ffffffff rc 0/-1 job:'ldlm_lock_repla.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827036.483146:0:2382:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1787827036.483148:0:2382:0:(client.c:3001:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787827036.483150:0:2382:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:1.0:1787827036.483153:0:2382:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1787827036.483156:0:2382:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1787827036.483158:0:2382:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1787827036.483160:0:2382:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1787827036.483170:0:2382:0:(import.c:2149:obd_at_measure()) add 5 to ffff9856110b0460 time=114 v=5 (5 5 5 5) 00000100:00001000:1.0:1787827036.483182:0:2382:0:(import.c:2149:obd_at_measure()) add 1 to ffff9856110b0418 time=0 v=5 (1 0 0 0) 00000100:00000001:1.0:1787827036.483185:0:2382:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1787827036.483186:0:2382:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1787827036.483189:0:2382:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1787827036.483193:0:2382:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827036.483196:0:2382:0:(client.c:3109:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1787827036.483198:0:2382:0:(client.c:3121:ptlrpc_free_committed()) lustre-OST0000-osc-ffff985602bc9800: skip recheck: last_committed 8589935703 00000100:00000001:1.0:1787827036.483200:0:2382:0:(client.c:3122:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1787827036.483205:0:2382:0:(client.c:1797:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1787827036.483210:0:2382:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9856140d8f00 x1874669228033024/t0(0) o400->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827052 ref 1 fl Rpc:RQU/2c0/0 rc 0/0 job:'ldlm_lock_repla.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827036.483214:0:2382:0:(client.c:2354:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1787827036.483217:0:2382:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1787827036.483218:0:2382:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787827036.483220:0:2382:0:(import.c:1528:completed_replay_interpret()) Process entered 00000100:00000001:1.0:1787827036.483222:0:2382:0:(import.c:1648:ptlrpc_import_recovery_state_machine()) Process entered 00000100:00080000:1.0:1787827036.483224:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9856110b0000 lustre-OST0000_UUID: changing import state from REPLAY_WAIT to RECOVER 00000100:00000001:1.0:1787827036.483226:0:2382:0:(recover.c:149:ptlrpc_resend()) Process entered 00000100:00080000:1.0:1787827036.483229:0:2382:0:(client.c:3247:ptlrpc_resend_req()) @@@ going to resend req@ffff9856140d8f00 x1874669228033024/t0(0) o400->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827052 ref 1 fl Interpret:RQU/2c0/0 rc 0/0 job:'ldlm_lock_repla.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827036.483238:0:2382:0:(client.c:3256:ptlrpc_resend_req()) @@@ it has reply, so skip it req@ffff9856140d8f00 x1874669228033024/t0(0) o400->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827052 ref 1 fl Interpret:RQU/2c0/0 rc 0/0 job:'ldlm_lock_repla.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827036.483242:0:2382:0:(recover.c:178:ptlrpc_resend()) Process leaving (rc=0 : 0 : 0) 00000100:00080000:1.0:1787827036.483244:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9856110b0000 lustre-OST0000_UUID: changing import state from RECOVER to FULL 00000100:00000001:1.0:1787827036.483246:0:2382:0:(obd_class.h:1222:obd_import_event()) Process entered 00000008:00000001:1.0:1787827036.483249:0:2382:0:(osc_request.c:4091:osc_import_event()) Process entered 00000008:00000001:1.0:1787827036.483252:0:2382:0:(obd_class.h:1242:obd_notify()) Process entered 00020000:00000001:1.0:1787827036.483262:0:2382:0:(lov_obd.c:430:lov_notify()) Process entered 00020000:00000001:1.0:1787827036.483265:0:2382:0:(lov_obd.c:349:lov_set_osc_active()) Process entered 00020000:00000040:1.0:1787827036.483267:0:2382:0:(lov_obd.c:351:lov_set_osc_active()) Searching in lov ffff985606a35590 for uuid lustre-OST0000_UUID event(1) 00020000:00000040:1.0:1787827036.483272:0:2382:0:(lov_obd.c:394:lov_set_osc_active()) OSC lustre-OST0000_UUID already active! 00020000:00000001:1.0:1787827036.483274:0:2382:0:(lov_obd.c:395:lov_set_osc_active()) Process leaving via out (rc=0 : 0 : 0x0) 00020000:00000001:1.0:1787827036.483277:0:2382:0:(lov_obd.c:421:lov_set_osc_active()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:1.0:1787827036.483280:0:2382:0:(llite_lib.c:386:ll_upcall_notify()) Process entered 00000080:00000001:1.0:1787827036.483284:0:2382:0:(lcommon_misc.c:85:cl_ocd_update()) Process entered 00000080:00000004:1.0:1787827036.483286:0:2382:0:(lcommon_misc.c:94:cl_ocd_update()) Changing connect_flags: 0xa0427af2e3440038 -> 0xa0427af2e3440038 00000080:00000001:1.0:1787827036.483288:0:2382:0:(lcommon_misc.c:37:cl_init_ea_size()) Process entered 00000080:00000001:1.0:1787827036.483290:0:2382:0:(obd_class.h:511:obd_get_info()) Process entered 00020000:00000001:1.0:1787827036.483293:0:2382:0:(lov_obd.c:1160:lov_get_info()) Process entered 00020000:00000001:1.0:1787827036.483295:0:2382:0:(lov_obd.c:1211:lov_get_info()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:1.0:1787827036.483297:0:2382:0:(obd_class.h:525:obd_get_info()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:1.0:1787827036.483298:0:2382:0:(obd_class.h:511:obd_get_info()) Process entered 00020000:00000001:1.0:1787827036.483299:0:2382:0:(lov_obd.c:1160:lov_get_info()) Process entered 00020000:00000001:1.0:1787827036.483300:0:2382:0:(lov_obd.c:1211:lov_get_info()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:1.0:1787827036.483301:0:2382:0:(obd_class.h:525:obd_get_info()) Process leaving (rc=0 : 0 : 0) 00000080:00080000:1.0:1787827036.483302:0:2382:0:(lcommon_misc.c:56:cl_init_ea_size()) updating def/max_easize: 72/65536 00800000:00000001:1.0:1787827036.483305:0:2382:0:(lmv_obd.c:286:lmv_init_ea_size()) Process entered 00800000:00000001:1.0:1787827036.483306:0:2382:0:(lmv_obd.c:298:lmv_init_ea_size()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:1.0:1787827036.483307:0:2382:0:(lcommon_misc.c:59:cl_init_ea_size()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:1.0:1787827036.483308:0:2382:0:(lcommon_misc.c:112:cl_ocd_update()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:1.0:1787827036.483309:0:2382:0:(llite_lib.c:411:ll_upcall_notify()) Process leaving (rc=0 : 0 : 0) 00020000:00000001:1.0:1787827036.483311:0:2382:0:(lov_obd.c:467:lov_notify()) Process leaving (rc=0 : 0 : 0) 00000008:00000001:1.0:1787827036.483312:0:2382:0:(obd_class.h:1255:obd_notify()) Process leaving (rc=0 : 0 : 0) 00000008:00000001:1.0:1787827036.483314:0:2382:0:(osc_request.c:4158:osc_import_event()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827036.483316:0:2382:0:(obd_class.h:1233:obd_import_event()) Process leaving 00000100:00080000:1.0:1787827036.483319:0:2382:0:(import.c:1743:ptlrpc_import_recovery_state_machine()) lustre-OST0000-osc-ffff985602bc9800: Connection restored to 192.168.203.114@tcp (at 192.168.203.114@tcp) 00000100:00000001:1.0:1787827036.483322:0:2382:0:(import.c:1754:ptlrpc_import_recovery_state_machine()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827036.483325:0:2382:0:(import.c:1546:completed_replay_interpret()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1787827036.483329:0:2382:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9856140d8f00 x1874669228033024/t0(0) o400->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827052 ref 1 fl Interpret:RQU/2c0/0 rc 0/0 job:'ldlm_lock_repla.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827036.483337:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228033024:192.168.203.114@tcp:400:ldlm_lock_repla.0 00000100:00000001:1.0:1787827036.483341:0:2382:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787827036.483344:0:2382:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1787827036.483347:0:2382:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9856140d8f00 x1874669228033024/t0(0) o400->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827052 ref 1 fl Complete:RQU/2c0/0 rc 0/0 job:'ldlm_lock_repla.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827036.483353:0:2382:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1787827036.483355:0:2382:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1787827036.483358:0:2382:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff985610db9800. 02000000:00000001:1.0:1787827036.483362:0:2382:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1787827036.483365:0:2382:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:1.0:1787827036.483367:0:2382:0:(genops.c:1223:class_import_put()) import ffff9856110b0000 refcount=2 obd=lustre-OST0000-osc-ffff985602bc9800 00000020:00000001:1.0:1787827036.483369:0:2382:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:1.0:1787827036.483371:0:2382:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff985615779b00. 02000000:00000001:1.0:1787827036.483375:0:2382:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1787827036.483377:0:2382:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1787827036.483379:0:2382:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff9856140d8f00. 00000100:00000001:1.0:1787827036.483383:0:2382:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1787827036.483385:0:2382:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787827036.483388:0:2382:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787827036.483395:0:2382:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1787827036.483397:0:2382:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827036.483399:0:2382:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1787827036.483401:0:2382:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827036.980796:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827036.980801:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827036.980805:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827036.980807:0:49842:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:0.0:1787827036.980809:0:49842:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:0.0:1787827036.980811:0:49842:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827036.980813:0:49842:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:0.0:1787827036.980815:0:49842:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827036.980816:0:49842:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827036.980818:0:49842:0:(ldlm_pool.c:496:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827036.980820:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827036.980821:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1787827036.980888:0:79887:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1787827036.980891:0:79887:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1787827036.980895:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1787827037.428773:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1787827037.428783:0:37:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1787827037.428795:0:37:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 904 at ffff985607f08b40. 00000020:00000040:0.0:1787827037.428801:0:37:0:(genops.c:1210:class_import_get()) import ffff98561118e000 refcount=4 obd=MGC192.168.203.114@tcp 00000100:00000001:0.0:1787827037.428805:0:37:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1787827037.428808:0:37:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1787827037.428813:0:37:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1787827037.428821:0:37:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9856030d2300. 00000100:00000001:0.0:1787827037.428827:0:37:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787827037.428838:0:37:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS req@ffff985607f08b40 x1874669228033152/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1787827037.428850:0:37:0:(jobid.c:939:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1787827037.428855:0:37:0:(jobid.c:979:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787827037.428861:0:37:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000008e93a775] to pc [ptlrpcd_00_01+1] req@ffff985607f08b40 x1874669228033152/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787827037.428897:0:37:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787827037.428901:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1787827037.428904:0:37:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1787827037.428907:0:37:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 904 at ffff985607f0a940. 00000020:00000040:0.0:1787827037.428910:0:37:0:(genops.c:1210:class_import_get()) import ffff985611182000 refcount=4 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000100:00000001:0.0:1787827037.428913:0:37:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1787827037.428914:0:37:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1787827037.428916:0:37:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1787827037.428919:0:37:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9856030d2500. 00000100:00000001:1.0:1787827037.428921:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1787827037.428922:0:37:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827037.428926:0:2384:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000040:0.0:1787827037.428929:0:37:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID req@ffff985607f0a940 x1874669228033280/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1787827037.428930:0:2384:0:(client.c:1813:ptlrpc_send_new_req()) Process entered 00000100:00000001:0.0:1787827037.428940:0:37:0:(jobid.c:939:lustre_get_jobid()) Process entered 00000100:00000040:1.0:1787827037.428941:0:2384:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff985607f08b40 x1874669228033152/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787827037.428943:0:37:0:(jobid.c:979:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787827037.428952:0:37:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000083e565bf] to pc [ptlrpcd_00_00+0] req@ffff985607f0a940 x1874669228033280/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787827037.428975:0:37:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827037.428976:0:2384:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000040:0.0:1787827037.428980:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1787827037.428981:0:2384:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787827037.428987:0:37:0:(pinger.c:119:ptlrpc_ping()) Process entered 02000000:00000001:1.0:1787827037.428988:0:2384:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000010:0.0:1787827037.428991:0:37:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 904 at ffff985607f0a1c0. 02000000:00000001:1.0:1787827037.428992:0:2384:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000020:00000040:0.0:1787827037.428996:0:37:0:(genops.c:1210:class_import_get()) import ffff985611185800 refcount=4 obd=lustre-MDT0001-mdc-ffff985602bc9800 02000000:00000001:1.0:1787827037.428997:0:2384:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787827037.429002:0:37:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 00000100:00100000:1.0:1787827037.429004:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607f08b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:3c946890-15ca-497a-b838-fe1cba61f4dd:2384:1874669228033152:192.168.203.114@tcp:400:kworker.0 02000000:00000001:0.0:1787827037.429005:0:37:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1787827037.429008:0:37:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827037.429012:0:2384:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000010:0.0:1787827037.429014:0:37:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9856030d2e00. 00000100:00000001:0.0:1787827037.429018:0:37:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1787827037.429018:0:2384:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1787827037.429022:0:2384:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787827037.429026:0:37:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID req@ffff985607f0a1c0 x1874669228033408/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 02000000:00000001:1.0:1787827037.429027:0:2384:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:0.0:1787827037.429034:0:37:0:(jobid.c:939:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1787827037.429036:0:37:0:(jobid.c:979:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787827037.429041:0:37:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000009dcd1e7d] to pc [ptlrpcd_00_01+1] req@ffff985607f0a1c0 x1874669228033408/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 02000000:00000010:1.0:1787827037.429042:0:2384:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff985610db8e00. 02000000:00000001:1.0:1787827037.429047:0:2384:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787827037.429055:0:37:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1787827037.429057:0:2384:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000040:0.0:1787827037.429059:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-OST0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1787827037.429064:0:37:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000400:00000010:1.0:1787827037.429064:0:2384:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff985602ecf898. 00000100:00000010:0.0:1787827037.429069:0:37:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 904 at ffff985607f0a580. 00000400:00000010:1.0:1787827037.429073:0:2384:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985606618aa0. 00000020:00000040:0.0:1787827037.429074:0:37:0:(genops.c:1210:class_import_get()) import ffff9856110b0000 refcount=3 obd=lustre-OST0000-osc-ffff985602bc9800 00000100:00000001:0.0:1787827037.429078:0:37:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1787827037.429080:0:37:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 00000100:00000200:1.0:1787827037.429081:0:2384:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1874669228033152, portal 25 02000000:00000001:0.0:1787827037.429082:0:37:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827037.429086:0:2384:0:(client.c:3275:ptlrpc_request_addref()) Process entered 02000000:00000010:0.0:1787827037.429089:0:37:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9856030d2900. 00000100:00000001:1.0:1787827037.429089:0:2384:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446630094000655168 : -113979708896448 : ffff985607f08b40) 00000100:00000001:0.0:1787827037.429095:0:37:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1787827037.429099:0:2384:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff985607f08b40 x1874669228033152/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827053 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1787827037.429104:0:37:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-OST0000_UUID req@ffff985607f0a580 x1874669228033536/t0(0) o400->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1787827037.429116:0:2384:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:0.0:1787827037.429120:0:37:0:(jobid.c:939:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1787827037.429122:0:37:0:(jobid.c:979:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787827037.429127:0:37:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000047c110d7] to pc [ptlrpcd_00_00+0] req@ffff985607f0a580 x1874669228033536/t0(0) o400->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1787827037.429133:0:2384:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.203.114@tcp 00000400:00000010:1.0:1787827037.429142:0:2384:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985606618198. 00000100:00000200:1.0:1787827037.429148:0:2384:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1874669228033152, offset 0 00000100:00000001:0.0:1787827037.429151:0:37:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787827037.429155:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-OST0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000400:00000200:1.0:1787827037.429157:0:2384:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.203.114@tcp 00000100:00000001:0.0:1787827037.429162:0:37:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000001:0.0:1787827037.429165:0:37:0:(import.c:2066:ptlrpc_disconnect_and_idle_import()) Process entered 00000100:00000001:0.0:1787827037.429169:0:37:0:(import.c:1762:ptlrpc_disconnect_prep_req()) Process entered 00000400:00000200:1.0:1787827037.429169:0:2384:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000010:0.0:1787827037.429174:0:37:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 904 at ffff985607f08f00. 00000020:00000040:0.0:1787827037.429179:0:37:0:(genops.c:1210:class_import_get()) import ffff9856110b1000 refcount=3 obd=lustre-OST0001-osc-ffff985602bc9800 00000400:00000200:1.0:1787827037.429179:0:2384:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.203.114@tcp local destination 00000100:00000001:0.0:1787827037.429184:0:37:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1787827037.429186:0:37:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 00000400:00000200:1.0:1787827037.429187:0:2384:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 02000000:00000001:0.0:1787827037.429188:0:37:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1787827037.429193:0:2384:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.203.14@tcp 02000000:00000010:0.0:1787827037.429195:0:37:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff98561238ea00. 00000100:00000001:0.0:1787827037.429199:0:37:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787827037.429201:0:37:0:(import.c:1800:ptlrpc_disconnect_prep_req()) Process leaving (rc=18446630094000656128 : -113979708895488 : ffff985607f08f00) 00000100:00080000:0.0:1787827037.429205:0:37:0:(import.c:65:import_set_state_nolock()) ffff9856110b1000 lustre-OST0001_UUID: changing import state from FULL to CONNECTING 00000400:00000200:1.0:1787827037.429206:0:2384:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.203.14@tcp lpni_is_preferred = 0 00000100:00080000:0.0:1787827037.429208:0:37:0:(import.c:2101:ptlrpc_disconnect_and_idle_import()) lustre-OST0001-osc-ffff985602bc9800: disconnect after 23s idle 00000400:00000200:1.0:1787827037.429211:0:2384:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.203.114@tcp 00000100:00000001:0.0:1787827037.429213:0:37:0:(jobid.c:939:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1787827037.429216:0:37:0:(jobid.c:979:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1787827037.429219:0:2384:0:(lib-move.c:1745:lnet_handle_send()) 192.168.203.14@tcp NI seq info: [21897:21897:256:4294967295] 192.168.203.114@tcp LPNI seq info [21897:21897:8:4294967295] 00000100:00000040:0.0:1787827037.429224:0:37:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000002a2b9b7a] to pc [ptlrpcd_rcv+-1] req@ffff985607f08f00 x1874669228033664/t0(0) o9->lustre-OST0001-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1787827037.429228:0:2384:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:1.0:1787827037.429242:0:2384:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp:) -> 192.168.203.114@tcp(192.168.203.114@tcp:192.168.203.114@tcp) : PUT try# 0 00000800:00000200:1.0:1787827037.429247:0:2384:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.203.114@tcp 00000100:00000001:0.0:1787827037.429251:0:37:0:(import.c:2105:ptlrpc_disconnect_and_idle_import()) Process leaving (rc=1 : 1 : 1) 00000800:00000010:1.0:1787827037.429253:0:2384:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff985607d95300. 00000100:00000001:0.0:1787827037.429256:0:37:0:(pinger.c:123:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787827037.429262:0:37:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (3193) 00000800:00000200:1.0:1787827037.429263:0:2384:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff985606751480] -> 12345-192.168.203.114@tcp (5) 00000800:00000200:1.0:1787827037.429272:0:2384:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.203.114@tcp ip 192.168.203.114:988 00000100:00000001:0.0:1787827037.429275:0:2383:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000200:1.0:1787827037.429277:0:2384:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff985607d95300 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1787827037.429278:0:2383:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1787827037.429280:0:2382:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1787827037.429282:0:2383:0:(client.c:1813:ptlrpc_send_new_req()) Process entered 00000100:00000001:2.0:1787827037.429283:0:2382:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1787827037.429286:0:2382:0:(client.c:1813:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1787827037.429289:0:2383:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff985607f0a940 x1874669228033280/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827037.429292:0:2384:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1787827037.429294:0:2382:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff985607f08f00 x1874669228033664/t0(0) o9->lustre-OST0001-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1787827037.429299:0:2384:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000001:1.0:1787827037.429303:0:2384:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.203.114@tcp: PM QoS new deadline estimation for cpu 2 is 4297865468 00000100:00000001:1.0:1787827037.429311:0:2384:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.203.114@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4297864446 00000100:00000001:0.0:1787827037.429313:0:2383:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1787827037.429314:0:2382:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1787827037.429316:0:2383:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827037.429317:0:2384:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.203.114@tcp: PM QoS new deadline estimation for cpu 3 is 4297865468 00000100:00000001:2.0:1787827037.429317:0:2382:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1787827037.429322:0:2383:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1787827037.429324:0:2382:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:1.0:1787827037.429325:0:2384:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.203.114@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4297864446 02000000:00000001:0.0:1787827037.429326:0:2383:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1787827037.429327:0:2382:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1787827037.429328:0:2384:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1787827037.429333:0:2383:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1787827037.429333:0:2382:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827037.429334:0:2384:0:(client.c:1941:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1787827037.429342:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607f08f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228033664:192.168.203.114@tcp:9:kworker.0 00000100:00100000:0.0:1787827037.429343:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607f0a940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669228033280:192.168.203.114@tcp:400:kworker.0 00000100:00000001:1.0:1787827037.429344:0:2384:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827037.429348:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1787827037.429348:0:2382:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:0.0:1787827037.429351:0:2383:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1787827037.429353:0:2382:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1787827037.429356:0:2383:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:1.0:1787827037.429357:0:2384:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 02000000:00000001:0.0:1787827037.429359:0:2383:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1787827037.429359:0:2382:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827037.429360:0:2384:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 02000000:00000001:0.0:1787827037.429364:0:2383:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:1.0:1787827037.429365:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:2.0:1787827037.429365:0:2382:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:1.0:1787827037.429368:0:2384:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1787827037.429369:0:2384:0:(client.c:1813:ptlrpc_send_new_req()) Process entered 02000000:00000010:2.0:1787827037.429371:0:2382:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff985607e32c00. 02000000:00000010:0.0:1787827037.429372:0:2383:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff98560540a400. 00000100:00000040:1.0:1787827037.429374:0:2384:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff985607f0a1c0 x1874669228033408/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1787827037.429374:0:2382:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1787827037.429377:0:2383:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1787827037.429381:0:2382:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:0.0:1787827037.429387:0:2383:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000010:2.0:1787827037.429387:0:2382:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff98561080e420. 00000100:00000001:1.0:1787827037.429391:0:2384:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000400:00000010:2.0:1787827037.429392:0:2382:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985610afa660. 00000100:00000001:1.0:1787827037.429393:0:2384:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1787827037.429394:0:2383:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9856094ac370. 00000100:00000200:2.0:1787827037.429395:0:2382:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1874669228033664, portal 4 00000100:00000001:2.0:1787827037.429399:0:2382:0:(client.c:3275:ptlrpc_request_addref()) Process entered 02000000:00000001:1.0:1787827037.429400:0:2384:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:2.0:1787827037.429401:0:2382:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446630094000656128 : -113979708895488 : ffff985607f08f00) 00000400:00000010:0.0:1787827037.429402:0:2383:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985610d30880. 02000000:00000001:1.0:1787827037.429402:0:2384:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1787827037.429407:0:2384:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1787827037.429409:0:2383:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1874669228033280, portal 10 00000100:00000040:2.0:1787827037.429409:0:2382:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff985607f08f00 x1874669228033664/t0(0) o9->lustre-OST0001-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827047 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000800:00000200:3.0:1787827037.429409:0:2377:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00100000:1.0:1787827037.429412:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607f0a1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669228033408:192.168.203.114@tcp:400:kworker.0 00000100:00000001:0.0:1787827037.429416:0:2383:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000800:00000010:3.0:1787827037.429416:0:2377:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff985607d95300. 00000100:00000001:1.0:1787827037.429418:0:2384:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:0.0:1787827037.429421:0:2383:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446630094000662848 : -113979708888768 : ffff985607f0a940) 02000000:00000001:1.0:1787827037.429422:0:2384:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:2.0:1787827037.429422:0:2382:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000001:1.0:1787827037.429425:0:2384:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1787827037.429427:0:2382:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.203.114@tcp 02000000:00000001:1.0:1787827037.429428:0:2384:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000200:3.0:1787827037.429429:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000010:2.0:1787827037.429430:0:2382:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985610afa550. 02000000:00000010:1.0:1787827037.429432:0:2384:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff985610db8c00. 00000100:00000040:0.0:1787827037.429433:0:2383:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff985607f0a940 x1874669228033280/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827053 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1787827037.429435:0:2384:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:2.0:1787827037.429435:0:2382:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 28, xid 1874669228033664, offset 0 00000400:00000200:3.0:1787827037.429442:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000400:00000200:1.0:1787827037.429444:0:2384:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827037.429446:0:2382:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.203.114@tcp 00000400:00000010:1.0:1787827037.429449:0:2384:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff985602ecf9f8. 00000400:00000200:3.0:1787827037.429453:0:2377:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985606618198 00000400:00000010:1.0:1787827037.429455:0:2384:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985606618110. 00000400:00000200:2.0:1787827037.429456:0:2382:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000010:3.0:1787827037.429460:0:2377:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985606618198. 00000100:00000001:0.0:1787827037.429461:0:2383:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000200:1.0:1787827037.429461:0:2384:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1874669228033408, portal 10 00000400:00000200:2.0:1787827037.429464:0:2382:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.203.114@tcp local destination 00000100:00000001:1.0:1787827037.429465:0:2384:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1787827037.429467:0:2384:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446630094000660928 : -113979708890688 : ffff985607f0a1c0) 00000400:00000200:2.0:1787827037.429468:0:2382:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:3.0:1787827037.429468:0:2377:0:(events.c:34:request_out_callback()) Process entered 00000100:00000040:0.0:1787827037.429470:0:2383:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.203.114@tcp 00000400:00000010:0.0:1787827037.429476:0:2383:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985610d30c38. 00000400:00000200:2.0:1787827037.429476:0:2382:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.203.14@tcp 00000100:00000040:1.0:1787827037.429479:0:2384:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff985607f0a1c0 x1874669228033408/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827053 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1787827037.429479:0:2382:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.203.14@tcp lpni_is_preferred = 0 00000100:00000200:3.0:1787827037.429479:0:2377:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff985607f08b40 x1874669228033152/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827053 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1787827037.429484:0:2383:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1874669228033280, offset 0 00000400:00000200:2.0:1787827037.429485:0:2382:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.203.114@tcp 00000400:00000200:2.0:1787827037.429492:0:2382:0:(lib-move.c:1745:lnet_handle_send()) 192.168.203.14@tcp NI seq info: [21898:21898:255:4294967295] 192.168.203.114@tcp LPNI seq info [21898:21898:7:4294967295] 00000400:00000200:0.0:1787827037.429497:0:2383:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.203.114@tcp 00000400:00000200:2.0:1787827037.429503:0:2382:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000001:1.0:1787827037.429507:0:2384:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000400:00000200:0.0:1787827037.429510:0:2383:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000040:1.0:1787827037.429512:0:2384:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.203.114@tcp 00000100:00000001:3.0:1787827037.429514:0:2377:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000400:00000200:2.0:1787827037.429518:0:2382:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp:) -> 192.168.203.114@tcp(192.168.203.114@tcp:192.168.203.114@tcp) : PUT try# 0 00000400:00000200:0.0:1787827037.429521:0:2383:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.203.114@tcp local destination 00000400:00000010:1.0:1787827037.429523:0:2384:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985606618b28. 00000100:00000040:3.0:1787827037.429524:0:2377:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff985607f08b40 x1874669228033152/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827053 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1787827037.429528:0:2384:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1874669228033408, offset 0 00000800:00000200:2.0:1787827037.429528:0:2382:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.203.114@tcp 00000400:00000200:0.0:1787827037.429529:0:2383:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000800:00000010:2.0:1787827037.429535:0:2382:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff985615779a00. 00000400:00000200:0.0:1787827037.429537:0:2383:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.203.14@tcp 00000400:00000200:1.0:1787827037.429541:0:2384:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.203.114@tcp 00000800:00000200:2.0:1787827037.429543:0:2382:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff985606751480] -> 12345-192.168.203.114@tcp (5) 00000400:00000200:0.0:1787827037.429545:0:2383:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.203.14@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1787827037.429552:0:2383:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.203.114@tcp 00000800:00000200:2.0:1787827037.429552:0:2382:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.203.114@tcp ip 192.168.203.114:988 00000800:00000200:2.0:1787827037.429558:0:2382:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff985615779a00 type 1, nob 320 niov 1 nkiov 1 00000400:00000200:0.0:1787827037.429559:0:2383:0:(lib-move.c:1745:lnet_handle_send()) 192.168.203.14@tcp NI seq info: [21899:21899:254:4294967295] 192.168.203.114@tcp LPNI seq info [21899:21899:6:4294967295] 00000100:00000001:2.0:1787827037.429562:0:2382:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1787827037.429568:0:2383:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000040:2.0:1787827037.429568:0:2382:0:(niobuf.c:635:kick_cpu_latency()) lustre-OST0001-osc-ffff985602bc9800: using a timeout of 13836 usecs (14 jiffies) 00000400:00000200:1.0:1787827037.429574:0:2384:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827037.429574:0:2382:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000001:2.0:1787827037.429578:0:2382:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0001-osc-ffff985602bc9800: PM QoS new deadline estimation for cpu 2 is 4297855502 00000400:00000200:1.0:1787827037.429579:0:2384:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.203.114@tcp local destination 00000100:00000001:2.0:1787827037.429582:0:2382:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0001-osc-ffff985602bc9800: PM QoS request active for cpu 2, keep current deadline 4297865468 00000400:00000200:1.0:1787827037.429585:0:2384:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:2.0:1787827037.429585:0:2382:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0001-osc-ffff985602bc9800: PM QoS new deadline estimation for cpu 3 is 4297855502 00000100:00000001:2.0:1787827037.429588:0:2382:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0001-osc-ffff985602bc9800: PM QoS request active for cpu 3, keep current deadline 4297865468 00000400:00000200:0.0:1787827037.429589:0:2383:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp:) -> 192.168.203.114@tcp(192.168.203.114@tcp:192.168.203.114@tcp) : PUT try# 0 00000400:00000200:1.0:1787827037.429590:0:2384:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.203.14@tcp 00000100:00000001:2.0:1787827037.429591:0:2382:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1787827037.429594:0:2382:0:(client.c:1941:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1787827037.429596:0:2384:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.203.14@tcp lpni_is_preferred = 0 00000100:00000001:2.0:1787827037.429599:0:2382:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1787827037.429600:0:2384:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.203.114@tcp 00000100:00000001:2.0:1787827037.429600:0:2382:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000800:00000200:0.0:1787827037.429605:0:2383:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.203.114@tcp 00000100:00000001:2.0:1787827037.429606:0:2382:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000400:00000200:1.0:1787827037.429608:0:2384:0:(lib-move.c:1745:lnet_handle_send()) 192.168.203.14@tcp NI seq info: [21900:21900:253:4294967295] 192.168.203.114@tcp LPNI seq info [21900:21900:5:4294967295] 00000100:00000001:2.0:1787827037.429609:0:2382:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=5 : 5 : 5) 00000800:00000010:0.0:1787827037.429614:0:2383:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff98561238e200. 00000100:00000001:2.0:1787827037.429614:0:2382:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1787827037.429616:0:2382:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000400:00000200:1.0:1787827037.429619:0:2384:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000001:2.0:1787827037.429621:0:2382:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1787827037.429623:0:2382:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827037.429624:0:2377:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1787827037.429628:0:2383:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff985606751480] -> 12345-192.168.203.114@tcp (5) 00000100:00000001:3.0:1787827037.429630:0:2377:0:(events.c:68:request_out_callback()) Process leaving 00000400:00000200:1.0:1787827037.429635:0:2384:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp:) -> 192.168.203.114@tcp(192.168.203.114@tcp:192.168.203.114@tcp) : PUT try# 0 00000800:00000200:0.0:1787827037.429641:0:2383:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.203.114@tcp ip 192.168.203.114:988 00000800:00000200:1.0:1787827037.429644:0:2384:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.203.114@tcp 00000800:00000200:0.0:1787827037.429646:0:2383:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff98561238e200 type 1, nob 320 niov 1 nkiov 1 00000800:00000010:1.0:1787827037.429649:0:2384:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff985607d95500. 00000100:00000001:0.0:1787827037.429652:0:2383:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1787827037.429654:0:2377:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000040:0.0:1787827037.429657:0:2383:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff985602bc9800: using a timeout of 38050 usecs (39 jiffies) 00000800:00000010:3.0:1787827037.429658:0:2377:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff985615779a00. 00000800:00000200:1.0:1787827037.429659:0:2384:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff985606751480] -> 12345-192.168.203.114@tcp (5) 00000400:00000200:3.0:1787827037.429664:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1787827037.429665:0:2383:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000001:0.0:1787827037.429670:0:2383:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff985602bc9800: PM QoS new deadline estimation for cpu 2 is 4297855527 00000400:00000200:3.0:1787827037.429671:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000100:00000001:0.0:1787827037.429677:0:2383:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff985602bc9800: PM QoS request active for cpu 2, keep current deadline 4297865468 00000400:00000200:3.0:1787827037.429678:0:2377:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985610afa550 00000800:00000200:1.0:1787827037.429683:0:2384:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.203.114@tcp ip 192.168.203.114:988 00000100:00000001:0.0:1787827037.429684:0:2383:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff985602bc9800: PM QoS new deadline estimation for cpu 3 is 4297855527 00000400:00000010:3.0:1787827037.429684:0:2377:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985610afa550. 00000100:00000001:0.0:1787827037.429714:0:2383:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff985602bc9800: PM QoS request active for cpu 3, keep current deadline 4297865468 00000800:00000200:1.0:1787827037.429715:0:2384:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff985607d95500 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1787827037.429716:0:2377:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:0.0:1787827037.429719:0:2383:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1787827037.429721:0:2384:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787827037.429724:0:2383:0:(client.c:1941:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:3.0:1787827037.429737:0:2377:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff985607f08f00 x1874669228033664/t0(0) o9->lustre-OST0001-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827047 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1787827037.429742:0:2384:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff985602bc9800: using a timeout of 11952 usecs (12 jiffies) 00000100:00000001:0.0:1787827037.429743:0:2383:0:(client.c:1813:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1787827037.429752:0:2383:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff985607f0a580 x1874669228033536/t0(0) o400->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1787827037.429754:0:2384:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000001:3.0:1787827037.429755:0:2377:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000001:1.0:1787827037.429760:0:2384:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff985602bc9800: PM QoS new deadline estimation for cpu 2 is 4297855501 00000100:00000040:3.0:1787827037.429764:0:2377:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff985607f08f00 x1874669228033664/t0(0) o9->lustre-OST0001-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827047 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827037.429766:0:2384:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff985602bc9800: PM QoS request active for cpu 2, keep current deadline 4297865468 00000100:00000001:0.0:1787827037.429769:0:2383:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1787827037.429773:0:2383:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827037.429773:0:2384:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff985602bc9800: PM QoS new deadline estimation for cpu 3 is 4297855501 00000100:00000001:3.0:1787827037.429776:0:2377:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827037.429779:0:2384:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff985602bc9800: PM QoS request active for cpu 3, keep current deadline 4297865468 02000000:00000001:0.0:1787827037.429780:0:2383:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:3.0:1787827037.429780:0:2377:0:(events.c:68:request_out_callback()) Process leaving 02000000:00000001:0.0:1787827037.429785:0:2383:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1787827037.429785:0:2384:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000800:00000200:3.0:1787827037.429791:0:2377:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 02000000:00000001:0.0:1787827037.429792:0:2383:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827037.429792:0:2384:0:(client.c:1941:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:3.0:1787827037.429794:0:2377:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff98561238e200. 00000100:00000001:1.0:1787827037.429799:0:2384:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1787827037.429799:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00100000:0.0:1787827037.429800:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607f0a580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669228033536:192.168.203.114@tcp:400:kworker.0 00000100:00000001:1.0:1787827037.429805:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1787827037.429805:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000100:00000001:0.0:1787827037.429808:0:2383:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000200:3.0:1787827037.429812:0:2377:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985610d30c38 02000000:00000001:0.0:1787827037.429813:0:2383:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:1.0:1787827037.429814:0:2384:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000400:00000010:3.0:1787827037.429814:0:2377:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985610d30c38. 02000000:00000001:0.0:1787827037.429817:0:2383:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827037.429818:0:2384:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1787827037.429818:0:2377:0:(events.c:34:request_out_callback()) Process entered 02000000:00000001:0.0:1787827037.429822:0:2383:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:1.0:1787827037.429824:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000200:3.0:1787827037.429824:0:2377:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff985607f0a940 x1874669228033280/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827053 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 02000000:00000010:0.0:1787827037.429828:0:2383:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff98560540ac00. 00000100:00000001:1.0:1787827037.429828:0:2384:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1787827037.429832:0:2384:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1787827037.429833:0:2383:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827037.429837:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827037.429840:0:2377:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000400:00000200:0.0:1787827037.429846:0:2383:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000040:3.0:1787827037.429848:0:2377:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff985607f0a940 x1874669228033280/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827053 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000400:00000010:0.0:1787827037.429852:0:2383:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9856094acf20. 00000400:00000010:0.0:1787827037.429859:0:2383:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985610d30660. 00000100:00000001:3.0:1787827037.429860:0:2377:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827037.429864:0:2377:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000200:0.0:1787827037.429865:0:2383:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1874669228033536, portal 4 00000100:00000001:0.0:1787827037.429869:0:2383:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1787827037.429871:0:2383:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446630094000661888 : -113979708889728 : ffff985607f0a580) 00000100:00000040:0.0:1787827037.429877:0:2383:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff985607f0a580 x1874669228033536/t0(0) o400->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827053 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787827037.429884:0:2383:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1787827037.429887:0:2383:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.203.114@tcp 00000400:00000010:0.0:1787827037.429890:0:2383:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985610d30110. 00000100:00000200:0.0:1787827037.429893:0:2383:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 28, xid 1874669228033536, offset 0 00000800:00000200:3.0:1787827037.429896:0:2377:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000400:00000200:0.0:1787827037.429899:0:2383:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.203.114@tcp 00000800:00000010:3.0:1787827037.429901:0:2377:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff985607d95500. 00000400:00000200:0.0:1787827037.429906:0:2383:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:3.0:1787827037.429907:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1787827037.429912:0:2383:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.203.114@tcp local destination 00000400:00000200:3.0:1787827037.429915:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000400:00000200:0.0:1787827037.429918:0:2383:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1787827037.429922:0:2383:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.203.14@tcp 00000400:00000200:0.0:1787827037.429924:0:2383:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.203.14@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1787827037.429927:0:2383:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.203.114@tcp 00000400:00000200:0.0:1787827037.429930:0:2383:0:(lib-move.c:1745:lnet_handle_send()) 192.168.203.14@tcp NI seq info: [21901:21901:255:4294967295] 192.168.203.114@tcp LPNI seq info [21901:21901:7:4294967295] 00000400:00000200:0.0:1787827037.429934:0:2383:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:0.0:1787827037.429940:0:2383:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp:) -> 192.168.203.114@tcp(192.168.203.114@tcp:192.168.203.114@tcp) : PUT try# 0 00000800:00000200:0.0:1787827037.429945:0:2383:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.203.114@tcp 00000800:00000010:0.0:1787827037.429948:0:2383:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff98561238e000. 00000800:00000200:0.0:1787827037.429953:0:2383:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff985606751480] -> 12345-192.168.203.114@tcp (5) 00000800:00000200:0.0:1787827037.429958:0:2383:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.203.114@tcp ip 192.168.203.114:988 00000800:00000200:0.0:1787827037.429960:0:2383:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff98561238e000 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1787827037.429963:0:2383:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787827037.429966:0:2383:0:(niobuf.c:635:kick_cpu_latency()) lustre-OST0000-osc-ffff985602bc9800: using a timeout of 14650 usecs (15 jiffies) 00000400:00000200:0.0:1787827037.429969:0:2383:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000001:0.0:1787827037.429971:0:2383:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff985602bc9800: PM QoS new deadline estimation for cpu 2 is 4297855504 00000100:00000001:0.0:1787827037.429973:0:2383:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff985602bc9800: PM QoS request active for cpu 2, keep current deadline 4297865468 00000100:00000001:0.0:1787827037.429975:0:2383:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff985602bc9800: PM QoS new deadline estimation for cpu 3 is 4297855504 00000100:00000001:0.0:1787827037.429977:0:2383:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff985602bc9800: PM QoS request active for cpu 3, keep current deadline 4297865468 00000100:00000001:0.0:1787827037.429980:0:2383:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1787827037.429982:0:2383:0:(client.c:1941:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787827037.429984:0:2383:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787827037.429985:0:2383:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1787827037.429989:0:2383:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1787827037.429990:0:2383:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1787827037.429992:0:2383:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1787827037.429993:0:2383:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1787827037.429995:0:2383:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787827037.429997:0:2383:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1787827037.430036:0:2377:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985606618b28 00000400:00000010:3.0:1787827037.430038:0:2377:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985606618b28. 00000100:00000001:3.0:1787827037.430041:0:2377:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1787827037.430045:0:2377:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff985607f0a1c0 x1874669228033408/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827053 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827037.430052:0:2377:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1787827037.430055:0:2377:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff985607f0a1c0 x1874669228033408/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827053 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827037.430060:0:2377:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827037.430061:0:2377:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1787827037.430098:0:2377:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1787827037.430100:0:2377:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff98561238e000. 00000400:00000200:3.0:1787827037.430103:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1787827037.430106:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000400:00000200:3.0:1787827037.430109:0:2377:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985610d30110 00000400:00000010:3.0:1787827037.430110:0:2377:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985610d30110. 00000100:00000001:3.0:1787827037.430112:0:2377:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1787827037.430115:0:2377:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff985607f0a580 x1874669228033536/t0(0) o400->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827053 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827037.430121:0:2377:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1787827037.430124:0:2377:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff985607f0a580 x1874669228033536/t0(0) o400->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827053 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827037.430130:0:2377:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827037.430131:0:2377:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:3.0:1787827037.430391:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1787827037.430401:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1787827037.430404:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1787827037.430407:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1787827037.430412:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:3.0:1787827037.430417:0:2377:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp) <- 192.168.203.114@tcp : PUT - for me 00000400:00000200:3.0:1787827037.430424:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:3.0:1787827037.430428:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:3.0:1787827037.430432:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:3.0:1787827037.430436:0:2377:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.203.114@tcp of length 224 into portal 25 MB=0x6a90071688480 00000400:00000200:3.0:1787827037.430440:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:3.0:1787827037.430445:0:2377:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.203.114@tcp of length 224/224 into md 0x32355 [1] + 224 00000800:00000001:3.0:1787827037.430450:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1787827037.430456:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1787827037.430458:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1787827037.430462:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000100:00000001:3.0:1787827037.430466:0:2377:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1787827037.430471:0:2377:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff985607f08b40 x1874669228033152/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827053 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1787827037.430480:0:2377:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff985607f08b40 x1874669228033152/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827053 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827037.430494:0:2377:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1787827037.430501:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1787827037.430504:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787827037.430530:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1787827037.430533:0:2384:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1787827037.430538:0:2384:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1787827037.430540:0:2384:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1787827037.430542:0:2384:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff985602ecf898. 00000400:00000200:1.0:1787827037.430545:0:2384:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985606618aa0 00000400:00000010:1.0:1787827037.430547:0:2384:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985606618aa0. 00000100:00000001:1.0:1787827037.430551:0:2384:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1787827037.430556:0:2384:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff985607f08b40 x1874669228033152/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827053 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1787827037.430577:0:2384:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff985607f08b40 x1874669228033152/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827053 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827037.430583:0:2384:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1787827037.430585:0:2384:0:(client.c:3001:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787827037.430588:0:2384:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:1.0:1787827037.430590:0:2384:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1787827037.430593:0:2384:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1787827037.430595:0:2384:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1787827037.430597:0:2384:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1787827037.430603:0:2384:0:(import.c:2149:obd_at_measure()) add 5 to ffff98561118e460 time=37 v=5 (5 5 5 5) 00000100:00001000:1.0:1787827037.430606:0:2384:0:(import.c:2149:obd_at_measure()) add 1 to ffff98561118e418 time=52 v=5 (1 0 0 0) 00000100:00000001:1.0:1787827037.430609:0:2384:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1787827037.430610:0:2384:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1787827037.430613:0:2384:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1787827037.430615:0:2384:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827037.430618:0:2384:0:(client.c:1797:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1787827037.430622:0:2384:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff985607f08b40 x1874669228033152/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827053 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827037.430629:0:2384:0:(client.c:2354:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1787827037.430631:0:2384:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1787827037.430633:0:2384:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1787827037.430637:0:2384:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff985607f08b40 x1874669228033152/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827053 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827037.430644:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607f08b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:3c946890-15ca-497a-b838-fe1cba61f4dd:2384:1874669228033152:192.168.203.114@tcp:400:kworker.0 00000100:00000001:1.0:1787827037.430649:0:2384:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827037.430651:0:2384:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1787827037.430654:0:2384:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607f08b40 x1874669228033152/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827053 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827037.430660:0:2384:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1787827037.430663:0:2384:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1787827037.430665:0:2384:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff985610db8e00. 02000000:00000001:1.0:1787827037.430668:0:2384:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1787827037.430671:0:2384:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:1.0:1787827037.430672:0:2384:0:(genops.c:1223:class_import_put()) import ffff98561118e000 refcount=3 obd=MGC192.168.203.114@tcp 00000020:00000001:1.0:1787827037.430674:0:2384:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:1.0:1787827037.430677:0:2384:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9856030d2300. 02000000:00000001:1.0:1787827037.430681:0:2384:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1787827037.430683:0:2384:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1787827037.430685:0:2384:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607f08b40. 00000100:00000001:1.0:1787827037.430709:0:2384:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1787827037.430712:0:2384:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787827037.430714:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827037.430717:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1787827037.430718:0:2384:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1787827037.430720:0:2384:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827037.430722:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1787827037.432958:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827037.432972:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1787827037.432977:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827037.432979:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827037.432987:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827037.432995:0:2378:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp) <- 192.168.203.114@tcp : PUT - for me 00000400:00000200:2.0:1787827037.433003:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827037.433007:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827037.433011:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827037.433015:0:2378:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.203.114@tcp of length 224 into portal 10 MB=0x6a90071688500 00000400:00000200:2.0:1787827037.433020:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827037.433025:0:2378:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.203.114@tcp of length 224/224 into md 0x32365 [1] + 224 00000800:00000001:2.0:1787827037.433032:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827037.433036:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827037.433039:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1787827037.433044:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000100:00000001:2.0:1787827037.433051:0:2378:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1787827037.433060:0:2378:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff985607f0a940 x1874669228033280/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827053 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1787827037.433071:0:2378:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff985607f0a940 x1874669228033280/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827053 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827037.433094:0:2378:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1787827037.433102:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827037.433106:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1787827037.433108:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827037.433110:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827037.433114:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827037.433117:0:2378:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp) <- 192.168.203.114@tcp : PUT - for me 00000400:00000200:2.0:1787827037.433121:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827037.433124:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827037.433127:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827037.433141:0:2378:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.203.114@tcp of length 224 into portal 10 MB=0x6a90071688580 00000400:00000200:2.0:1787827037.433144:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827037.433148:0:2378:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.203.114@tcp of length 224/224 into md 0x32375 [1] + 224 00000800:00000001:2.0:1787827037.433153:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827037.433156:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827037.433158:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1787827037.433160:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000100:00000001:2.0:1787827037.433163:0:2378:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1787827037.433167:0:2378:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff985607f0a1c0 x1874669228033408/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827053 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1787827037.433175:0:2378:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff985607f0a1c0 x1874669228033408/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827053 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827037.433195:0:2378:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1787827037.433199:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827037.433203:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1787827037.433205:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827037.433207:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827037.433210:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827037.433214:0:2378:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp) <- 192.168.203.114@tcp : PUT - for me 00000400:00000200:2.0:1787827037.433218:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827037.433225:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827037.433228:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827037.433232:0:2378:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.203.114@tcp of length 224 into portal 4 MB=0x6a90071688680 00000400:00000200:2.0:1787827037.433241:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827037.433247:0:2378:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 4 from 12345-192.168.203.114@tcp of length 224/224 into md 0x3235d [1] + 224 00000800:00000001:2.0:1787827037.433251:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827037.433255:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827037.433256:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1787827037.433260:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000100:00000001:2.0:1787827037.433262:0:2378:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1787827037.433266:0:2378:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff985607f08f00 x1874669228033664/t0(0) o9->lustre-OST0001-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827047 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1787827037.433274:0:2378:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff985607f08f00 x1874669228033664/t0(0) o9->lustre-OST0001-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827047 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827037.433334:0:2382:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1787827037.433339:0:2382:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1787827037.433349:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1787827037.433353:0:2384:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1787827037.433359:0:2384:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1787827037.433360:0:2384:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1787827037.433363:0:2384:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff985602ecf9f8. 00000400:00000200:1.0:1787827037.433367:0:2384:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985606618110 00000400:00000010:1.0:1787827037.433369:0:2384:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985606618110. 00000100:00000001:1.0:1787827037.433372:0:2384:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1787827037.433378:0:2384:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff985607f0a1c0 x1874669228033408/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827053 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1787827037.433388:0:2384:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff985607f0a1c0 x1874669228033408/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827053 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827037.433395:0:2384:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1787827037.433397:0:2384:0:(client.c:3001:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1787827037.433399:0:2383:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1787827037.433400:0:2384:0:(client.c:1593:after_reply()) Process entered 00000100:00000001:0.0:1787827037.433403:0:2383:0:(client.c:1990:ptlrpc_check_set()) Process entered 02000000:00000001:1.0:1787827037.433405:0:2384:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000400:00000001:0.0:1787827037.433408:0:2383:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000001:1.0:1787827037.433409:0:2384:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000400:00000001:0.0:1787827037.433413:0:2383:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000001:1.0:1787827037.433414:0:2384:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1787827037.433416:0:2383:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9856094ac370. 02000000:00000001:1.0:1787827037.433418:0:2384:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1787827037.433421:0:2383:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985610d30880 00000400:00000010:0.0:1787827037.433425:0:2383:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985610d30880. 00000100:00000001:0.0:1787827037.433428:0:2383:0:(events.c:78:reply_in_callback()) Process entered 00000100:00001000:1.0:1787827037.433430:0:2384:0:(import.c:2149:obd_at_measure()) add 5 to ffff985611185c60 time=37 v=5 (5 5 5 5) 00000100:00000200:0.0:1787827037.433434:0:2383:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff985607f0a940 x1874669228033280/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827053 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00001000:1.0:1787827037.433437:0:2384:0:(import.c:2149:obd_at_measure()) add 1 to ffff985611185c18 time=37 v=5 (1 1 1 1) 00000100:00000200:0.0:1787827037.433448:0:2383:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff985607f0a940 x1874669228033280/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827053 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827037.433449:0:2384:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1787827037.433452:0:2384:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1787827037.433458:0:2384:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:0.0:1787827037.433460:0:2383:0:(events.c:163:reply_in_callback()) Process leaving 00010000:00000001:1.0:1787827037.433464:0:2384:0:(ldlm_request.c:1723:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787827037.433465:0:2383:0:(client.c:3001:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1787827037.433470:0:2383:0:(client.c:1593:after_reply()) Process entered 00000100:00000001:1.0:1787827037.433471:0:2384:0:(client.c:3109:ptlrpc_free_committed()) Process entered 02000000:00000001:0.0:1787827037.433472:0:2383:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1787827037.433475:0:2383:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000040:1.0:1787827037.433475:0:2384:0:(client.c:3121:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff985602bc9800: skip recheck: last_committed 0 00000100:00000001:0.0:1787827037.433479:0:2383:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827037.433480:0:2384:0:(client.c:3122:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1787827037.433483:0:2384:0:(client.c:1797:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1787827037.433489:0:2384:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff985607f0a1c0 x1874669228033408/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827053 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827037.433503:0:2384:0:(client.c:2354:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1787827037.433506:0:2384:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1787827037.433508:0:2384:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1787827037.433512:0:2384:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff985607f0a1c0 x1874669228033408/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827053 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827037.433520:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607f0a1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669228033408:192.168.203.114@tcp:400:kworker.0 00000100:00000001:1.0:1787827037.433525:0:2384:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787827037.433527:0:2384:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1787827037.433530:0:2384:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607f0a1c0 x1874669228033408/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827053 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827037.433537:0:2384:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1787827037.433538:0:2383:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1787827037.433540:0:2384:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1787827037.433544:0:2384:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff985610db8c00. 02000000:00000001:1.0:1787827037.433547:0:2384:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1787827037.433550:0:2384:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:1.0:1787827037.433551:0:2384:0:(genops.c:1223:class_import_put()) import ffff985611185800 refcount=3 obd=lustre-MDT0001-mdc-ffff985602bc9800 00000020:00000001:1.0:1787827037.433554:0:2384:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:1.0:1787827037.433556:0:2384:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9856030d2e00. 02000000:00000001:1.0:1787827037.433565:0:2384:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1787827037.433567:0:2384:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1787827037.433569:0:2384:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607f0a1c0. 00000100:00001000:0.0:1787827037.433572:0:2383:0:(import.c:2149:obd_at_measure()) add 5 to ffff985611182460 time=115 v=5 (5 5 5 5) 00000100:00000001:1.0:1787827037.433575:0:2384:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00001000:0.0:1787827037.433578:0:2383:0:(import.c:2149:obd_at_measure()) add 1 to ffff985611182418 time=51 v=5 (1 0 0 0) 00000100:00000001:1.0:1787827037.433578:0:2384:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787827037.433583:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1787827037.433588:0:2383:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1787827037.433591:0:2383:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827037.433593:0:2384:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00010000:00000001:0.0:1787827037.433594:0:2383:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:1.0:1787827037.433597:0:2384:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827037.433598:0:2383:0:(ldlm_request.c:1723:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827037.433602:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1787827037.433603:0:2383:0:(client.c:3109:ptlrpc_free_committed()) Process entered 00000100:00000001:1.0:1787827037.433605:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787827037.433607:0:2383:0:(client.c:3121:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: skip recheck: last_committed 219043334508 00000100:00000001:0.0:1787827037.433611:0:2383:0:(client.c:3122:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1787827037.433612:0:2383:0:(client.c:1797:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787827037.433617:0:2383:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff985607f0a940 x1874669228033280/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827053 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787827037.433623:0:2383:0:(client.c:2354:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1787827037.433626:0:2383:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1787827037.433627:0:2383:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1787827037.433631:0:2383:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff985607f0a940 x1874669228033280/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827053 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827037.433638:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607f0a940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669228033280:192.168.203.114@tcp:400:kworker.0 00000100:00000001:0.0:1787827037.433642:0:2383:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787827037.433644:0:2383:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1787827037.433736:0:2383:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607f0a940 x1874669228033280/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827053 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787827037.433743:0:2383:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1787827037.433745:0:2383:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1787827037.433747:0:2383:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff98560540a400. 02000000:00000001:0.0:1787827037.433750:0:2383:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1787827037.433752:0:2383:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:0.0:1787827037.433753:0:2383:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=3 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:0.0:1787827037.433754:0:2383:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:0.0:1787827037.433756:0:2383:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9856030d2500. 02000000:00000001:0.0:1787827037.433759:0:2383:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1787827037.433760:0:2383:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1787827037.433761:0:2383:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607f0a940. 00000100:00000001:0.0:1787827037.433763:0:2383:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1787827037.433765:0:2383:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1787827037.433767:0:2383:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787827037.433769:0:2383:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1787827037.433770:0:2383:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1787827037.433772:0:2383:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787827037.433774:0:2383:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1787827037.433829:0:2378:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1787827037.433847:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000400:00000001:3.0:1787827037.433848:0:2382:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1787827037.433852:0:2382:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000800:00000001:2.0:1787827037.433854:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:3.0:1787827037.433855:0:2382:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff98561080e420. 00000400:00000200:3.0:1787827037.433860:0:2382:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985610afa660 00000400:00000010:3.0:1787827037.433862:0:2382:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985610afa660. 00000100:00000001:3.0:1787827037.433866:0:2382:0:(events.c:78:reply_in_callback()) Process entered 00000800:00000001:2.0:1787827037.433872:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000200:3.0:1787827037.433872:0:2382:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff985607f08f00 x1874669228033664/t0(0) o9->lustre-OST0001-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827047 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1787827037.433878:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:3.0:1787827037.433883:0:2382:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff985607f08f00 x1874669228033664/t0(0) o9->lustre-OST0001-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827047 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1787827037.433884:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827037.433895:0:2378:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp) <- 192.168.203.114@tcp : PUT - for me 00000100:00000001:3.0:1787827037.433902:0:2382:0:(events.c:163:reply_in_callback()) Process leaving 00000400:00000200:2.0:1787827037.433905:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000001:3.0:1787827037.433906:0:2382:0:(client.c:3001:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827037.433911:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000001:3.0:1787827037.433911:0:2382:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:3.0:1787827037.433914:0:2382:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000400:00000200:2.0:1787827037.433918:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000001:3.0:1787827037.433918:0:2382:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1787827037.433921:0:2382:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1787827037.433925:0:2378:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.203.114@tcp of length 224 into portal 4 MB=0x6a90071688600 02000000:00000001:3.0:1787827037.433926:0:2382:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1787827037.433932:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827037.433936:0:2378:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 4 from 12345-192.168.203.114@tcp of length 224/224 into md 0x3237d [1] + 224 00000800:00000001:2.0:1787827037.433941:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00001000:3.0:1787827037.433946:0:2382:0:(import.c:2149:obd_at_measure()) add 5 to ffff9856110b1460 time=115 v=5 (5 5 5 5) 00000800:00000001:2.0:1787827037.433948:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:3.0:1787827037.433952:0:2382:0:(import.c:2149:obd_at_measure()) add 1 to ffff9856110b1418 time=16 v=5 (1 1 1 1) 00000400:00000200:2.0:1787827037.433953:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:3.0:1787827037.433959:0:2382:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000400:00000200:2.0:1787827037.433960:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000100:00000001:3.0:1787827037.433961:0:2382:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1787827037.433965:0:2382:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:2.0:1787827037.433966:0:2378:0:(events.c:78:reply_in_callback()) Process entered 00010000:00000001:3.0:1787827037.433969:0:2382:0:(ldlm_request.c:1703:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:2.0:1787827037.433973:0:2378:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff985607f0a580 x1874669228033536/t0(0) o400->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827053 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827037.433975:0:2382:0:(client.c:3109:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1787827037.433978:0:2382:0:(client.c:3121:ptlrpc_free_committed()) lustre-OST0001-osc-ffff985602bc9800: skip recheck: last_committed 4294967636 00000100:00000001:3.0:1787827037.433982:0:2382:0:(client.c:3122:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1787827037.433985:0:2382:0:(client.c:1797:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1787827037.433990:0:2378:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff985607f0a580 x1874669228033536/t0(0) o400->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827053 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1787827037.433993:0:2382:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff985607f08f00 x1874669228033664/t0(0) o9->lustre-OST0001-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827047 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827037.434013:0:2382:0:(client.c:2354:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1787827037.434015:0:2382:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1787827037.434016:0:2382:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1787827037.434020:0:2378:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00080000:3.0:1787827037.434021:0:2382:0:(import.c:2011:ptlrpc_disconnect_idle_interpret()) @@@ inflight=1, refcount=3: rc = 0 req@ffff985607f08f00 x1874669228033664/t0(0) o9->lustre-OST0001-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827047 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1787827037.434026:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827037.434032:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00080000:3.0:1787827037.434034:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9856110b1000 lustre-OST0001_UUID: changing import state from CONNECTING to IDLE 00000100:00000040:3.0:1787827037.434041:0:2382:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff985607f08f00 x1874669228033664/t0(0) o9->lustre-OST0001-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827047 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827037.434049:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607f08f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228033664:192.168.203.114@tcp:9:kworker.0 00000100:00000001:3.0:1787827037.434053:0:2382:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1787827037.434055:0:2382:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1787827037.434058:0:2382:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607f08f00 x1874669228033664/t0(0) o9->lustre-OST0001-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827047 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827037.434065:0:2382:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1787827037.434067:0:2382:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1787827037.434069:0:2382:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff985607e32c00. 02000000:00000001:3.0:1787827037.434073:0:2382:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1787827037.434074:0:2382:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:3.0:1787827037.434076:0:2382:0:(genops.c:1223:class_import_put()) import ffff9856110b1000 refcount=2 obd=lustre-OST0001-osc-ffff985602bc9800 00000020:00000001:3.0:1787827037.434078:0:2382:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:3.0:1787827037.434079:0:2382:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff98561238ea00. 02000000:00000001:3.0:1787827037.434081:0:2382:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1787827037.434082:0:2382:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1787827037.434083:0:2382:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607f08f00. 00000100:00000001:3.0:1787827037.434086:0:2382:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1787827037.434086:0:2382:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1787827037.434088:0:2382:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1787827037.434091:0:2382:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1787827037.434092:0:2382:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827037.434093:0:2382:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1787827037.434094:0:2382:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787827037.434137:0:2383:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1787827037.434140:0:2383:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1787827037.434142:0:2383:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1787827037.434144:0:2383:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1787827037.434146:0:2383:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9856094acf20. 00000400:00000200:0.0:1787827037.434149:0:2383:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985610d30660 00000400:00000010:0.0:1787827037.434151:0:2383:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985610d30660. 00000100:00000001:0.0:1787827037.434154:0:2383:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1787827037.434158:0:2383:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff985607f0a580 x1874669228033536/t0(0) o400->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827053 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1787827037.434166:0:2383:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff985607f0a580 x1874669228033536/t0(0) o400->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827053 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787827037.434172:0:2383:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1787827037.434173:0:2383:0:(client.c:3001:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1787827037.434175:0:2383:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:0.0:1787827037.434177:0:2383:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1787827037.434178:0:2383:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1787827037.434180:0:2383:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1787827037.434181:0:2383:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1787827037.434186:0:2383:0:(import.c:2149:obd_at_measure()) add 5 to ffff9856110b0460 time=115 v=5 (5 5 5 5) 00000100:00001000:0.0:1787827037.434190:0:2383:0:(import.c:2149:obd_at_measure()) add 1 to ffff9856110b0418 time=1 v=5 (1 0 0 0) 00000100:00000001:0.0:1787827037.434192:0:2383:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1787827037.434193:0:2383:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827037.434195:0:2383:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1787827037.434197:0:2383:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787827037.434200:0:2383:0:(client.c:3109:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1787827037.434201:0:2383:0:(client.c:3121:ptlrpc_free_committed()) lustre-OST0000-osc-ffff985602bc9800: skip recheck: last_committed 8589935703 00000100:00000001:0.0:1787827037.434203:0:2383:0:(client.c:3122:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1787827037.434204:0:2383:0:(client.c:1797:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787827037.434208:0:2383:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff985607f0a580 x1874669228033536/t0(0) o400->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827053 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787827037.434214:0:2383:0:(client.c:2354:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1787827037.434216:0:2383:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1787827037.434217:0:2383:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1787827037.434221:0:2383:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff985607f0a580 x1874669228033536/t0(0) o400->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827053 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827037.434227:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607f0a580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669228033536:192.168.203.114@tcp:400:kworker.0 00000100:00000001:0.0:1787827037.434231:0:2383:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1787827037.434233:0:2383:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1787827037.434235:0:2383:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607f0a580 x1874669228033536/t0(0) o400->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827053 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787827037.434241:0:2383:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1787827037.434243:0:2383:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1787827037.434244:0:2383:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff98560540ac00. 02000000:00000001:0.0:1787827037.434247:0:2383:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1787827037.434248:0:2383:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:0.0:1787827037.434249:0:2383:0:(genops.c:1223:class_import_put()) import ffff9856110b0000 refcount=2 obd=lustre-OST0000-osc-ffff985602bc9800 00000020:00000001:0.0:1787827037.434251:0:2383:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:0.0:1787827037.434253:0:2383:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9856030d2900. 02000000:00000001:0.0:1787827037.434255:0:2383:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1787827037.434257:0:2383:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1787827037.434263:0:2383:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607f0a580. 00000100:00000001:0.0:1787827037.434266:0:2383:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1787827037.434268:0:2383:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1787827037.434270:0:2383:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1787827037.434273:0:2383:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1787827037.434274:0:2383:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787827037.434276:0:2383:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1787827037.434278:0:2383:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:2.0:1787827037.940820:0:37:0:(rw.c:1502:ll_writepages()) Process entered 00000080:00000001:2.0:1787827037.940824:0:37:0:(file.c:5312:cl_sync_file_range()) Process entered 00000020:00000001:2.0:1787827037.940826:0:37:0:(cl_object.c:754:cl_env_obtain()) Process entered 00000020:00000001:2.0:1787827037.940829:0:37:0:(cl_object.c:785:cl_env_obtain()) Process leaving (rc=18446630094152538088 : -113979557013528 : ffff985610fe17e8) 00000020:00001000:2.0:1787827037.940834:0:37:0:(cl_object.c:823:cl_env_get()) 1@ffff985610fe17e0 00000020:00000001:2.0:1787827037.940838:0:37:0:(cl_object.c:143:cl_object_top()) ffff985605435d00 -> ffff985605435d00 00000020:00000001:2.0:1787827037.940841:0:37:0:(cl_io.c:125:__cl_io_init()) Process entered 00000080:00000001:2.0:1787827037.940843:0:37:0:(vvp_io.c:1880:vvp_io_init()) Process entered 00000080:00200000:2.0:1787827037.940845:0:37:0:(vvp_io.c:1885:vvp_io_init()) [0x2000182d1:0x643:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000020:00000001:2.0:1787827037.940847:0:37:0:(cl_io.c:902:cl_io_slice_add()) Process entered 00000020:00000001:2.0:1787827037.940848:0:37:0:(cl_io.c:908:cl_io_slice_add()) Process leaving 00000080:00000001:2.0:1787827037.940849:0:37:0:(vvp_io.c:1957:vvp_io_init()) Process leaving (rc=0 : 0 : 0) 00020000:00000002:2.0:1787827037.940853:0:37:0:(lov_object.c:1600:lov_io_init()) [0x2000182d1:0x643:0x0]io ffff985605499698 type 6 ignore/verify layout 1/0, compressed 0 00020000:00000002:2.0:1787827037.940856:0:37:0:(lov_object.c:1264:lov_conf_freeze()) To take share lov(ffff985607d54aa8) owner 0000000000000000/ffff98560134cf80 00020000:00000001:2.0:1787827037.940860:0:37:0:(lov_io.c:2452:lov_io_init_composite()) Process entered 00020000:00000001:2.0:1787827037.940867:0:37:0:(lov_io.c:764:lov_io_slice_init()) Process entered 00020000:00000001:2.0:1787827037.940869:0:37:0:(lov_io.c:478:lov_io_mirror_init()) Process entered 00020000:00000001:2.0:1787827037.940871:0:37:0:(lov_io.c:489:lov_io_mirror_init()) Process leaving (rc=0 : 0 : 0) 00020000:00000001:2.0:1787827037.940872:0:37:0:(lov_io.c:916:lov_io_slice_init()) Process leaving via out (rc=0 : 0 : 0x0) 00020000:00000001:2.0:1787827037.940873:0:37:0:(lov_io.c:206:lov_io_subio_init()) Process entered 00020000:00000001:2.0:1787827037.940875:0:37:0:(lov_io.c:214:lov_io_subio_init()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1787827037.940875:0:37:0:(cl_io.c:902:cl_io_slice_add()) Process entered 00000020:00000001:2.0:1787827037.940877:0:37:0:(cl_io.c:908:cl_io_slice_add()) Process leaving 00020000:00000001:2.0:1787827037.940878:0:37:0:(lov_io.c:2465:lov_io_init_composite()) Process leaving 00020000:00000002:2.0:1787827037.940880:0:37:0:(lov_object.c:1272:lov_conf_thaw()) To release share lov(ffff985607d54aa8) owner 0000000000000000/ffff98560134cf80 00000020:00000001:2.0:1787827037.940883:0:37:0:(cl_io.c:146:__cl_io_init()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1787827037.940888:0:37:0:(cl_io.c:827:cl_io_loop()) Process entered 00000020:00000001:2.0:1787827037.940890:0:37:0:(cl_io.c:403:cl_io_iter_init()) Process entered 00020000:00000001:2.0:1787827037.940892:0:37:0:(lov_io.c:1218:lov_io_iter_init()) Process entered 00020000:00200000:2.0:1787827037.940896:0:37:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00020000:00000002:2.0:1787827037.940900:0:37:0:(lov_offset.c:253:lov_stripe_intersects()) [0->-1] -> [(0) 0->-1 (0)] 00020000:00000001:2.0:1787827037.940903:0:37:0:(lov_io.c:158:lov_sub_get()) Process entered 00020000:00000001:2.0:1787827037.940905:0:37:0:(lov_io.c:105:lov_io_sub_init()) Process entered 00000020:00000001:2.0:1787827037.940907:0:37:0:(cl_object.c:754:cl_env_obtain()) Process entered 00000020:00000001:2.0:1787827037.940909:0:37:0:(cl_object.c:785:cl_env_obtain()) Process leaving (rc=18446630094152536360 : -113979557015256 : ffff985610fe1128) 00000020:00001000:2.0:1787827037.940910:0:37:0:(cl_object.c:823:cl_env_get()) 1@ffff985610fe1120 00000020:00000001:2.0:1787827037.940913:0:37:0:(cl_object.c:143:cl_object_top()) ffff985605435850 -> ffff985605435d00 00000020:00000001:2.0:1787827037.940916:0:37:0:(cl_io.c:125:__cl_io_init()) Process entered 00000020:00000001:2.0:1787827037.940919:0:37:0:(cl_io.c:902:cl_io_slice_add()) Process entered 00000020:00000001:2.0:1787827037.940921:0:37:0:(cl_io.c:908:cl_io_slice_add()) Process leaving 00000020:00000001:2.0:1787827037.940922:0:37:0:(cl_io.c:146:__cl_io_init()) Process leaving (rc=0 : 0 : 0) 00020000:00000001:2.0:1787827037.940924:0:37:0:(lov_io.c:149:lov_io_sub_init()) Process leaving (rc=0 : 0 : 0) 00020000:00000001:2.0:1787827037.940926:0:37:0:(lov_io.c:186:lov_sub_get()) Process leaving (rc=18446630094151581016 : -113979557970600 : ffff985610ef7d58) 00000020:00000001:2.0:1787827037.940929:0:37:0:(cl_io.c:403:cl_io_iter_init()) Process entered 00000020:00000001:2.0:1787827037.940931:0:37:0:(cl_io.c:415:cl_io_iter_init()) Process leaving (rc=0 : 0 : 0) 00020000:00000001:2.0:1787827037.940933:0:37:0:(lov_io.c:1359:lov_io_iter_init()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1787827037.940935:0:37:0:(cl_io.c:415:cl_io_iter_init()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1787827037.940937:0:37:0:(cl_io.c:318:cl_io_lock()) Process entered 00020000:00000001:2.0:1787827037.940939:0:37:0:(lov_io.c:1502:lov_io_lock()) Process entered 00020000:00000001:2.0:1787827037.940942:0:37:0:(lov_io.c:1480:lov_io_call()) Process entered 00000020:00000001:2.0:1787827037.940943:0:37:0:(cl_io.c:318:cl_io_lock()) Process entered 00000020:00000001:2.0:1787827037.940945:0:37:0:(cl_io.c:290:cl_lockset_lock()) Process entered 00000020:00000001:2.0:1787827037.940947:0:37:0:(cl_io.c:299:cl_lockset_lock()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1787827037.940949:0:37:0:(cl_io.c:338:cl_io_lock()) Process leaving (rc=0 : 0 : 0) 00020000:00000001:2.0:1787827037.940951:0:37:0:(lov_io.c:1497:lov_io_call()) Process leaving (rc=0 : 0 : 0) 00020000:00000001:2.0:1787827037.940953:0:37:0:(lov_io.c:1503:lov_io_lock()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1787827037.940955:0:37:0:(cl_io.c:290:cl_lockset_lock()) Process entered 00000020:00000001:2.0:1787827037.940956:0:37:0:(cl_io.c:299:cl_lockset_lock()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1787827037.940958:0:37:0:(cl_io.c:338:cl_io_lock()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1787827037.940960:0:37:0:(cl_io.c:532:cl_io_start()) Process entered 00020000:00000001:2.0:1787827037.940962:0:37:0:(lov_io.c:1552:lov_io_start()) Process entered 00020000:00000001:2.0:1787827037.940964:0:37:0:(lov_io.c:1480:lov_io_call()) Process entered 00000020:00000001:2.0:1787827037.940965:0:37:0:(cl_io.c:532:cl_io_start()) Process entered 00000008:00000001:2.0:1787827037.940967:0:37:0:(osc_io.c:1125:osc_io_fsync_start()) Process entered 00000008:00000001:2.0:1787827037.940978:0:37:0:(osc_cache.c:3376:osc_cache_writeback_range()) Process entered 00000008:00000020:2.0:1787827037.940986:0:37:0:(osc_cache.c:3516:osc_cache_writeback_range()) obj ffff98560856afc0 ready 0|-|- wr 0|-|- rd 0|- pageout [0, 18446744073709551615] npages 0: rc=0. 00000008:00000001:2.0:1787827037.940991:0:37:0:(osc_cache.c:3531:osc_cache_writeback_range()) Process leaving (rc=0 : 0 : 0) 00000008:00000001:2.0:1787827037.940994:0:37:0:(osc_io.c:1194:osc_io_fsync_start()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1787827037.940995:0:37:0:(cl_io.c:544:cl_io_start()) Process leaving (rc=0 : 0 : 0) 00020000:00000001:2.0:1787827037.940996:0:37:0:(lov_io.c:1497:lov_io_call()) Process leaving (rc=0 : 0 : 0) 00020000:00000001:2.0:1787827037.940997:0:37:0:(lov_io.c:1553:lov_io_start()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1787827037.940998:0:37:0:(cl_io.c:544:cl_io_start()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1787827037.941000:0:37:0:(cl_io.c:559:cl_io_end()) Process entered 00020000:00000001:2.0:1787827037.941001:0:37:0:(lov_io.c:2153:lov_io_fsync_end()) Process entered 00020000:00000001:2.0:1787827037.941002:0:37:0:(lov_io.c:1558:lov_io_end_wrapper()) Process entered 00000020:00000001:2.0:1787827037.941003:0:37:0:(cl_io.c:559:cl_io_end()) Process entered 00000020:00000001:2.0:1787827037.941004:0:37:0:(cl_io.c:567:cl_io_end()) Process leaving 00020000:00000001:2.0:1787827037.941005:0:37:0:(lov_io.c:1568:lov_io_end_wrapper()) Process leaving (rc=0 : 0 : 0) 00020000:00000001:2.0:1787827037.941006:0:37:0:(lov_io.c:2164:lov_io_fsync_end()) Process leaving 00000020:00000001:2.0:1787827037.941008:0:37:0:(cl_io.c:567:cl_io_end()) Process leaving 00000020:00000001:2.0:1787827037.941009:0:37:0:(cl_io.c:356:cl_io_unlock()) Process entered 00020000:00000001:2.0:1787827037.941011:0:37:0:(lov_io.c:1646:lov_io_unlock()) Process entered 00020000:00000001:2.0:1787827037.941012:0:37:0:(lov_io.c:1480:lov_io_call()) Process entered 00000020:00000001:2.0:1787827037.941012:0:37:0:(cl_io.c:356:cl_io_unlock()) Process entered 00000020:00000001:2.0:1787827037.941013:0:37:0:(cl_io.c:377:cl_io_unlock()) Process leaving 00020000:00000001:2.0:1787827037.941014:0:37:0:(lov_io.c:1580:lov_io_unlock_wrapper()) Process leaving (rc=0 : 0 : 0) 00020000:00000001:2.0:1787827037.941015:0:37:0:(lov_io.c:1497:lov_io_call()) Process leaving (rc=0 : 0 : 0) 00020000:00000001:2.0:1787827037.941016:0:37:0:(lov_io.c:1649:lov_io_unlock()) Process leaving 00000020:00000001:2.0:1787827037.941017:0:37:0:(cl_io.c:377:cl_io_unlock()) Process leaving 00000020:00000001:2.0:1787827037.941018:0:37:0:(cl_io.c:453:cl_io_rw_advance()) Process entered 00000020:00000001:2.0:1787827037.941019:0:37:0:(cl_io.c:469:cl_io_rw_advance()) Process leaving 00000020:00000001:2.0:1787827037.941020:0:37:0:(cl_io.c:433:cl_io_iter_fini()) Process entered 00020000:00000001:2.0:1787827037.941021:0:37:0:(lov_io.c:1627:lov_io_iter_fini()) Process entered 00020000:00000001:2.0:1787827037.941022:0:37:0:(lov_io.c:1480:lov_io_call()) Process entered 00000020:00000001:2.0:1787827037.941023:0:37:0:(cl_io.c:433:cl_io_iter_fini()) Process entered 00000020:00000001:2.0:1787827037.941024:0:37:0:(cl_io.c:439:cl_io_iter_fini()) Process leaving 00020000:00000001:2.0:1787827037.941024:0:37:0:(lov_io.c:1574:lov_io_iter_fini_wrapper()) Process leaving (rc=0 : 0 : 0) 00020000:00000001:2.0:1787827037.941025:0:37:0:(lov_io.c:1497:lov_io_call()) Process leaving (rc=0 : 0 : 0) 00020000:00000001:2.0:1787827037.941026:0:37:0:(lov_io.c:1638:lov_io_iter_fini()) Process leaving 00000020:00000001:2.0:1787827037.941027:0:37:0:(cl_io.c:439:cl_io_iter_fini()) Process leaving 00000020:00000001:2.0:1787827037.941028:0:37:0:(cl_io.c:877:cl_io_loop()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1787827037.941029:0:37:0:(cl_io.c:71:cl_io_fini()) Process entered 00020000:00000001:2.0:1787827037.941031:0:37:0:(lov_io.c:1054:lov_io_fini()) Process entered 00020000:00000001:2.0:1787827037.941032:0:37:0:(lov_io.c:74:lov_io_sub_fini()) Process entered 00000020:00000001:2.0:1787827037.941033:0:37:0:(cl_io.c:71:cl_io_fini()) Process entered 00000020:00000001:2.0:1787827037.941034:0:37:0:(cl_io.c:112:cl_io_fini()) Process leaving 00000020:00001000:2.0:1787827037.941036:0:37:0:(cl_object.c:917:cl_env_put()) 1@ffff985610fe1120 00020000:00000001:2.0:1787827037.941039:0:37:0:(lov_io.c:82:lov_io_sub_fini()) Process leaving 00020000:00000001:2.0:1787827037.941041:0:37:0:(lov_io.c:1073:lov_io_fini()) Process leaving 00000080:00000001:2.0:1787827037.941043:0:37:0:(vvp_io.c:296:vvp_io_fini()) Process entered 00000080:00200000:2.0:1787827037.941044:0:37:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x643:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00000001:2.0:1787827037.941048:0:37:0:(vvp_io.c:402:vvp_io_fini()) Process leaving 00000020:00000001:2.0:1787827037.941048:0:37:0:(cl_io.c:112:cl_io_fini()) Process leaving 00000020:00001000:2.0:1787827037.941051:0:37:0:(cl_object.c:917:cl_env_put()) 1@ffff985610fe17e0 00000080:00000001:2.0:1787827037.941053:0:37:0:(file.c:5349:cl_sync_file_range()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:2.0:1787827037.941056:0:37:0:(rw.c:1576:ll_writepages()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827038.004803:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827038.004808:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827038.004811:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827038.004812:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827038.004814:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827038.004815:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1787827038.004867:0:15163:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1787827038.004870:0:15163:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1787827038.004872:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:0.0:1787827039.028791:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827039.028798:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827039.028803:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827039.028804:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827039.028807:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827039.028810:0:49842:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:0.0:1787827039.028812:0:49842:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:0.0:1787827039.028817:0:49842:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:0.0:1787827039.028819:0:49842:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:0.0:1787827039.028821:0:49842:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827039.028823:0:49842:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:0.0:1787827039.028826:0:49842:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827039.028828:0:49842:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827039.028830:0:49842:0:(ldlm_pool.c:496:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1787827039.028885:0:33341:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1787827039.028888:0:33341:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1787827039.028892:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:0.0:1787827040.052808:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827040.052815:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827040.052820:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827040.052822:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827040.052825:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827040.052826:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1787827040.052848:0:15164:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1787827040.052850:0:15164:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1787827040.052853:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000001:3.0:1787827041.012858:0:15156:0:(service.c:1710:ptlrpc_at_check_timed()) Process entered 00000100:00000001:3.0:1787827041.012866:0:15156:0:(service.c:1721:ptlrpc_at_check_timed()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827041.076799:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827041.076806:0:49842:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:0.0:1787827041.076808:0:49842:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:0.0:1787827041.076812:0:49842:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827041.076816:0:49842:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:0.0:1787827041.076818:0:49842:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827041.076820:0:49842:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827041.076822:0:49842:0:(ldlm_pool.c:496:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827041.076826:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827041.076827:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827041.076830:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827041.076832:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1787827041.076851:0:33342:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1787827041.076853:0:33342:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1787827041.076856:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:0.0:1787827042.100827:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827042.100834:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827042.100840:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827042.100840:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827042.100844:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827042.100845:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1787827042.100908:0:79887:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1787827042.100912:0:79887:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1787827042.100917:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1787827042.548789:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1787827042.548803:0:38:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1787827042.548816:0:38:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 904 at ffff985607f0a580. 00000020:00000040:0.0:1787827042.548823:0:38:0:(genops.c:1210:class_import_get()) import ffff98561118e000 refcount=4 obd=MGC192.168.203.114@tcp 00000100:00000001:0.0:1787827042.548828:0:38:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1787827042.548832:0:38:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1787827042.548839:0:38:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1787827042.548850:0:38:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff98561238ea00. 00000100:00000001:0.0:1787827042.548857:0:38:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787827042.548870:0:38:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS req@ffff985607f0a580 x1874669228033792/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1787827042.548879:0:38:0:(jobid.c:939:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1787827042.548883:0:38:0:(jobid.c:979:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787827042.548888:0:38:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000047c110d7] to pc [ptlrpcd_00_01+1] req@ffff985607f0a580 x1874669228033792/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787827042.548918:0:38:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787827042.548921:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1787827042.548923:0:38:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1787827042.548926:0:38:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 904 at ffff985607f0a940. 00000020:00000040:0.0:1787827042.548928:0:38:0:(genops.c:1210:class_import_get()) import ffff985611182000 refcount=4 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000100:00000001:0.0:1787827042.548930:0:38:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1787827042.548931:0:38:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1787827042.548932:0:38:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1787827042.548935:0:38:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff98561238e000. 00000100:00000001:0.0:1787827042.548937:0:38:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787827042.548939:0:38:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID req@ffff985607f0a940 x1874669228033920/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1787827042.548947:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1787827042.548950:0:38:0:(jobid.c:939:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1787827042.548953:0:38:0:(jobid.c:979:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827042.548954:0:2384:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000040:0.0:1787827042.548956:0:38:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000083e565bf] to pc [ptlrpcd_00_00+0] req@ffff985607f0a940 x1874669228033920/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827042.548958:0:2384:0:(client.c:1813:ptlrpc_send_new_req()) Process entered 00000100:00000001:0.0:1787827042.548965:0:38:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787827042.548968:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:1.0:1787827042.548970:0:2384:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff985607f0a580 x1874669228033792/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787827042.548971:0:38:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1787827042.548975:0:38:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 904 at ffff985607f09e00. 00000020:00000040:0.0:1787827042.548978:0:38:0:(genops.c:1210:class_import_get()) import ffff985611185800 refcount=4 obd=lustre-MDT0001-mdc-ffff985602bc9800 00000100:00000001:0.0:1787827042.548981:0:38:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1787827042.548983:0:38:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1787827042.548985:0:38:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1787827042.548989:0:38:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff98561238e200. 00000100:00000001:1.0:1787827042.548991:0:2384:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1787827042.548992:0:38:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827042.548995:0:2384:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787827042.548997:0:38:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID req@ffff985607f09e00 x1874669228034048/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 02000000:00000001:1.0:1787827042.549001:0:2384:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:0.0:1787827042.549002:0:38:0:(jobid.c:939:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1787827042.549005:0:38:0:(jobid.c:979:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1787827042.549007:0:2384:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000040:0.0:1787827042.549008:0:38:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000090f91eaf] to pc [ptlrpcd_00_01+1] req@ffff985607f09e00 x1874669228034048/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1787827042.549013:0:2384:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787827042.549019:0:38:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787827042.549022:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-OST0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:1.0:1787827042.549022:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607f0a580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:3c946890-15ca-497a-b838-fe1cba61f4dd:2384:1874669228033792:192.168.203.114@tcp:400:kworker.0 00000100:00000001:0.0:1787827042.549026:0:38:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1787827042.549030:0:38:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 904 at ffff985607f08f00. 00000100:00000001:1.0:1787827042.549031:0:2384:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000020:00000040:0.0:1787827042.549033:0:38:0:(genops.c:1210:class_import_get()) import ffff9856110b0000 refcount=3 obd=lustre-OST0000-osc-ffff985602bc9800 00000100:00000001:0.0:1787827042.549035:0:38:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1787827042.549036:0:38:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1787827042.549037:0:38:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1787827042.549039:0:38:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff98561324cf00. 02000000:00000001:1.0:1787827042.549040:0:2384:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:0.0:1787827042.549042:0:38:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1787827042.549044:0:2384:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787827042.549047:0:38:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-OST0000_UUID req@ffff985607f08f00 x1874669228034176/t0(0) o400->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 02000000:00000001:1.0:1787827042.549050:0:2384:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:0.0:1787827042.549055:0:38:0:(jobid.c:939:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1787827042.549057:0:38:0:(jobid.c:979:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1787827042.549059:0:2384:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff985610db8c00. 00000100:00000040:0.0:1787827042.549061:0:38:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000002a2b9b7a] to pc [ptlrpcd_00_00+0] req@ffff985607f08f00 x1874669228034176/t0(0) o400->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1787827042.549063:0:2384:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787827042.549073:0:38:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1787827042.549077:0:2384:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000010:1.0:1787827042.549084:0:2384:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff985602ecf9f8. 00000100:00000040:0.0:1787827042.549086:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000400:00000010:1.0:1787827042.549093:0:2384:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985606618110. 00000100:00000040:0.0:1787827042.549094:0:38:0:(pinger.c:247:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000040:0.0:1787827042.549102:0:38:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (3198) 00000100:00000200:1.0:1787827042.549103:0:2384:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1874669228033792, portal 25 00000100:00000001:1.0:1787827042.549107:0:2384:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1787827042.549109:0:2384:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446630094000661888 : -113979708889728 : ffff985607f0a580) 00000100:00000040:1.0:1787827042.549117:0:2384:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff985607f0a580 x1874669228033792/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827058 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787827042.549119:0:2383:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1787827042.549123:0:2383:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1787827042.549126:0:2383:0:(client.c:1813:ptlrpc_send_new_req()) Process entered 00000100:00000001:1.0:1787827042.549130:0:2384:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1787827042.549134:0:2383:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff985607f0a940 x1874669228033920/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1787827042.549138:0:2384:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.203.114@tcp 00000400:00000010:1.0:1787827042.549142:0:2384:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985606618aa0. 00000100:00000200:1.0:1787827042.549148:0:2384:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1874669228033792, offset 0 00000100:00000001:0.0:1787827042.549155:0:2383:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1787827042.549157:0:2383:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1787827042.549160:0:2383:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 00000400:00000200:1.0:1787827042.549162:0:2384:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.203.114@tcp 02000000:00000001:0.0:1787827042.549163:0:2383:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1787827042.549168:0:2383:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1787827042.549172:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607f0a940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669228033920:192.168.203.114@tcp:400:kworker.0 00000100:00000001:0.0:1787827042.549175:0:2383:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000200:1.0:1787827042.549178:0:2384:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 02000000:00000001:0.0:1787827042.549179:0:2383:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1787827042.549182:0:2383:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1787827042.549184:0:2383:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1787827042.549188:0:2383:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff98560540ac00. 00000400:00000200:1.0:1787827042.549190:0:2384:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.203.114@tcp local destination 02000000:00000001:0.0:1787827042.549192:0:2383:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1787827042.549198:0:2383:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:1.0:1787827042.549201:0:2384:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000010:0.0:1787827042.549203:0:2383:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9856094acf20. 00000400:00000200:1.0:1787827042.549209:0:2384:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.203.14@tcp 00000400:00000010:0.0:1787827042.549210:0:2383:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985610d30660. 00000400:00000200:1.0:1787827042.549215:0:2384:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.203.14@tcp lpni_is_preferred = 0 00000100:00000200:0.0:1787827042.549216:0:2383:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1874669228033920, portal 10 00000400:00000200:1.0:1787827042.549220:0:2384:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.203.114@tcp 00000400:00000200:1.0:1787827042.549225:0:2384:0:(lib-move.c:1745:lnet_handle_send()) 192.168.203.14@tcp NI seq info: [21902:21902:256:4294967295] 192.168.203.114@tcp LPNI seq info [21902:21902:8:4294967295] 00000100:00000001:0.0:1787827042.549229:0:2383:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1787827042.549232:0:2383:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446630094000662848 : -113979708888768 : ffff985607f0a940) 00000400:00000200:1.0:1787827042.549233:0:2384:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000040:0.0:1787827042.549244:0:2383:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff985607f0a940 x1874669228033920/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827058 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1787827042.549253:0:2384:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp:) -> 192.168.203.114@tcp(192.168.203.114@tcp:192.168.203.114@tcp) : PUT try# 0 00000100:00000001:0.0:1787827042.549258:0:2383:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1787827042.549264:0:2383:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.203.114@tcp 00000400:00000010:0.0:1787827042.549269:0:2383:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985610d30880. 00000800:00000200:1.0:1787827042.549269:0:2384:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.203.114@tcp 00000100:00000200:0.0:1787827042.549275:0:2383:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1874669228033920, offset 0 00000800:00000010:1.0:1787827042.549278:0:2384:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff985607d95d00. 00000400:00000200:0.0:1787827042.549283:0:2383:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.203.114@tcp 00000400:00000200:0.0:1787827042.549290:0:2383:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000800:00000200:1.0:1787827042.549290:0:2384:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff985606751480] -> 12345-192.168.203.114@tcp (5) 00000400:00000200:0.0:1787827042.549298:0:2383:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.203.114@tcp local destination 00000400:00000200:0.0:1787827042.549303:0:2383:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000800:00000200:1.0:1787827042.549305:0:2384:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.203.114@tcp ip 192.168.203.114:988 00000400:00000200:0.0:1787827042.549308:0:2383:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.203.14@tcp 00000800:00000200:1.0:1787827042.549311:0:2384:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff985607d95d00 type 1, nob 320 niov 1 nkiov 1 00000400:00000200:0.0:1787827042.549313:0:2383:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.203.14@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1787827042.549318:0:2383:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.203.114@tcp 00000400:00000200:0.0:1787827042.549322:0:2383:0:(lib-move.c:1745:lnet_handle_send()) 192.168.203.14@tcp NI seq info: [21903:21903:255:4294967295] 192.168.203.114@tcp LPNI seq info [21903:21903:7:4294967295] 00000400:00000200:0.0:1787827042.549326:0:2383:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:0.0:1787827042.549333:0:2383:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp:) -> 192.168.203.114@tcp(192.168.203.114@tcp:192.168.203.114@tcp) : PUT try# 0 00000100:00000001:1.0:1787827042.549335:0:2384:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1787827042.549340:0:2383:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.203.114@tcp 00000400:00000200:1.0:1787827042.549344:0:2384:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000800:00000010:0.0:1787827042.549346:0:2383:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff98561324cb00. 00000100:00000001:1.0:1787827042.549348:0:2384:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.203.114@tcp: PM QoS new deadline estimation for cpu 2 is 4297870588 00000800:00000200:0.0:1787827042.549354:0:2383:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff985606751480] -> 12345-192.168.203.114@tcp (5) 00000800:00000200:0.0:1787827042.549361:0:2383:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.203.114@tcp ip 192.168.203.114:988 00000100:00000001:1.0:1787827042.549361:0:2384:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.203.114@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4297865468 00000800:00000200:0.0:1787827042.549367:0:2383:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff98561324cb00 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1787827042.549369:0:2384:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.203.114@tcp: PM QoS new deadline estimation for cpu 3 is 4297870588 00000100:00000001:0.0:1787827042.549371:0:2383:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827042.549376:0:2384:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.203.114@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4297865468 00000100:00000040:0.0:1787827042.549378:0:2383:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff985602bc9800: using a timeout of 38047 usecs (39 jiffies) 00000100:00000001:1.0:1787827042.549382:0:2384:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000200:0.0:1787827042.549383:0:2383:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000001:0.0:1787827042.549387:0:2383:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff985602bc9800: PM QoS new deadline estimation for cpu 2 is 4297860647 00000100:00000001:1.0:1787827042.549388:0:2384:0:(client.c:1941:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787827042.549390:0:2383:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff985602bc9800: PM QoS request active for cpu 2, keep current deadline 4297870588 00000100:00000001:0.0:1787827042.549394:0:2383:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff985602bc9800: PM QoS new deadline estimation for cpu 3 is 4297860647 00000100:00000001:1.0:1787827042.549394:0:2384:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827042.549399:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1787827042.549406:0:2383:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff985602bc9800: PM QoS request active for cpu 3, keep current deadline 4297870588 00000100:00000001:1.0:1787827042.549409:0:2384:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1787827042.549412:0:2384:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1787827042.549414:0:2383:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1787827042.549417:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1787827042.549419:0:2383:0:(client.c:1941:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827042.549421:0:2384:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1787827042.549423:0:2383:0:(client.c:1813:ptlrpc_send_new_req()) Process entered 00000100:00000001:1.0:1787827042.549424:0:2384:0:(client.c:1813:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1787827042.549430:0:2383:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff985607f08f00 x1874669228034176/t0(0) o400->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1787827042.549432:0:2384:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff985607f09e00 x1874669228034048/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000800:00000200:3.0:1787827042.549438:0:2377:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1787827042.549445:0:2377:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff985607d95d00. 00000100:00000001:0.0:1787827042.549454:0:2383:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000400:00000200:3.0:1787827042.549454:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:1.0:1787827042.549456:0:2384:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1787827042.549458:0:2383:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827042.549460:0:2384:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1787827042.549464:0:2383:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 00000400:00000200:3.0:1787827042.549464:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 02000000:00000001:1.0:1787827042.549465:0:2384:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1787827042.549469:0:2383:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1787827042.549470:0:2384:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000400:00000200:3.0:1787827042.549472:0:2377:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985606618aa0 00000400:00000010:3.0:1787827042.549476:0:2377:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985606618aa0. 02000000:00000001:0.0:1787827042.549477:0:2383:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1787827042.549477:0:2384:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827042.549482:0:2377:0:(events.c:34:request_out_callback()) Process entered 00000100:00100000:0.0:1787827042.549486:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607f08f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669228034176:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827042.549486:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607f09e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669228034048:192.168.203.114@tcp:400:kworker.0 00000100:00000200:3.0:1787827042.549490:0:2377:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff985607f0a580 x1874669228033792/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827058 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827042.549495:0:2384:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:0.0:1787827042.549496:0:2383:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1787827042.549500:0:2384:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1787827042.549502:0:2383:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1787827042.549504:0:2384:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1787827042.549505:0:2383:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1787827042.549508:0:2384:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000001:0.0:1787827042.549509:0:2383:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:3.0:1787827042.549510:0:2377:0:(client.c:2867:__ptlrpc_req_put()) Process entered 02000000:00000010:0.0:1787827042.549513:0:2383:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff98560540a400. 02000000:00000001:0.0:1787827042.549517:0:2383:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1787827042.549517:0:2377:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff985607f0a580 x1874669228033792/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827058 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000400:00000200:0.0:1787827042.549521:0:2383:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 02000000:00000010:1.0:1787827042.549523:0:2384:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff985610db8e00. 00000400:00000010:0.0:1787827042.549526:0:2383:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9856094ac370. 02000000:00000001:1.0:1787827042.549530:0:2384:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1787827042.549532:0:2383:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985610d30b28. 00000100:00000200:0.0:1787827042.549535:0:2383:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1874669228034176, portal 4 00000400:00000200:1.0:1787827042.549536:0:2384:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000001:0.0:1787827042.549537:0:2383:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1787827042.549541:0:2383:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446630094000656128 : -113979708895488 : ffff985607f08f00) 00000400:00000010:1.0:1787827042.549543:0:2384:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff985602ecf898. 00000400:00000010:1.0:1787827042.549551:0:2384:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985606618cc0. 00000100:00000040:0.0:1787827042.549552:0:2383:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff985607f08f00 x1874669228034176/t0(0) o400->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827058 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827042.549553:0:2377:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:1.0:1787827042.549556:0:2384:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1874669228034048, portal 10 00000100:00000001:3.0:1787827042.549558:0:2377:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:1.0:1787827042.549563:0:2384:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1787827042.549564:0:2383:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:1.0:1787827042.549566:0:2384:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446630094000659968 : -113979708891648 : ffff985607f09e00) 00000100:00000040:0.0:1787827042.549569:0:2383:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.203.114@tcp 00000400:00000010:0.0:1787827042.549574:0:2383:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985610d304c8. 00000100:00000040:1.0:1787827042.549577:0:2384:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff985607f09e00 x1874669228034048/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827058 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1787827042.549578:0:2383:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 28, xid 1874669228034176, offset 0 00000400:00000200:0.0:1787827042.549585:0:2383:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.203.114@tcp 00000400:00000200:0.0:1787827042.549590:0:2383:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:0.0:1787827042.549596:0:2383:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.203.114@tcp local destination 00000100:00000001:1.0:1787827042.549601:0:2384:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000400:00000200:0.0:1787827042.549602:0:2383:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1787827042.549607:0:2383:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.203.14@tcp 00000100:00000040:1.0:1787827042.549609:0:2384:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.203.114@tcp 00000400:00000200:0.0:1787827042.549610:0:2383:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.203.14@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1787827042.549614:0:2383:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.203.114@tcp 00000400:00000200:0.0:1787827042.549616:0:2383:0:(lib-move.c:1745:lnet_handle_send()) 192.168.203.14@tcp NI seq info: [21904:21904:255:4294967295] 192.168.203.114@tcp LPNI seq info [21904:21904:7:4294967295] 00000400:00000010:1.0:1787827042.549616:0:2384:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985606618a18. 00000400:00000200:0.0:1787827042.549622:0:2383:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000200:1.0:1787827042.549623:0:2384:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1874669228034048, offset 0 00000400:00000200:0.0:1787827042.549631:0:2383:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp:) -> 192.168.203.114@tcp(192.168.203.114@tcp:192.168.203.114@tcp) : PUT try# 0 00000800:00000200:3.0:1787827042.549632:0:2377:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000400:00000200:1.0:1787827042.549635:0:2384:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.203.114@tcp 00000800:00000010:3.0:1787827042.549637:0:2377:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff98561324cb00. 00000400:00000200:3.0:1787827042.549643:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1787827042.549647:0:2384:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000800:00000200:0.0:1787827042.549648:0:2383:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.203.114@tcp 00000400:00000200:3.0:1787827042.549651:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000400:00000200:1.0:1787827042.549655:0:2384:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.203.114@tcp local destination 00000800:00000010:0.0:1787827042.549656:0:2383:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff98561324c000. 00000400:00000200:3.0:1787827042.549656:0:2377:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985610d30880 00000400:00000200:1.0:1787827042.549660:0:2384:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000010:3.0:1787827042.549661:0:2377:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985610d30880. 00000100:00000001:3.0:1787827042.549665:0:2377:0:(events.c:34:request_out_callback()) Process entered 00000800:00000200:0.0:1787827042.549667:0:2383:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff985606751480] -> 12345-192.168.203.114@tcp (5) 00000400:00000200:1.0:1787827042.549667:0:2384:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.203.14@tcp 00000400:00000200:1.0:1787827042.549672:0:2384:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.203.14@tcp lpni_is_preferred = 0 00000100:00000200:3.0:1787827042.549672:0:2377:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff985607f0a940 x1874669228033920/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827058 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1787827042.549679:0:2384:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.203.114@tcp 00000800:00000200:0.0:1787827042.549680:0:2383:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.203.114@tcp ip 192.168.203.114:988 00000400:00000200:1.0:1787827042.549686:0:2384:0:(lib-move.c:1745:lnet_handle_send()) 192.168.203.14@tcp NI seq info: [21905:21905:254:4294967295] 192.168.203.114@tcp LPNI seq info [21905:21905:6:4294967295] 00000800:00000200:0.0:1787827042.549688:0:2383:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff98561324c000 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1787827042.549727:0:2383:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787827042.549730:0:2383:0:(niobuf.c:635:kick_cpu_latency()) lustre-OST0000-osc-ffff985602bc9800: using a timeout of 14643 usecs (15 jiffies) 00000400:00000200:0.0:1787827042.549737:0:2383:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:1.0:1787827042.549738:0:2384:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000001:0.0:1787827042.549742:0:2383:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff985602bc9800: PM QoS new deadline estimation for cpu 2 is 4297860624 00000100:00000001:0.0:1787827042.549748:0:2383:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff985602bc9800: PM QoS request active for cpu 2, keep current deadline 4297870588 00000400:00000200:1.0:1787827042.549749:0:2384:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp:) -> 192.168.203.114@tcp(192.168.203.114@tcp:192.168.203.114@tcp) : PUT try# 0 00000100:00000001:0.0:1787827042.549754:0:2383:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff985602bc9800: PM QoS new deadline estimation for cpu 3 is 4297860624 00000800:00000200:1.0:1787827042.549757:0:2384:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.203.114@tcp 00000100:00000001:0.0:1787827042.549759:0:2383:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff985602bc9800: PM QoS request active for cpu 3, keep current deadline 4297870588 00000800:00000010:1.0:1787827042.549762:0:2384:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff985607d95600. 00000100:00000001:0.0:1787827042.549763:0:2383:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1787827042.549767:0:2383:0:(client.c:1941:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787827042.549771:0:2383:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1787827042.549772:0:2384:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff985606751480] -> 12345-192.168.203.114@tcp (5) 00000100:00000001:0.0:1787827042.549776:0:2383:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000800:00000200:1.0:1787827042.549779:0:2384:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.203.114@tcp ip 192.168.203.114:988 00000800:00000200:1.0:1787827042.549783:0:2384:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff985607d95600 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1787827042.549784:0:2383:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1787827042.549786:0:2384:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787827042.549787:0:2383:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000040:1.0:1787827042.549790:0:2384:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff985602bc9800: using a timeout of 11942 usecs (12 jiffies) 00000100:00000001:0.0:1787827042.549792:0:2383:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:1.0:1787827042.549795:0:2384:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000001:0.0:1787827042.549796:0:2383:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1787827042.549800:0:2384:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff985602bc9800: PM QoS new deadline estimation for cpu 2 is 4297860621 00000100:00000001:0.0:1787827042.549801:0:2383:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827042.549804:0:2384:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff985602bc9800: PM QoS request active for cpu 2, keep current deadline 4297870588 00000100:00000001:0.0:1787827042.549806:0:2383:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827042.549807:0:2384:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff985602bc9800: PM QoS new deadline estimation for cpu 3 is 4297860621 00000100:00000001:1.0:1787827042.549812:0:2384:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff985602bc9800: PM QoS request active for cpu 3, keep current deadline 4297870588 00000100:00000001:1.0:1787827042.549814:0:2384:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1787827042.549816:0:2384:0:(client.c:1941:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827042.549819:0:2384:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827042.549819:0:2377:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000001:1.0:1787827042.549820:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787827042.549825:0:2384:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1787827042.549827:0:2384:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000040:3.0:1787827042.549828:0:2377:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff985607f0a940 x1874669228033920/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827058 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827042.549830:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1787827042.549832:0:2384:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1787827042.549834:0:2384:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827042.549838:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827042.549844:0:2377:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827042.549846:0:2377:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1787827042.549862:0:2377:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1787827042.549864:0:2377:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff98561324c000. 00000400:00000200:3.0:1787827042.549867:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1787827042.549870:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000400:00000200:3.0:1787827042.549873:0:2377:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985610d304c8 00000400:00000010:3.0:1787827042.549875:0:2377:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985610d304c8. 00000100:00000001:3.0:1787827042.549877:0:2377:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1787827042.549881:0:2377:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff985607f08f00 x1874669228034176/t0(0) o400->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827058 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827042.549887:0:2377:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1787827042.549889:0:2377:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff985607f08f00 x1874669228034176/t0(0) o400->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827058 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827042.549896:0:2377:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827042.549897:0:2377:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1787827042.549931:0:2377:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1787827042.549935:0:2377:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff985607d95600. 00000400:00000200:3.0:1787827042.549938:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1787827042.549941:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000400:00000200:3.0:1787827042.549944:0:2377:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985606618a18 00000400:00000010:3.0:1787827042.549945:0:2377:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985606618a18. 00000100:00000001:3.0:1787827042.549947:0:2377:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1787827042.549950:0:2377:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff985607f09e00 x1874669228034048/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827058 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827042.549957:0:2377:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1787827042.549960:0:2377:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff985607f09e00 x1874669228034048/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827058 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827042.549966:0:2377:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827042.549967:0:2377:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1787827042.550566:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827042.550579:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1787827042.550582:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827042.550584:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827042.550589:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827042.550596:0:2378:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp) <- 192.168.203.114@tcp : PUT - for me 00000400:00000200:2.0:1787827042.550603:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827042.550607:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827042.550611:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827042.550615:0:2378:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.203.114@tcp of length 224 into portal 25 MB=0x6a90071688700 00000400:00000200:2.0:1787827042.550622:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827042.550628:0:2378:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.203.114@tcp of length 224/224 into md 0x32385 [1] + 224 00000800:00000001:2.0:1787827042.550636:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827042.550640:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827042.550642:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1787827042.550646:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000100:00000001:2.0:1787827042.550650:0:2378:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1787827042.550656:0:2378:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff985607f0a580 x1874669228033792/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827058 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1787827042.550666:0:2378:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff985607f0a580 x1874669228033792/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827058 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827042.550681:0:2378:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1787827042.550719:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1787827042.550724:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000001:2.0:1787827042.550724:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787827042.550728:0:2384:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000800:00000001:2.0:1787827042.550729:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827042.550735:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:1.0:1787827042.550736:0:2384:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1787827042.550740:0:2384:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000200:2.0:1787827042.550741:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000010:1.0:1787827042.550745:0:2384:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff985602ecf9f8. 00000400:00000200:2.0:1787827042.550749:0:2378:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp) <- 192.168.203.114@tcp : PUT - for me 00000400:00000200:1.0:1787827042.550752:0:2384:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985606618110 00000400:00000010:1.0:1787827042.550756:0:2384:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985606618110. 00000400:00000200:2.0:1787827042.550756:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000001:1.0:1787827042.550762:0:2384:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000200:2.0:1787827042.550763:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000200:1.0:1787827042.550770:0:2384:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff985607f0a580 x1874669228033792/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827058 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1787827042.550770:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827042.550778:0:2378:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.203.114@tcp of length 224 into portal 10 MB=0x6a90071688780 00000400:00000200:2.0:1787827042.550787:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000200:1.0:1787827042.550792:0:2384:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff985607f0a580 x1874669228033792/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827058 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1787827042.550796:0:2378:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.203.114@tcp of length 224/224 into md 0x3238d [1] + 224 00000800:00000001:2.0:1787827042.550806:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1787827042.550807:0:2384:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1787827042.550811:0:2384:0:(client.c:3001:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787827042.550814:0:2384:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:1.0:1787827042.550817:0:2384:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1787827042.550820:0:2384:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1787827042.550823:0:2384:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1787827042.550826:0:2384:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1787827042.550833:0:2384:0:(import.c:2149:obd_at_measure()) add 5 to ffff98561118e460 time=42 v=5 (5 5 5 5) 00000100:00001000:1.0:1787827042.550838:0:2384:0:(import.c:2149:obd_at_measure()) add 1 to ffff98561118e418 time=57 v=5 (1 0 0 0) 00000100:00000001:1.0:1787827042.550841:0:2384:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1787827042.550843:0:2384:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1787827042.550846:0:2384:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1787827042.550849:0:2384:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827042.550853:0:2384:0:(client.c:1797:after_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1787827042.550856:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1787827042.550859:0:2384:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff985607f0a580 x1874669228033792/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827058 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1787827042.550861:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1787827042.550871:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000100:00000001:1.0:1787827042.550876:0:2384:0:(client.c:2354:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1787827042.550878:0:2378:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:1.0:1787827042.550881:0:2384:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1787827042.550885:0:2384:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:2.0:1787827042.550885:0:2378:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff985607f0a940 x1874669228033920/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827058 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1787827042.550893:0:2384:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff985607f0a580 x1874669228033792/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827058 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827042.550911:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607f0a580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:3c946890-15ca-497a-b838-fe1cba61f4dd:2384:1874669228033792:192.168.203.114@tcp:400:kworker.0 00000100:00000001:1.0:1787827042.550917:0:2384:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827042.550920:0:2384:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827042.550924:0:2378:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff985607f0a940 x1874669228033920/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827058 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1787827042.550926:0:2384:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607f0a580 x1874669228033792/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827058 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827042.550945:0:2384:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1787827042.550948:0:2384:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:2.0:1787827042.550950:0:2378:0:(events.c:163:reply_in_callback()) Process leaving 02000000:00000010:1.0:1787827042.550951:0:2384:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff985610db8c00. 02000000:00000001:1.0:1787827042.550957:0:2384:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000800:00000001:2.0:1787827042.550959:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000020:00000001:1.0:1787827042.550960:0:2384:0:(genops.c:1217:class_import_put()) Process entered 00000800:00000001:2.0:1787827042.550963:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000020:00000040:1.0:1787827042.550964:0:2384:0:(genops.c:1223:class_import_put()) import ffff98561118e000 refcount=3 obd=MGC192.168.203.114@tcp 00000800:00000001:2.0:1787827042.550967:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000020:00000001:1.0:1787827042.550970:0:2384:0:(genops.c:1230:class_import_put()) Process leaving 00000800:00000001:2.0:1787827042.550971:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 02000000:00000010:1.0:1787827042.550975:0:2384:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff98561238ea00. 00000400:00000200:2.0:1787827042.550976:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 02000000:00000001:1.0:1787827042.550983:0:2384:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 00000400:00000200:2.0:1787827042.550983:0:2378:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp) <- 192.168.203.114@tcp : PUT - for me 00000100:00000001:0.0:1787827042.550985:0:2383:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:1.0:1787827042.550988:0:2384:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000001:0.0:1787827042.550990:0:2383:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000400:00000200:2.0:1787827042.550990:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000010:1.0:1787827042.550991:0:2384:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607f0a580. 00000400:00000001:0.0:1787827042.550996:0:2383:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000200:2.0:1787827042.550996:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000001:1.0:1787827042.550998:0:2384:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000400:00000001:0.0:1787827042.551000:0:2383:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000001:1.0:1787827042.551002:0:2384:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827042.551003:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000010:0.0:1787827042.551005:0:2383:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9856094acf20. 00000100:00000001:1.0:1787827042.551008:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1787827042.551009:0:2378:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.203.114@tcp of length 224 into portal 10 MB=0x6a90071688800 00000400:00000200:0.0:1787827042.551012:0:2383:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985610d30660 00000100:00000001:1.0:1787827042.551012:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:2.0:1787827042.551016:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000001:1.0:1787827042.551017:0:2384:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000400:00000010:0.0:1787827042.551018:0:2383:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985610d30660. 00000100:00000001:1.0:1787827042.551021:0:2384:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787827042.551023:0:2383:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:1.0:1787827042.551025:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1787827042.551028:0:2378:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.203.114@tcp of length 224/224 into md 0x3239d [1] + 224 00000100:00000200:0.0:1787827042.551032:0:2383:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff985607f0a940 x1874669228033920/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827058 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1787827042.551035:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827042.551040:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827042.551043:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000200:0.0:1787827042.551052:0:2383:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff985607f0a940 x1874669228033920/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827058 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1787827042.551052:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000100:00000001:2.0:1787827042.551060:0:2378:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:0.0:1787827042.551064:0:2383:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1787827042.551067:0:2383:0:(client.c:3001:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:2.0:1787827042.551067:0:2378:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff985607f09e00 x1874669228034048/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827058 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787827042.551072:0:2383:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:0.0:1787827042.551075:0:2383:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1787827042.551077:0:2383:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1787827042.551079:0:2383:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1787827042.551081:0:2383:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1787827042.551093:0:2378:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff985607f09e00 x1874669228034048/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827058 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00001000:0.0:1787827042.551097:0:2383:0:(import.c:2149:obd_at_measure()) add 5 to ffff985611182460 time=120 v=5 (5 5 5 5) 00000100:00001000:0.0:1787827042.551106:0:2383:0:(import.c:2149:obd_at_measure()) add 1 to ffff985611182418 time=56 v=5 (1 0 0 0) 00000100:00000001:0.0:1787827042.551108:0:2383:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1787827042.551110:0:2383:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827042.551111:0:2383:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:2.0:1787827042.551113:0:2378:0:(events.c:163:reply_in_callback()) Process leaving 00010000:00000001:0.0:1787827042.551114:0:2383:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787827042.551119:0:2383:0:(client.c:3109:ptlrpc_free_committed()) Process entered 00000800:00000001:2.0:1787827042.551120:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000040:0.0:1787827042.551122:0:2383:0:(client.c:3121:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: skip recheck: last_committed 219043334508 00000100:00000001:0.0:1787827042.551125:0:2383:0:(client.c:3122:ptlrpc_free_committed()) Process leaving 00000800:00000001:2.0:1787827042.551126:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1787827042.551127:0:2383:0:(client.c:1797:after_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1787827042.551130:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827042.551134:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1787827042.551135:0:2383:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff985607f0a940 x1874669228033920/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827058 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787827042.551144:0:2383:0:(client.c:2354:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1787827042.551145:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1787827042.551149:0:2384:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000400:00000200:2.0:1787827042.551152:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000001:0.0:1787827042.551154:0:2383:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000400:00000001:1.0:1787827042.551154:0:2384:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1787827042.551158:0:2384:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000001:0.0:1787827042.551159:0:2383:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827042.551160:0:2378:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp) <- 192.168.203.114@tcp : PUT - for me 00000400:00000010:1.0:1787827042.551163:0:2384:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff985602ecf898. 00000100:00000040:0.0:1787827042.551167:0:2383:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff985607f0a940 x1874669228033920/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827058 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1787827042.551168:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:1.0:1787827042.551169:0:2384:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985606618cc0 00000400:00000010:1.0:1787827042.551174:0:2384:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985606618cc0. 00000400:00000200:2.0:1787827042.551178:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000001:1.0:1787827042.551180:0:2384:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000200:2.0:1787827042.551188:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000200:1.0:1787827042.551190:0:2384:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff985607f09e00 x1874669228034048/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827058 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1787827042.551197:0:2378:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.203.114@tcp of length 224 into portal 4 MB=0x6a90071688880 00000400:00000200:2.0:1787827042.551208:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000200:1.0:1787827042.551216:0:2384:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff985607f09e00 x1874669228034048/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827058 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827042.551219:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607f0a940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669228033920:192.168.203.114@tcp:400:kworker.0 00000400:00000200:2.0:1787827042.551225:0:2378:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 4 from 12345-192.168.203.114@tcp of length 224/224 into md 0x32395 [1] + 224 00000100:00000001:0.0:1787827042.551230:0:2383:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787827042.551235:0:2383:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000001:1.0:1787827042.551237:0:2384:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1787827042.551237:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000040:0.0:1787827042.551243:0:2383:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607f0a940 x1874669228033920/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827058 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827042.551243:0:2384:0:(client.c:3001:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1787827042.551246:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787827042.551249:0:2384:0:(client.c:1593:after_reply()) Process entered 00000400:00000200:2.0:1787827042.551252:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 02000000:00000001:1.0:1787827042.551253:0:2384:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1787827042.551256:0:2384:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1787827042.551259:0:2383:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1787827042.551262:0:2383:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:1.0:1787827042.551264:0:2384:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1787827042.551265:0:2383:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff98560540ac00. 00000400:00000200:2.0:1787827042.551266:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 02000000:00000001:1.0:1787827042.551269:0:2384:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1787827042.551273:0:2383:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000001:2.0:1787827042.551274:0:2378:0:(events.c:78:reply_in_callback()) Process entered 00000100:00001000:1.0:1787827042.551276:0:2384:0:(import.c:2149:obd_at_measure()) add 5 to ffff985611185c60 time=42 v=5 (5 5 5 5) 00000020:00000001:0.0:1787827042.551277:0:2383:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:0.0:1787827042.551281:0:2383:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=3 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000100:00000200:2.0:1787827042.551282:0:2378:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff985607f08f00 x1874669228034176/t0(0) o400->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827058 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00001000:1.0:1787827042.551284:0:2384:0:(import.c:2149:obd_at_measure()) add 1 to ffff985611185c18 time=42 v=5 (1 1 1 1) 00000020:00000001:0.0:1787827042.551286:0:2383:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:0.0:1787827042.551289:0:2383:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff98561238e000. 00000100:00000001:1.0:1787827042.551291:0:2384:0:(client.c:1495:ptlrpc_check_status()) Process entered 02000000:00000001:0.0:1787827042.551294:0:2383:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:1.0:1787827042.551295:0:2384:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1787827042.551298:0:2383:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1787827042.551302:0:2383:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607f0a940. 00010000:00000001:1.0:1787827042.551302:0:2384:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000040:2.0:1787827042.551305:0:2378:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff985607f08f00 x1874669228034176/t0(0) o400->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827058 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00000001:1.0:1787827042.551307:0:2384:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787827042.551308:0:2383:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1787827042.551312:0:2383:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787827042.551314:0:2384:0:(client.c:3109:ptlrpc_free_committed()) Process entered 00000100:00000001:0.0:1787827042.551317:0:2383:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787827042.551322:0:2383:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1787827042.551322:0:2378:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1787827042.551324:0:2383:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000040:1.0:1787827042.551324:0:2384:0:(client.c:3121:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff985602bc9800: skip recheck: last_committed 0 00000100:00000001:1.0:1787827042.551328:0:2384:0:(client.c:3122:ptlrpc_free_committed()) Process leaving 00000800:00000001:2.0:1787827042.551328:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1787827042.551330:0:2384:0:(client.c:1797:after_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1787827042.551334:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:0.0:1787827042.551337:0:2383:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000040:1.0:1787827042.551338:0:2384:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff985607f09e00 x1874669228034048/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827058 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000400:00000001:0.0:1787827042.551340:0:2383:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1787827042.551344:0:2383:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9856094ac370. 00000400:00000200:0.0:1787827042.551350:0:2383:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985610d30b28 00000100:00000001:1.0:1787827042.551350:0:2384:0:(client.c:2354:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1787827042.551353:0:2384:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000400:00000010:0.0:1787827042.551354:0:2383:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985610d30b28. 00000100:00000001:1.0:1787827042.551355:0:2384:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1787827042.551360:0:2383:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000040:1.0:1787827042.551361:0:2384:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff985607f09e00 x1874669228034048/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827058 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1787827042.551367:0:2383:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff985607f08f00 x1874669228034176/t0(0) o400->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827058 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827042.551371:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607f09e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669228034048:192.168.203.114@tcp:400:kworker.0 00000100:00000001:1.0:1787827042.551378:0:2384:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:0.0:1787827042.551385:0:2383:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff985607f08f00 x1874669228034176/t0(0) o400->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827058 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827042.551387:0:2384:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1787827042.551395:0:2384:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607f09e00 x1874669228034048/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827058 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787827042.551397:0:2383:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1787827042.551401:0:2383:0:(client.c:3001:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787827042.551404:0:2384:0:(client.c:2793:__ptlrpc_free_req()) Process entered 00000100:00000001:0.0:1787827042.551405:0:2383:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:0.0:1787827042.551409:0:2383:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 02000000:00000001:1.0:1787827042.551409:0:2384:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:0.0:1787827042.551410:0:2383:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1787827042.551413:0:2383:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1787827042.551414:0:2384:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff985610db8e00. 02000000:00000001:0.0:1787827042.551416:0:2383:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1787827042.551420:0:2384:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00001000:0.0:1787827042.551421:0:2383:0:(import.c:2149:obd_at_measure()) add 5 to ffff9856110b0460 time=120 v=5 (5 5 5 5) 00000020:00000001:1.0:1787827042.551424:0:2384:0:(genops.c:1217:class_import_put()) Process entered 00000100:00001000:0.0:1787827042.551426:0:2383:0:(import.c:2149:obd_at_measure()) add 1 to ffff9856110b0418 time=6 v=5 (1 0 0 0) 00000020:00000040:1.0:1787827042.551428:0:2384:0:(genops.c:1223:class_import_put()) import ffff985611185800 refcount=3 obd=lustre-MDT0001-mdc-ffff985602bc9800 00000100:00000001:0.0:1787827042.551431:0:2383:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1787827042.551434:0:2383:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827042.551436:0:2383:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000020:00000001:1.0:1787827042.551438:0:2384:0:(genops.c:1230:class_import_put()) Process leaving 00010000:00000001:0.0:1787827042.551439:0:2383:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1787827042.551443:0:2384:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff98561238e200. 00000100:00000001:0.0:1787827042.551444:0:2383:0:(client.c:3109:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1787827042.551447:0:2383:0:(client.c:3121:ptlrpc_free_committed()) lustre-OST0000-osc-ffff985602bc9800: skip recheck: last_committed 8589935703 02000000:00000001:1.0:1787827042.551449:0:2384:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:0.0:1787827042.551450:0:2383:0:(client.c:3122:ptlrpc_free_committed()) Process leaving 02000000:00000001:1.0:1787827042.551452:0:2384:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000001:0.0:1787827042.551454:0:2383:0:(client.c:1797:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:1.0:1787827042.551456:0:2384:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607f09e00. 00000100:00000040:0.0:1787827042.551462:0:2383:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff985607f08f00 x1874669228034176/t0(0) o400->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827058 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827042.551462:0:2384:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1787827042.551465:0:2384:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787827042.551469:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1787827042.551477:0:2383:0:(client.c:2354:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1787827042.551477:0:2384:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1787827042.551481:0:2383:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1787827042.551481:0:2384:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787827042.551484:0:2383:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787827042.551486:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1787827042.551489:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787827042.551490:0:2383:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff985607f08f00 x1874669228034176/t0(0) o400->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827058 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827042.551501:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607f08f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669228034176:192.168.203.114@tcp:400:kworker.0 00000100:00000001:0.0:1787827042.551505:0:2383:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1787827042.551507:0:2383:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1787827042.551510:0:2383:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607f08f00 x1874669228034176/t0(0) o400->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827058 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787827042.551516:0:2383:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1787827042.551518:0:2383:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1787827042.551520:0:2383:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff98560540a400. 02000000:00000001:0.0:1787827042.551522:0:2383:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1787827042.551523:0:2383:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:0.0:1787827042.551525:0:2383:0:(genops.c:1223:class_import_put()) import ffff9856110b0000 refcount=2 obd=lustre-OST0000-osc-ffff985602bc9800 00000020:00000001:0.0:1787827042.551527:0:2383:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:0.0:1787827042.551529:0:2383:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff98561324cf00. 02000000:00000001:0.0:1787827042.551531:0:2383:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1787827042.551532:0:2383:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1787827042.551534:0:2383:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607f08f00. 00000100:00000001:0.0:1787827042.551537:0:2383:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1787827042.551538:0:2383:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1787827042.551540:0:2383:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1787827042.551543:0:2383:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1787827042.551544:0:2383:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787827042.551546:0:2383:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1787827042.551548:0:2383:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827043.124831:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827043.124838:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827043.124843:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827043.124844:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827043.124847:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827043.124847:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1787827043.124921:0:15163:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1787827043.124926:0:15163:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1787827043.124930:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:0.0:1787827044.148826:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827044.148833:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827044.148839:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827044.148840:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827044.148843:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827044.148844:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1787827044.148900:0:33341:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1787827044.148904:0:33341:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1787827044.148907:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:0.0:1787827045.172820:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827045.172828:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827045.172835:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827045.172836:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827045.172840:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827045.172841:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1787827045.172864:0:15164:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1787827045.172865:0:15164:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1787827045.172868:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:0.0:1787827046.196785:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827046.196791:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827046.196797:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827046.196798:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827046.196802:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827046.196803:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1787827046.196824:0:33342:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1787827046.196827:0:33342:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1787827046.196829:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:0.0:1787827047.220816:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827047.220822:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827047.220829:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827047.220832:0:49842:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:0.0:1787827047.220834:0:49842:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:0.0:1787827047.220837:0:49842:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827047.220841:0:49842:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:0.0:1787827047.220843:0:49842:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827047.220845:0:49842:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827047.220848:0:49842:0:(ldlm_pool.c:496:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827047.220852:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827047.220853:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1787827047.220900:0:79887:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1787827047.220905:0:79887:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1787827047.220907:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:2.0:1787827047.668874:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1787827047.668882:0:38:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1787827047.668891:0:38:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 904 at ffff9856140d8b40. 00000020:00000040:2.0:1787827047.668897:0:38:0:(genops.c:1210:class_import_get()) import ffff98561118e000 refcount=4 obd=MGC192.168.203.114@tcp 00000100:00000001:2.0:1787827047.668901:0:38:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1787827047.668904:0:38:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1787827047.668909:0:38:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1787827047.668916:0:38:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff985615779500. 00000100:00000001:2.0:1787827047.668921:0:38:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1787827047.668930:0:38:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS req@ffff9856140d8b40 x1874669228034304/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1787827047.668940:0:38:0:(jobid.c:939:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1787827047.668943:0:38:0:(jobid.c:979:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1787827047.668948:0:38:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000056fda74b] to pc [ptlrpcd_01_00+0] req@ffff9856140d8b40 x1874669228034304/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827047.668976:0:38:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1787827047.668979:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1787827047.668981:0:38:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1787827047.668984:0:38:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 904 at ffff9856140dbc00. 00000020:00000040:2.0:1787827047.668987:0:38:0:(genops.c:1210:class_import_get()) import ffff985611182000 refcount=4 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000100:00000001:2.0:1787827047.668990:0:38:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1787827047.668991:0:38:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1787827047.668993:0:38:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1787827047.669004:0:38:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff985615779000. 00000100:00000001:3.0:1787827047.669007:0:2385:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1787827047.669009:0:38:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827047.669012:0:2385:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000040:2.0:1787827047.669017:0:38:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID req@ffff9856140dbc00 x1874669228034432/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1787827047.669017:0:2385:0:(client.c:1813:ptlrpc_send_new_req()) Process entered 00000100:00000040:3.0:1787827047.669027:0:2385:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9856140d8b40 x1874669228034304/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827047.669029:0:38:0:(jobid.c:939:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1787827047.669034:0:38:0:(jobid.c:979:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827047.669042:0:2385:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000040:2.0:1787827047.669043:0:38:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000f0935ea7] to pc [ptlrpcd_01_01+1] req@ffff9856140dbc00 x1874669228034432/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827047.669046:0:2385:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1787827047.669051:0:2385:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1787827047.669055:0:2385:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1787827047.669057:0:38:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1787827047.669059:0:2385:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1787827047.669061:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1787827047.669065:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2385:1874669228034304:192.168.203.114@tcp:400:kworker.0 00000100:00000001:3.0:1787827047.669070:0:2385:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:2.0:1787827047.669075:0:38:0:(pinger.c:119:ptlrpc_ping()) Process entered 02000000:00000001:3.0:1787827047.669076:0:2385:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 00000100:00000010:2.0:1787827047.669078:0:38:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 904 at ffff9856140d9a40. 02000000:00000001:3.0:1787827047.669079:0:2385:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:2.0:1787827047.669083:0:38:0:(genops.c:1210:class_import_get()) import ffff985611185800 refcount=4 obd=lustre-MDT0001-mdc-ffff985602bc9800 02000000:00000001:3.0:1787827047.669084:0:2385:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:2.0:1787827047.669088:0:38:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1787827047.669090:0:38:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000010:3.0:1787827047.669091:0:2385:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff985611861e00. 02000000:00000001:2.0:1787827047.669093:0:38:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1787827047.669094:0:2385:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1787827047.669099:0:38:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff985615779100. 00000400:00000200:3.0:1787827047.669104:0:2385:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000001:2.0:1787827047.669105:0:38:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1787827047.669109:0:2385:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff985602ad93c8. 00000100:00000040:2.0:1787827047.669113:0:38:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID req@ffff9856140d9a40 x1874669228034560/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000400:00000010:3.0:1787827047.669117:0:2385:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985609573bb0. 00000100:00000200:3.0:1787827047.669126:0:2385:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1874669228034304, portal 25 00000100:00000001:2.0:1787827047.669128:0:38:0:(jobid.c:939:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1787827047.669131:0:38:0:(jobid.c:979:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827047.669131:0:2385:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1787827047.669135:0:2385:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446630094203882304 : -113979505669312 : ffff9856140d8b40) 00000100:00000040:2.0:1787827047.669139:0:38:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000fb315b6c] to pc [ptlrpcd_01_00+0] req@ffff9856140d9a40 x1874669228034560/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1787827047.669145:0:2385:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9856140d8b40 x1874669228034304/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827063 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827047.669158:0:38:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827047.669160:0:2385:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:2.0:1787827047.669162:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-OST0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1787827047.669167:0:38:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1787827047.669172:0:38:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 904 at ffff9856140d8f00. 00000020:00000040:2.0:1787827047.669173:0:38:0:(genops.c:1210:class_import_get()) import ffff9856110b0000 refcount=3 obd=lustre-OST0000-osc-ffff985602bc9800 00000100:00000040:3.0:1787827047.669176:0:2385:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.203.114@tcp 00000100:00000001:2.0:1787827047.669177:0:38:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1787827047.669180:0:38:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1787827047.669182:0:38:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1787827047.669183:0:2385:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985609573770. 02000000:00000010:2.0:1787827047.669185:0:38:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff985615779d00. 00000100:00000001:2.0:1787827047.669189:0:38:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:3.0:1787827047.669189:0:2385:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1874669228034304, offset 0 00000100:00000040:2.0:1787827047.669195:0:38:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-OST0000_UUID req@ffff9856140d8f00 x1874669228034688/t0(0) o400->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000400:00000200:3.0:1787827047.669202:0:2385:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.203.114@tcp 00000100:00000001:2.0:1787827047.669203:0:38:0:(jobid.c:939:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1787827047.669206:0:38:0:(jobid.c:979:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1787827047.669210:0:38:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000e9d4c698] to pc [ptlrpcd_01_01+1] req@ffff9856140d8f00 x1874669228034688/t0(0) o400->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827047.669217:0:38:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1787827047.669217:0:2385:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000040:2.0:1787827047.669221:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:2.0:1787827047.669224:0:38:0:(pinger.c:247:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000040:2.0:1787827047.669227:0:38:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (3203) 00000400:00000200:3.0:1787827047.669228:0:2385:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.203.114@tcp local destination 00000400:00000200:3.0:1787827047.669235:0:2385:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:3.0:1787827047.669239:0:2385:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.203.14@tcp 00000400:00000200:3.0:1787827047.669243:0:2385:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.203.14@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1787827047.669246:0:2385:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.203.114@tcp 00000400:00000200:3.0:1787827047.669250:0:2385:0:(lib-move.c:1745:lnet_handle_send()) 192.168.203.14@tcp NI seq info: [21906:21906:256:4294967295] 192.168.203.114@tcp LPNI seq info [21906:21906:8:4294967295] 00000100:00000001:2.0:1787827047.669251:0:2386:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1787827047.669253:0:2386:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1787827047.669256:0:2386:0:(client.c:1813:ptlrpc_send_new_req()) Process entered 00000400:00000200:3.0:1787827047.669259:0:2385:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000040:2.0:1787827047.669261:0:2386:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9856140dbc00 x1874669228034432/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1787827047.669276:0:2385:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp:) -> 192.168.203.114@tcp(192.168.203.114@tcp:192.168.203.114@tcp) : PUT try# 0 00000100:00000001:2.0:1787827047.669280:0:2386:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1787827047.669283:0:2386:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1787827047.669286:0:2386:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1787827047.669288:0:2386:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000800:00000200:3.0:1787827047.669292:0:2385:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.203.114@tcp 02000000:00000001:2.0:1787827047.669293:0:2386:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1787827047.669300:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228034432:192.168.203.114@tcp:400:kworker.0 00000800:00000010:3.0:1787827047.669300:0:2385:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff985609025f00. 00000100:00000001:2.0:1787827047.669307:0:2386:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1787827047.669309:0:2386:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 00000800:00000200:3.0:1787827047.669311:0:2385:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff985606751480] -> 12345-192.168.203.114@tcp (5) 02000000:00000001:2.0:1787827047.669312:0:2386:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1787827047.669316:0:2386:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:2.0:1787827047.669320:0:2386:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff985607e32800. 02000000:00000001:2.0:1787827047.669322:0:2386:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1787827047.669326:0:2385:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.203.114@tcp ip 192.168.203.114:988 00000400:00000200:2.0:1787827047.669331:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000800:00000200:3.0:1787827047.669332:0:2385:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff985609025f00 type 1, nob 320 niov 1 nkiov 1 00000400:00000010:2.0:1787827047.669335:0:2386:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff98561080ea50. 00000400:00000010:2.0:1787827047.669341:0:2386:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985610afa000. 00000100:00000001:3.0:1787827047.669344:0:2385:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:2.0:1787827047.669345:0:2386:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1874669228034432, portal 10 00000100:00000001:2.0:1787827047.669349:0:2386:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000400:00000200:3.0:1787827047.669352:0:2385:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000001:2.0:1787827047.669353:0:2386:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446630094203894784 : -113979505656832 : ffff9856140dbc00) 00000100:00000001:3.0:1787827047.669356:0:2385:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.203.114@tcp: PM QoS new deadline estimation for cpu 2 is 4297875708 00000100:00000040:2.0:1787827047.669361:0:2386:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9856140dbc00 x1874669228034432/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827063 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827047.669364:0:2385:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.203.114@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4297870588 00000100:00000001:3.0:1787827047.669371:0:2385:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.203.114@tcp: PM QoS new deadline estimation for cpu 3 is 4297875708 00000100:00000001:2.0:1787827047.669372:0:2386:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:2.0:1787827047.669376:0:2386:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.203.114@tcp 00000100:00000001:3.0:1787827047.669378:0:2385:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.203.114@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4297870588 00000400:00000010:2.0:1787827047.669379:0:2386:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985610afa6e8. 00000100:00000001:3.0:1787827047.669384:0:2385:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1787827047.669386:0:2385:0:(client.c:1941:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827047.669390:0:2385:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827047.669393:0:2385:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:2.0:1787827047.669394:0:2386:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1874669228034432, offset 0 00000400:00000200:2.0:1787827047.669401:0:2386:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.203.114@tcp 00000400:00000200:2.0:1787827047.669406:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827047.669409:0:2386:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.203.114@tcp local destination 00000400:00000200:2.0:1787827047.669412:0:2386:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:2.0:1787827047.669415:0:2386:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.203.14@tcp 00000400:00000200:2.0:1787827047.669417:0:2386:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.203.14@tcp lpni_is_preferred = 0 00000400:00000200:2.0:1787827047.669419:0:2386:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.203.114@tcp 00000400:00000200:2.0:1787827047.669423:0:2386:0:(lib-move.c:1745:lnet_handle_send()) 192.168.203.14@tcp NI seq info: [21907:21907:255:4294967295] 192.168.203.114@tcp LPNI seq info [21907:21907:7:4294967295] 00000400:00000200:2.0:1787827047.669427:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827047.669432:0:2386:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp:) -> 192.168.203.114@tcp(192.168.203.114@tcp:192.168.203.114@tcp) : PUT try# 0 00000800:00000200:2.0:1787827047.669435:0:2386:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.203.114@tcp 00000800:00000010:2.0:1787827047.669438:0:2386:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff985615779a00. 00000100:00000001:3.0:1787827047.669441:0:2385:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000800:00000200:2.0:1787827047.669442:0:2386:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff985606751480] -> 12345-192.168.203.114@tcp (5) 00000100:00000001:3.0:1787827047.669445:0:2385:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000800:00000200:2.0:1787827047.669450:0:2386:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.203.114@tcp ip 192.168.203.114:988 00000100:00000001:3.0:1787827047.669451:0:2385:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1787827047.669454:0:2385:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000800:00000200:2.0:1787827047.669455:0:2386:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff985615779a00 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1787827047.669456:0:2385:0:(client.c:1813:ptlrpc_send_new_req()) Process entered 00000100:00000001:2.0:1787827047.669461:0:2386:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1787827047.669463:0:2385:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9856140d9a40 x1874669228034560/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1787827047.669467:0:2386:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff985602bc9800: using a timeout of 38046 usecs (39 jiffies) 00000400:00000200:2.0:1787827047.669476:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000001:3.0:1787827047.669477:0:2385:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1787827047.669480:0:2386:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff985602bc9800: PM QoS new deadline estimation for cpu 2 is 4297865767 00000100:00000001:3.0:1787827047.669480:0:2385:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1787827047.669486:0:2386:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff985602bc9800: PM QoS request active for cpu 2, keep current deadline 4297875708 02000000:00000001:3.0:1787827047.669486:0:2385:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1787827047.669488:0:2385:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1787827047.669490:0:2386:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff985602bc9800: PM QoS new deadline estimation for cpu 3 is 4297865767 02000000:00000001:3.0:1787827047.669492:0:2385:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1787827047.669494:0:2386:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff985602bc9800: PM QoS request active for cpu 3, keep current deadline 4297875708 00000100:00000001:2.0:1787827047.669498:0:2386:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1787827047.669500:0:2386:0:(client.c:1941:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1787827047.669503:0:2386:0:(client.c:1813:ptlrpc_send_new_req()) Process entered 00000100:00100000:3.0:1787827047.669506:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669228034560:192.168.203.114@tcp:400:kworker.0 00000100:00000040:2.0:1787827047.669510:0:2386:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9856140d8f00 x1874669228034688/t0(0) o400->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827047.669514:0:2385:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:3.0:1787827047.669518:0:2385:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1787827047.669520:0:2385:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1787827047.669521:0:2386:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1787827047.669525:0:2386:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1787827047.669525:0:2385:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000001:2.0:1787827047.669530:0:2386:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000010:3.0:1787827047.669531:0:2385:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff985611860e00. 02000000:00000001:2.0:1787827047.669532:0:2386:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1787827047.669535:0:2385:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1787827047.669536:0:2386:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1787827047.669542:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228034688:192.168.203.114@tcp:400:kworker.0 00000400:00000200:3.0:1787827047.669542:0:2385:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000010:3.0:1787827047.669548:0:2385:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff985602ad9790. 00000100:00000001:2.0:1787827047.669549:0:2386:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1787827047.669553:0:2386:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 00000400:00000010:3.0:1787827047.669554:0:2385:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985609573088. 02000000:00000001:2.0:1787827047.669556:0:2386:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:3.0:1787827047.669559:0:2385:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1874669228034560, portal 10 02000000:00000001:2.0:1787827047.669560:0:2386:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:3.0:1787827047.669563:0:2385:0:(client.c:3275:ptlrpc_request_addref()) Process entered 02000000:00000010:2.0:1787827047.669567:0:2386:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff985607e33c00. 00000100:00000001:3.0:1787827047.669567:0:2385:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446630094203886144 : -113979505665472 : ffff9856140d9a40) 02000000:00000001:2.0:1787827047.669572:0:2386:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1787827047.669577:0:2385:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9856140d9a40 x1874669228034560/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827063 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1787827047.669578:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000010:2.0:1787827047.669584:0:2386:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff98561080eb00. 00000400:00000010:2.0:1787827047.669591:0:2386:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985610afa3b8. 00000100:00000001:3.0:1787827047.669591:0:2385:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1787827047.669596:0:2385:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.203.114@tcp 00000100:00000200:2.0:1787827047.669597:0:2386:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1874669228034688, portal 4 00000400:00000010:3.0:1787827047.669600:0:2385:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9856095733b8. 00000100:00000001:2.0:1787827047.669601:0:2386:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1787827047.669604:0:2386:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446630094203883264 : -113979505668352 : ffff9856140d8f00) 00000100:00000200:3.0:1787827047.669605:0:2385:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1874669228034560, offset 0 00000400:00000200:3.0:1787827047.669613:0:2385:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.203.114@tcp 00000100:00000040:2.0:1787827047.669615:0:2386:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9856140d8f00 x1874669228034688/t0(0) o400->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827063 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1787827047.669620:0:2385:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:3.0:1787827047.669626:0:2385:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.203.114@tcp local destination 00000100:00000001:2.0:1787827047.669629:0:2386:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000400:00000200:3.0:1787827047.669632:0:2385:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000040:2.0:1787827047.669636:0:2386:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.203.114@tcp 00000400:00000200:3.0:1787827047.669637:0:2385:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.203.14@tcp 00000400:00000010:2.0:1787827047.669641:0:2386:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985610afab28. 00000400:00000200:3.0:1787827047.669641:0:2385:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.203.14@tcp lpni_is_preferred = 0 00000100:00000200:2.0:1787827047.669646:0:2386:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 28, xid 1874669228034688, offset 0 00000400:00000200:3.0:1787827047.669647:0:2385:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.203.114@tcp 00000400:00000200:3.0:1787827047.669653:0:2385:0:(lib-move.c:1745:lnet_handle_send()) 192.168.203.14@tcp NI seq info: [21908:21908:254:4294967295] 192.168.203.114@tcp LPNI seq info [21908:21908:6:4294967295] 00000400:00000200:3.0:1787827047.669672:0:2385:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827047.669673:0:2386:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.203.114@tcp 00000400:00000200:3.0:1787827047.669682:0:2385:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp:) -> 192.168.203.114@tcp(192.168.203.114@tcp:192.168.203.114@tcp) : PUT try# 0 00000800:00000200:3.0:1787827047.669688:0:2385:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.203.114@tcp 00000400:00000200:2.0:1787827047.669713:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827047.669719:0:2386:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.203.114@tcp local destination 00000400:00000200:2.0:1787827047.669722:0:2386:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:2.0:1787827047.669727:0:2386:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.203.14@tcp 00000800:00000010:3.0:1787827047.669727:0:2385:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff985609025200. 00000800:00000200:3.0:1787827047.669735:0:2385:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff985606751480] -> 12345-192.168.203.114@tcp (5) 00000400:00000200:2.0:1787827047.669741:0:2386:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.203.14@tcp lpni_is_preferred = 0 00000800:00000200:3.0:1787827047.669743:0:2385:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.203.114@tcp ip 192.168.203.114:988 00000400:00000200:2.0:1787827047.669748:0:2386:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.203.114@tcp 00000800:00000200:3.0:1787827047.669749:0:2385:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff985609025200 type 1, nob 320 niov 1 nkiov 1 00000400:00000200:2.0:1787827047.669754:0:2386:0:(lib-move.c:1745:lnet_handle_send()) 192.168.203.14@tcp NI seq info: [21909:21909:253:4294967295] 192.168.203.114@tcp LPNI seq info [21909:21909:5:4294967295] 00000400:00000200:2.0:1787827047.669759:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000001:3.0:1787827047.669762:0:2385:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1787827047.669767:0:2385:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff985602bc9800: using a timeout of 11928 usecs (12 jiffies) 00000400:00000200:2.0:1787827047.669768:0:2386:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp:) -> 192.168.203.114@tcp(192.168.203.114@tcp:192.168.203.114@tcp) : PUT try# 0 00000400:00000200:3.0:1787827047.669775:0:2385:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000800:00000200:2.0:1787827047.669779:0:2386:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.203.114@tcp 00000100:00000001:3.0:1787827047.669780:0:2385:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff985602bc9800: PM QoS new deadline estimation for cpu 2 is 4297865741 00000100:00000001:3.0:1787827047.669785:0:2385:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff985602bc9800: PM QoS request active for cpu 2, keep current deadline 4297875708 00000800:00000010:2.0:1787827047.669786:0:2386:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff985615779b00. 00000100:00000001:3.0:1787827047.669788:0:2385:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff985602bc9800: PM QoS new deadline estimation for cpu 3 is 4297865741 00000100:00000001:3.0:1787827047.669791:0:2385:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff985602bc9800: PM QoS request active for cpu 3, keep current deadline 4297875708 00000100:00000001:3.0:1787827047.669796:0:2385:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000800:00000200:2.0:1787827047.669799:0:2386:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff985606751480] -> 12345-192.168.203.114@tcp (5) 00000100:00000001:3.0:1787827047.669800:0:2385:0:(client.c:1941:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827047.669806:0:2385:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1787827047.669810:0:2386:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.203.114@tcp ip 192.168.203.114:988 00000100:00000001:3.0:1787827047.669811:0:2385:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000800:00000200:2.0:1787827047.669816:0:2386:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff985615779b00 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1787827047.669816:0:2385:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1787827047.669819:0:2385:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1787827047.669822:0:2386:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827047.669823:0:2385:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1787827047.669826:0:2385:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000040:2.0:1787827047.669827:0:2386:0:(niobuf.c:635:kick_cpu_latency()) lustre-OST0000-osc-ffff985602bc9800: using a timeout of 14633 usecs (15 jiffies) 00000100:00000001:3.0:1787827047.669828:0:2385:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1787827047.669832:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000001:3.0:1787827047.669833:0:2385:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1787827047.669835:0:2386:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff985602bc9800: PM QoS new deadline estimation for cpu 2 is 4297865744 00000100:00000001:2.0:1787827047.669839:0:2386:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff985602bc9800: PM QoS request active for cpu 2, keep current deadline 4297875708 00000100:00000001:2.0:1787827047.669840:0:2386:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff985602bc9800: PM QoS new deadline estimation for cpu 3 is 4297865744 00000100:00000001:2.0:1787827047.669842:0:2386:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff985602bc9800: PM QoS request active for cpu 3, keep current deadline 4297875708 00000100:00000001:2.0:1787827047.669843:0:2386:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1787827047.669845:0:2386:0:(client.c:1941:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1787827047.669846:0:2386:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1787827047.669848:0:2386:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1787827047.669851:0:2386:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1787827047.669852:0:2386:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1787827047.669853:0:2386:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1787827047.669855:0:2386:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1787827047.669856:0:2386:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1787827047.669857:0:2386:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1787827047.669870:0:2377:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1787827047.669873:0:2377:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff985609025f00. 00000400:00000200:3.0:1787827047.669879:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1787827047.669883:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000400:00000200:3.0:1787827047.669886:0:2377:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985609573770 00000400:00000010:3.0:1787827047.669888:0:2377:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985609573770. 00000100:00000001:3.0:1787827047.669890:0:2377:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1787827047.669894:0:2377:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff9856140d8b40 x1874669228034304/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827063 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827047.669900:0:2377:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1787827047.669903:0:2377:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9856140d8b40 x1874669228034304/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827063 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827047.669907:0:2377:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827047.669908:0:2377:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1787827047.669916:0:2377:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1787827047.669917:0:2377:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff985615779a00. 00000400:00000200:3.0:1787827047.669920:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1787827047.669922:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000400:00000200:3.0:1787827047.669923:0:2377:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985610afa6e8 00000400:00000010:3.0:1787827047.669924:0:2377:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985610afa6e8. 00000100:00000001:3.0:1787827047.669926:0:2377:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1787827047.669927:0:2377:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff9856140dbc00 x1874669228034432/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827063 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827047.669931:0:2377:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1787827047.669933:0:2377:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9856140dbc00 x1874669228034432/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827063 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827047.669937:0:2377:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827047.669938:0:2377:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1787827047.669941:0:2377:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1787827047.669942:0:2377:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff985609025200. 00000400:00000200:3.0:1787827047.669943:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1787827047.669945:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000400:00000200:3.0:1787827047.669946:0:2377:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9856095733b8 00000400:00000010:3.0:1787827047.669947:0:2377:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9856095733b8. 00000100:00000001:3.0:1787827047.669948:0:2377:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1787827047.669950:0:2377:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff9856140d9a40 x1874669228034560/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827063 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827047.669953:0:2377:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1787827047.669955:0:2377:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9856140d9a40 x1874669228034560/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827063 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827047.669959:0:2377:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827047.669959:0:2377:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1787827047.670010:0:2377:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1787827047.670013:0:2377:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff985615779b00. 00000400:00000200:3.0:1787827047.670015:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1787827047.670019:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000400:00000200:3.0:1787827047.670021:0:2377:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985610afab28 00000400:00000010:3.0:1787827047.670023:0:2377:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985610afab28. 00000100:00000001:3.0:1787827047.670025:0:2377:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1787827047.670029:0:2377:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff9856140d8f00 x1874669228034688/t0(0) o400->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827063 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827047.670035:0:2377:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1787827047.670036:0:2377:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9856140d8f00 x1874669228034688/t0(0) o400->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827063 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827047.670040:0:2377:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827047.670041:0:2377:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1787827047.671208:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827047.671226:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1787827047.671229:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827047.671232:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827047.671238:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827047.671243:0:2378:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp) <- 192.168.203.114@tcp : PUT - for me 00000400:00000200:2.0:1787827047.671267:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827047.671270:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827047.671274:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827047.671278:0:2378:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.203.114@tcp of length 224 into portal 25 MB=0x6a90071688900 00000400:00000200:2.0:1787827047.671282:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827047.671288:0:2378:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.203.114@tcp of length 224/224 into md 0x323a5 [1] + 224 00000800:00000001:2.0:1787827047.671294:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827047.671299:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827047.671302:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1787827047.671305:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000100:00000001:2.0:1787827047.671311:0:2378:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1787827047.671317:0:2378:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff9856140d8b40 x1874669228034304/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827063 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1787827047.671326:0:2378:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9856140d8b40 x1874669228034304/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827063 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827047.671352:0:2378:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1787827047.671376:0:2385:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000001:2.0:1787827047.671377:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:3.0:1787827047.671380:0:2385:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000800:00000001:2.0:1787827047.671384:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:3.0:1787827047.671386:0:2385:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000800:00000001:2.0:1787827047.671387:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000400:00000001:3.0:1787827047.671390:0:2385:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000800:00000001:2.0:1787827047.671392:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:3.0:1787827047.671395:0:2385:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff985602ad93c8. 00000400:00000200:2.0:1787827047.671399:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:3.0:1787827047.671401:0:2385:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985609573bb0 00000400:00000010:3.0:1787827047.671405:0:2385:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985609573bb0. 00000400:00000200:2.0:1787827047.671406:0:2378:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp) <- 192.168.203.114@tcp : PUT - for me 00000100:00000001:3.0:1787827047.671410:0:2385:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000200:2.0:1787827047.671414:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000200:3.0:1787827047.671418:0:2385:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff9856140d8b40 x1874669228034304/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827063 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1787827047.671420:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827047.671427:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827047.671434:0:2378:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.203.114@tcp of length 224 into portal 10 MB=0x6a90071688980 00000100:00000200:3.0:1787827047.671438:0:2385:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff9856140d8b40 x1874669228034304/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827063 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827047.671446:0:2385:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1787827047.671448:0:2385:0:(client.c:3001:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1787827047.671451:0:2385:0:(client.c:1593:after_reply()) Process entered 00000400:00000200:2.0:1787827047.671453:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 02000000:00000001:3.0:1787827047.671455:0:2385:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1787827047.671458:0:2385:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000400:00000200:2.0:1787827047.671460:0:2378:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.203.114@tcp of length 224/224 into md 0x323b5 [1] + 224 00000100:00000001:3.0:1787827047.671461:0:2385:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1787827047.671466:0:2385:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1787827047.671469:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00001000:3.0:1787827047.671474:0:2385:0:(import.c:2149:obd_at_measure()) add 5 to ffff98561118e460 time=47 v=5 (5 5 5 5) 00000800:00000001:2.0:1787827047.671475:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:3.0:1787827047.671481:0:2385:0:(import.c:2149:obd_at_measure()) add 1 to ffff98561118e418 time=62 v=5 (1 0 0 0) 00000100:00000001:3.0:1787827047.671484:0:2385:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1787827047.671487:0:2385:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1787827047.671488:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00010000:00000001:3.0:1787827047.671491:0:2385:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000400:00000200:2.0:1787827047.671495:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00010000:00000001:3.0:1787827047.671495:0:2385:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827047.671501:0:2385:0:(client.c:1797:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1787827047.671502:0:2378:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000040:3.0:1787827047.671507:0:2385:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9856140d8b40 x1874669228034304/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827063 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1787827047.671509:0:2378:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff9856140dbc00 x1874669228034432/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827063 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827047.671518:0:2385:0:(client.c:2354:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1787827047.671523:0:2385:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1787827047.671526:0:2385:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827047.671531:0:2378:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9856140dbc00 x1874669228034432/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827063 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1787827047.671535:0:2385:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9856140d8b40 x1874669228034304/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827063 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827047.671545:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2385:1874669228034304:192.168.203.114@tcp:400:kworker.0 00000100:00000001:3.0:1787827047.671551:0:2385:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827047.671553:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1787827047.671559:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9856140d8b40 x1874669228034304/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827063 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827047.671567:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 00000100:00000001:2.0:1787827047.671569:0:2378:0:(events.c:163:reply_in_callback()) Process leaving 02000000:00000001:3.0:1787827047.671569:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1787827047.671574:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff985611861e00. 00000800:00000001:2.0:1787827047.671576:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 02000000:00000001:3.0:1787827047.671578:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000800:00000001:2.0:1787827047.671582:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000020:00000001:3.0:1787827047.671582:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:3.0:1787827047.671585:0:2385:0:(genops.c:1223:class_import_put()) import ffff98561118e000 refcount=3 obd=MGC192.168.203.114@tcp 00000800:00000001:2.0:1787827047.671586:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000020:00000001:3.0:1787827047.671590:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 00000800:00000001:2.0:1787827047.671591:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 02000000:00000010:3.0:1787827047.671594:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff985615779500. 02000000:00000001:3.0:1787827047.671601:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 00000400:00000200:2.0:1787827047.671603:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 02000000:00000001:3.0:1787827047.671603:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1787827047.671606:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff9856140d8b40. 00000100:00000001:3.0:1787827047.671610:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1787827047.671612:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1787827047.671617:0:2385:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1787827047.671618:0:2378:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp) <- 192.168.203.114@tcp : PUT - for me 00000100:00000001:3.0:1787827047.671621:0:2385:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1787827047.671624:0:2385:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000400:00000200:2.0:1787827047.671626:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000001:3.0:1787827047.671627:0:2385:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827047.671631:0:2385:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1787827047.671632:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827047.671643:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827047.671646:0:2378:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.203.114@tcp of length 224 into portal 10 MB=0x6a90071688a00 00000400:00000200:2.0:1787827047.671650:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000001:3.0:1787827047.671652:0:2386:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:2.0:1787827047.671665:0:2378:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.203.114@tcp of length 224/224 into md 0x323c5 [1] + 224 00000100:00000001:3.0:1787827047.671666:0:2386:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000400:00000001:3.0:1787827047.671671:0:2386:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000800:00000001:2.0:1787827047.671672:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000400:00000001:3.0:1787827047.671673:0:2386:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1787827047.671676:0:2386:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff98561080ea50. 00000800:00000001:2.0:1787827047.671679:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1787827047.671679:0:2386:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985610afa000 00000400:00000010:3.0:1787827047.671682:0:2386:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985610afa000. 00000100:00000001:3.0:1787827047.671684:0:2386:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1787827047.671688:0:2386:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff9856140dbc00 x1874669228034432/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827063 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1787827047.671718:0:2386:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff9856140dbc00 x1874669228034432/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827063 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1787827047.671719:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1787827047.671728:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000100:00000001:3.0:1787827047.671730:0:2386:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1787827047.671733:0:2378:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:3.0:1787827047.671734:0:2386:0:(client.c:3001:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1787827047.671738:0:2386:0:(client.c:1593:after_reply()) Process entered 00000100:00000200:2.0:1787827047.671740:0:2378:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff9856140d9a40 x1874669228034560/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827063 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1787827047.671740:0:2386:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1787827047.671743:0:2386:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1787827047.671746:0:2386:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1787827047.671750:0:2386:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1787827047.671757:0:2378:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9856140d9a40 x1874669228034560/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827063 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00001000:3.0:1787827047.671758:0:2386:0:(import.c:2149:obd_at_measure()) add 5 to ffff985611182460 time=125 v=5 (5 5 5 5) 00000100:00001000:3.0:1787827047.671764:0:2386:0:(import.c:2149:obd_at_measure()) add 1 to ffff985611182418 time=61 v=5 (1 0 0 0) 00000100:00000001:3.0:1787827047.671766:0:2386:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1787827047.671767:0:2386:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1787827047.671768:0:2386:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1787827047.671770:0:2386:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827047.671774:0:2386:0:(client.c:3109:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1787827047.671777:0:2386:0:(client.c:3121:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: skip recheck: last_committed 219043334508 00000100:00000001:3.0:1787827047.671779:0:2386:0:(client.c:3122:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1787827047.671782:0:2386:0:(client.c:1797:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1787827047.671788:0:2378:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000040:3.0:1787827047.671794:0:2386:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9856140dbc00 x1874669228034432/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827063 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827047.671801:0:2386:0:(client.c:2354:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000800:00000001:2.0:1787827047.671802:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:3.0:1787827047.671805:0:2386:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1787827047.671808:0:2386:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1787827047.671809:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1787827047.671813:0:2386:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9856140dbc00 x1874669228034432/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827063 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1787827047.671814:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827047.671817:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827047.671826:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00100000:3.0:1787827047.671830:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228034432:192.168.203.114@tcp:400:kworker.0 00000100:00000001:3.0:1787827047.671836:0:2386:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827047.671838:0:2386:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1787827047.671841:0:2386:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9856140dbc00 x1874669228034432/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827063 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1787827047.671844:0:2378:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp) <- 192.168.203.114@tcp : PUT - for me 00000100:00000001:3.0:1787827047.671853:0:2386:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1787827047.671855:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 00000400:00000200:2.0:1787827047.671856:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 02000000:00000010:3.0:1787827047.671858:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff985607e32800. 02000000:00000001:3.0:1787827047.671861:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1787827047.671862:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:3.0:1787827047.671864:0:2386:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=3 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:3.0:1787827047.671866:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 00000400:00000200:2.0:1787827047.671868:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 02000000:00000010:3.0:1787827047.671869:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff985615779000. 02000000:00000001:3.0:1787827047.671874:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1787827047.671876:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000400:00000200:2.0:1787827047.671877:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000010:3.0:1787827047.671879:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff9856140dbc00. 00000400:00000200:2.0:1787827047.671884:0:2378:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.203.114@tcp of length 224 into portal 4 MB=0x6a90071688a80 00000100:00000001:3.0:1787827047.671884:0:2386:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1787827047.671887:0:2386:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827047.671891:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000001:3.0:1787827047.671891:0:2386:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827047.671895:0:2386:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1787827047.671897:0:2386:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1787827047.671899:0:2386:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827047.671901:0:2386:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1787827047.671906:0:2378:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 4 from 12345-192.168.203.114@tcp of length 224/224 into md 0x323cd [1] + 224 00000100:00000001:3.0:1787827047.671911:0:2385:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000001:2.0:1787827047.671912:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:3.0:1787827047.671913:0:2385:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000400:00000001:3.0:1787827047.671917:0:2385:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1787827047.671918:0:2385:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1787827047.671920:0:2385:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff985602ad9790. 00000800:00000001:2.0:1787827047.671922:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1787827047.671923:0:2385:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985609573088 00000400:00000010:3.0:1787827047.671925:0:2385:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985609573088. 00000400:00000200:2.0:1787827047.671928:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:3.0:1787827047.671929:0:2385:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1787827047.671933:0:2385:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff9856140d9a40 x1874669228034560/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827063 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1787827047.671937:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000100:00000001:2.0:1787827047.671946:0:2378:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1787827047.671946:0:2385:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff9856140d9a40 x1874669228034560/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827063 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1787827047.671954:0:2378:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff9856140d8f00 x1874669228034688/t0(0) o400->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827063 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827047.671954:0:2385:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1787827047.671957:0:2385:0:(client.c:3001:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1787827047.671966:0:2385:0:(client.c:1593:after_reply()) Process entered 00000100:00000040:2.0:1787827047.671968:0:2378:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9856140d8f00 x1874669228034688/t0(0) o400->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827063 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1787827047.671968:0:2385:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1787827047.671971:0:2385:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1787827047.671974:0:2385:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1787827047.671979:0:2385:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1787827047.671985:0:2385:0:(import.c:2149:obd_at_measure()) add 5 to ffff985611185c60 time=47 v=5 (5 5 5 5) 00000100:00001000:3.0:1787827047.671989:0:2385:0:(import.c:2149:obd_at_measure()) add 1 to ffff985611185c18 time=47 v=5 (1 1 1 1) 00000100:00000001:2.0:1787827047.671991:0:2378:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1787827047.671992:0:2385:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1787827047.671995:0:2385:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1787827047.671997:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00010000:00000001:3.0:1787827047.671997:0:2385:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1787827047.672002:0:2385:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1787827047.672004:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1787827047.672005:0:2385:0:(client.c:3109:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1787827047.672007:0:2385:0:(client.c:3121:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff985602bc9800: skip recheck: last_committed 0 00000100:00000001:3.0:1787827047.672010:0:2385:0:(client.c:3122:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1787827047.672011:0:2385:0:(client.c:1797:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1787827047.672015:0:2385:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9856140d9a40 x1874669228034560/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827063 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827047.672017:0:2386:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1787827047.672021:0:2386:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000400:00000001:2.0:1787827047.672025:0:2386:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000001:3.0:1787827047.672025:0:2385:0:(client.c:2354:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000400:00000001:2.0:1787827047.672028:0:2386:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000001:3.0:1787827047.672030:0:2385:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000400:00000010:2.0:1787827047.672032:0:2386:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff98561080eb00. 00000100:00000001:3.0:1787827047.672033:0:2385:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827047.672039:0:2386:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985610afa3b8 00000100:00000040:3.0:1787827047.672040:0:2385:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9856140d9a40 x1874669228034560/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827063 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000400:00000010:2.0:1787827047.672044:0:2386:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985610afa3b8. 00000100:00000001:2.0:1787827047.672048:0:2386:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1787827047.672055:0:2386:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff9856140d8f00 x1874669228034688/t0(0) o400->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827063 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827047.672057:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669228034560:192.168.203.114@tcp:400:kworker.0 00000100:00000200:2.0:1787827047.672066:0:2386:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff9856140d8f00 x1874669228034688/t0(0) o400->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827063 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827047.672078:0:2385:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1787827047.672081:0:2386:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1787827047.672082:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000001:2.0:1787827047.672084:0:2386:0:(client.c:3001:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1787827047.672088:0:2386:0:(client.c:1593:after_reply()) Process entered 00000100:00000040:3.0:1787827047.672089:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9856140d9a40 x1874669228034560/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827063 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1787827047.672091:0:2386:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1787827047.672093:0:2386:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1787827047.672095:0:2386:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1787827047.672099:0:2386:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1787827047.672105:0:2386:0:(import.c:2149:obd_at_measure()) add 5 to ffff9856110b0460 time=125 v=5 (5 5 5 5) 00000100:00000001:3.0:1787827047.672105:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1787827047.672108:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 00000100:00001000:2.0:1787827047.672109:0:2386:0:(import.c:2149:obd_at_measure()) add 1 to ffff9856110b0418 time=11 v=5 (1 0 0 0) 02000000:00000010:3.0:1787827047.672112:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff985611860e00. 00000100:00000001:2.0:1787827047.672113:0:2386:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1787827047.672115:0:2386:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1787827047.672117:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00010000:00000001:2.0:1787827047.672119:0:2386:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000020:00000001:3.0:1787827047.672119:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:3.0:1787827047.672123:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611185800 refcount=3 obd=lustre-MDT0001-mdc-ffff985602bc9800 00010000:00000001:2.0:1787827047.672124:0:2386:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1787827047.672125:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:3.0:1787827047.672127:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff985615779100. 02000000:00000001:3.0:1787827047.672128:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1787827047.672129:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1787827047.672130:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff9856140d9a40. 00000100:00000001:3.0:1787827047.672133:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1787827047.672134:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1787827047.672137:0:2385:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1787827047.672140:0:2386:0:(client.c:3109:ptlrpc_free_committed()) Process entered 00000100:00000040:2.0:1787827047.672143:0:2386:0:(client.c:3121:ptlrpc_free_committed()) lustre-OST0000-osc-ffff985602bc9800: skip recheck: last_committed 8589935703 00000100:00000001:3.0:1787827047.672143:0:2385:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1787827047.672147:0:2385:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1787827047.672148:0:2386:0:(client.c:3122:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1787827047.672151:0:2385:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1787827047.672152:0:2386:0:(client.c:1797:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827047.672155:0:2385:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1787827047.672160:0:2386:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9856140d8f00 x1874669228034688/t0(0) o400->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827063 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827047.672166:0:2386:0:(client.c:2354:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1787827047.672168:0:2386:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1787827047.672169:0:2386:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1787827047.672172:0:2386:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9856140d8f00 x1874669228034688/t0(0) o400->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827063 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827047.672177:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228034688:192.168.203.114@tcp:400:kworker.0 00000100:00000001:2.0:1787827047.672180:0:2386:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1787827047.672181:0:2386:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827047.672183:0:2386:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9856140d8f00 x1874669228034688/t0(0) o400->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827063 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827047.672186:0:2386:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827047.672187:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827047.672189:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff985607e33c00. 02000000:00000001:2.0:1787827047.672190:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827047.672191:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827047.672192:0:2386:0:(genops.c:1223:class_import_put()) import ffff9856110b0000 refcount=2 obd=lustre-OST0000-osc-ffff985602bc9800 00000020:00000001:2.0:1787827047.672194:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827047.672195:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff985615779d00. 02000000:00000001:2.0:1787827047.672197:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827047.672198:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827047.672198:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff9856140d8f00. 00000100:00000001:2.0:1787827047.672200:0:2386:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827047.672201:0:2386:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1787827047.672202:0:2386:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1787827047.672206:0:2386:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1787827047.672207:0:2386:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1787827047.672208:0:2386:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1787827047.672209:0:2386:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827048.246775:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827048.246782:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827048.246788:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827048.246789:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827048.246792:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827048.246796:0:49842:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:0.0:1787827048.246798:0:49842:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:0.0:1787827048.246805:0:49842:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:0.0:1787827048.246807:0:49842:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:0.0:1787827048.246809:0:49842:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827048.246811:0:49842:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:0.0:1787827048.246814:0:49842:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827048.246816:0:49842:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827048.246818:0:49842:0:(ldlm_pool.c:496:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1787827048.246881:0:15163:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1787827048.246885:0:15163:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1787827048.246887:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:0.0:1787827049.268829:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827049.268836:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827049.268841:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827049.268842:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827049.268845:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827049.268846:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1787827049.268899:0:33341:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1787827049.268903:0:33341:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1787827049.268906:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:0.0:1787827050.292970:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827050.292979:0:49842:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:0.0:1787827050.292982:0:49842:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:0.0:1787827050.292984:0:49842:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827050.292989:0:49842:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:0.0:1787827050.292991:0:49842:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827050.292992:0:49842:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827050.292994:0:49842:0:(ldlm_pool.c:496:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827050.292997:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827050.292999:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827050.293001:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827050.293002:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1787827050.293027:0:15164:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1787827050.293036:0:15164:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1787827050.293038:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:0.0:1787827051.316798:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827051.316803:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827051.316808:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827051.316809:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827051.316811:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827051.316811:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1787827051.316833:0:33342:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1787827051.316836:0:33342:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1787827051.316839:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:0.0:1787827052.340820:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827052.340826:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827052.340831:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827052.340832:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827052.340834:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827052.340835:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1787827052.340893:0:79887:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1787827052.340897:0:79887:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1787827052.340900:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1787827052.788826:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1787827052.788833:0:38:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1787827052.788842:0:38:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 904 at ffff985607f08f00. 00000020:00000040:0.0:1787827052.788846:0:38:0:(genops.c:1210:class_import_get()) import ffff98561118e000 refcount=4 obd=MGC192.168.203.114@tcp 00000100:00000001:0.0:1787827052.788849:0:38:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1787827052.788852:0:38:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1787827052.788855:0:38:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1787827052.788861:0:38:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff98561324cf00. 00000100:00000001:0.0:1787827052.788866:0:38:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787827052.788874:0:38:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS req@ffff985607f08f00 x1874669228034816/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1787827052.788881:0:38:0:(jobid.c:939:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1787827052.788884:0:38:0:(jobid.c:979:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787827052.788889:0:38:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000002a2b9b7a] to pc [ptlrpcd_00_01+1] req@ffff985607f08f00 x1874669228034816/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787827052.788909:0:38:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787827052.788911:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1787827052.788913:0:38:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1787827052.788915:0:38:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 904 at ffff985607f0a940. 00000020:00000040:0.0:1787827052.788916:0:38:0:(genops.c:1210:class_import_get()) import ffff985611182000 refcount=4 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000100:00000001:0.0:1787827052.788918:0:38:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1787827052.788918:0:38:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1787827052.788920:0:38:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1787827052.788922:0:38:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff98561324c400. 00000100:00000001:0.0:1787827052.788923:0:38:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787827052.788925:0:38:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID req@ffff985607f0a940 x1874669228034944/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1787827052.788930:0:38:0:(jobid.c:939:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1787827052.788931:0:38:0:(jobid.c:979:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787827052.788933:0:38:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000083e565bf] to pc [ptlrpcd_00_00+0] req@ffff985607f0a940 x1874669228034944/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787827052.788939:0:38:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787827052.788940:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1787827052.788943:0:38:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1787827052.788944:0:38:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 904 at ffff985607f0a1c0. 00000020:00000040:0.0:1787827052.788945:0:38:0:(genops.c:1210:class_import_get()) import ffff985611185800 refcount=4 obd=lustre-MDT0001-mdc-ffff985602bc9800 00000100:00000001:0.0:1787827052.788946:0:38:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1787827052.788947:0:38:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1787827052.788948:0:38:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1787827052.788950:0:38:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff98561324c200. 00000100:00000001:0.0:1787827052.788951:0:38:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787827052.788953:0:38:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID req@ffff985607f0a1c0 x1874669228035072/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1787827052.788957:0:38:0:(jobid.c:939:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1787827052.788959:0:38:0:(jobid.c:979:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787827052.788961:0:38:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000009dcd1e7d] to pc [ptlrpcd_00_01+1] req@ffff985607f0a1c0 x1874669228035072/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787827052.788964:0:38:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787827052.788966:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-OST0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1787827052.788967:0:38:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1787827052.788969:0:38:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 904 at ffff985607f08b40. 00000020:00000040:0.0:1787827052.788971:0:38:0:(genops.c:1210:class_import_get()) import ffff9856110b0000 refcount=3 obd=lustre-OST0000-osc-ffff985602bc9800 00000100:00000001:0.0:1787827052.788972:0:38:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1787827052.788973:0:38:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1787827052.788974:0:38:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1787827052.788976:0:38:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff98561324ca00. 00000100:00000001:0.0:1787827052.788977:0:38:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787827052.788979:0:38:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-OST0000_UUID req@ffff985607f08b40 x1874669228035200/t0(0) o400->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1787827052.788983:0:38:0:(jobid.c:939:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1787827052.788984:0:38:0:(jobid.c:979:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787827052.788986:0:38:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000008e93a775] to pc [ptlrpcd_00_00+0] req@ffff985607f08b40 x1874669228035200/t0(0) o400->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827052.788989:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1787827052.788991:0:38:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787827052.788996:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:0.0:1787827052.789000:0:38:0:(pinger.c:247:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000001:1.0:1787827052.789000:0:2384:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1787827052.789009:0:2384:0:(client.c:1813:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1787827052.789011:0:38:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (3208) 00000100:00000001:0.0:1787827052.789025:0:2383:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1787827052.789028:0:2383:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1787827052.789030:0:2383:0:(client.c:1813:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1787827052.789034:0:2384:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff985607f08f00 x1874669228034816/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1787827052.789035:0:2383:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff985607f0a940 x1874669228034944/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787827052.789070:0:2383:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1787827052.789073:0:2384:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1787827052.789074:0:2383:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827052.789078:0:2384:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1787827052.789081:0:2383:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1787827052.789084:0:2384:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1787827052.789085:0:2383:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1787827052.789090:0:2384:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1787827052.789091:0:2383:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1787827052.789095:0:2384:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1787827052.789102:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607f08f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:3c946890-15ca-497a-b838-fe1cba61f4dd:2384:1874669228034816:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787827052.789109:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607f0a940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669228034944:192.168.203.114@tcp:400:kworker.0 00000100:00000001:1.0:1787827052.789111:0:2384:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:0.0:1787827052.789118:0:2383:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1787827052.789123:0:2383:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1787827052.789125:0:2384:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1787827052.789126:0:2383:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1787827052.789131:0:2384:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1787827052.789132:0:2383:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000001:1.0:1787827052.789137:0:2384:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1787827052.789140:0:2383:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff98560540ac00. 02000000:00000001:0.0:1787827052.789144:0:2383:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1787827052.789147:0:2384:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff985610db8c00. 00000400:00000200:0.0:1787827052.789153:0:2383:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 02000000:00000001:1.0:1787827052.789153:0:2384:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1787827052.789163:0:2383:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9856094ac370. 00000400:00000200:1.0:1787827052.789166:0:2384:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000010:0.0:1787827052.789174:0:2383:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985610d30b28. 00000400:00000010:1.0:1787827052.789174:0:2384:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff985602ecf898. 00000400:00000010:1.0:1787827052.789185:0:2384:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985606618cc0. 00000100:00000200:0.0:1787827052.789190:0:2383:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1874669228034944, portal 10 00000100:00000200:1.0:1787827052.789193:0:2384:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1874669228034816, portal 25 00000100:00000001:0.0:1787827052.789195:0:2383:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1787827052.789198:0:2384:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1787827052.789200:0:2383:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446630094000662848 : -113979708888768 : ffff985607f0a940) 00000100:00000001:1.0:1787827052.789202:0:2384:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446630094000656128 : -113979708895488 : ffff985607f08f00) 00000100:00000040:0.0:1787827052.789216:0:2383:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff985607f0a940 x1874669228034944/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827068 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1787827052.789217:0:2384:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff985607f08f00 x1874669228034816/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827068 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787827052.789252:0:2383:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1787827052.789260:0:2383:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.203.114@tcp 00000400:00000010:0.0:1787827052.789267:0:2383:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985610d30660. 00000100:00000200:0.0:1787827052.789272:0:2383:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1874669228034944, offset 0 00000100:00000001:1.0:1787827052.789282:0:2384:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000400:00000200:0.0:1787827052.789283:0:2383:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.203.114@tcp 00000100:00000040:1.0:1787827052.789290:0:2384:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.203.114@tcp 00000400:00000010:1.0:1787827052.789295:0:2384:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985606618110. 00000400:00000200:0.0:1787827052.789296:0:2383:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000200:1.0:1787827052.789303:0:2384:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1874669228034816, offset 0 00000400:00000200:0.0:1787827052.789311:0:2383:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.203.114@tcp local destination 00000400:00000200:0.0:1787827052.789319:0:2383:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1787827052.789320:0:2384:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.203.114@tcp 00000400:00000200:0.0:1787827052.789329:0:2383:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.203.14@tcp 00000400:00000200:0.0:1787827052.789333:0:2383:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.203.14@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1787827052.789336:0:2383:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.203.114@tcp 00000400:00000200:0.0:1787827052.789339:0:2383:0:(lib-move.c:1745:lnet_handle_send()) 192.168.203.14@tcp NI seq info: [21910:21910:256:4294967295] 192.168.203.114@tcp LPNI seq info [21910:21910:8:4294967295] 00000400:00000200:0.0:1787827052.789349:0:2383:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:0.0:1787827052.789361:0:2383:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp:) -> 192.168.203.114@tcp(192.168.203.114@tcp:192.168.203.114@tcp) : PUT try# 0 00000400:00000200:1.0:1787827052.789369:0:2384:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000800:00000200:0.0:1787827052.789371:0:2383:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.203.114@tcp 00000800:00000010:0.0:1787827052.789380:0:2383:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff98561324c000. 00000400:00000200:1.0:1787827052.789382:0:2384:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.203.114@tcp local destination 00000400:00000200:1.0:1787827052.789389:0:2384:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000800:00000200:0.0:1787827052.789393:0:2383:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff985606751480] -> 12345-192.168.203.114@tcp (5) 00000400:00000200:1.0:1787827052.789398:0:2384:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.203.14@tcp 00000400:00000200:1.0:1787827052.789404:0:2384:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.203.14@tcp lpni_is_preferred = 0 00000800:00000200:0.0:1787827052.789411:0:2383:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.203.114@tcp ip 192.168.203.114:988 00000400:00000200:1.0:1787827052.789413:0:2384:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.203.114@tcp 00000800:00000200:0.0:1787827052.789422:0:2383:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff98561324c000 type 1, nob 320 niov 1 nkiov 1 00000400:00000200:1.0:1787827052.789424:0:2384:0:(lib-move.c:1745:lnet_handle_send()) 192.168.203.14@tcp NI seq info: [21911:21911:255:4294967295] 192.168.203.114@tcp LPNI seq info [21911:21911:7:4294967295] 00000400:00000200:1.0:1787827052.789439:0:2384:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000001:0.0:1787827052.789448:0:2383:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1787827052.789451:0:2384:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp:) -> 192.168.203.114@tcp(192.168.203.114@tcp:192.168.203.114@tcp) : PUT try# 0 00000100:00000040:0.0:1787827052.789456:0:2383:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff985602bc9800: using a timeout of 38043 usecs (39 jiffies) 00000800:00000200:1.0:1787827052.789461:0:2384:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.203.114@tcp 00000400:00000200:0.0:1787827052.789469:0:2383:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000800:00000010:1.0:1787827052.789473:0:2384:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff985607d95100. 00000100:00000001:0.0:1787827052.789482:0:2383:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff985602bc9800: PM QoS new deadline estimation for cpu 2 is 4297870887 00000800:00000200:1.0:1787827052.789486:0:2384:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff985606751480] -> 12345-192.168.203.114@tcp (5) 00000100:00000001:0.0:1787827052.789488:0:2383:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff985602bc9800: PM QoS request active for cpu 2, keep current deadline 4297875708 00000100:00000001:0.0:1787827052.789494:0:2383:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff985602bc9800: PM QoS new deadline estimation for cpu 3 is 4297870887 00000100:00000001:0.0:1787827052.789499:0:2383:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff985602bc9800: PM QoS request active for cpu 3, keep current deadline 4297875708 00000800:00000200:1.0:1787827052.789499:0:2384:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.203.114@tcp ip 192.168.203.114:988 00000800:00000200:1.0:1787827052.789507:0:2384:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff985607d95100 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1787827052.789508:0:2383:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1787827052.789515:0:2383:0:(client.c:1941:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827052.789518:0:2384:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787827052.789521:0:2383:0:(client.c:1813:ptlrpc_send_new_req()) Process entered 00000400:00000200:1.0:1787827052.789526:0:2384:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000040:0.0:1787827052.789533:0:2383:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff985607f08b40 x1874669228035200/t0(0) o400->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827052.789533:0:2384:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.203.114@tcp: PM QoS new deadline estimation for cpu 2 is 4297880828 00000100:00000001:0.0:1787827052.789547:0:2383:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1787827052.789549:0:2383:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1787827052.789553:0:2383:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:1.0:1787827052.789555:0:2384:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.203.114@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4297875708 02000000:00000001:0.0:1787827052.789556:0:2383:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1787827052.789564:0:2383:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827052.789569:0:2384:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.203.114@tcp: PM QoS new deadline estimation for cpu 3 is 4297880828 00000100:00100000:0.0:1787827052.789570:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607f08b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669228035200:192.168.203.114@tcp:400:kworker.0 00000800:00000200:3.0:1787827052.789572:0:2377:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:1.0:1787827052.789577:0:2384:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.203.114@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4297875708 00000800:00000010:3.0:1787827052.789579:0:2377:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff98561324c000. 00000100:00000001:0.0:1787827052.789591:0:2383:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:1.0:1787827052.789591:0:2384:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000200:3.0:1787827052.789595:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 02000000:00000001:0.0:1787827052.789598:0:2383:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:1.0:1787827052.789604:0:2384:0:(client.c:1941:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1787827052.789606:0:2383:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827052.789614:0:2384:0:(client.c:1813:ptlrpc_send_new_req()) Process entered 02000000:00000001:0.0:1787827052.789615:0:2383:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000200:3.0:1787827052.789615:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000100:00000040:1.0:1787827052.789633:0:2384:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff985607f0a1c0 x1874669228035072/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 02000000:00000010:0.0:1787827052.789635:0:2383:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff98560540bc00. 00000400:00000200:3.0:1787827052.789637:0:2377:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985610d30660 02000000:00000001:0.0:1787827052.789644:0:2383:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1787827052.789645:0:2377:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985610d30660. 00000400:00000200:0.0:1787827052.789663:0:2383:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000001:1.0:1787827052.789665:0:2384:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000400:00000010:0.0:1787827052.789670:0:2383:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9856094acf20. 00000100:00000001:1.0:1787827052.789670:0:2384:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1787827052.789676:0:2384:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 00000400:00000010:0.0:1787827052.789680:0:2383:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985610d306e8. 02000000:00000001:1.0:1787827052.789681:0:2384:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1787827052.789689:0:2384:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1787827052.789734:0:2383:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1874669228035200, portal 4 00000100:00100000:1.0:1787827052.789735:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607f0a1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669228035072:192.168.203.114@tcp:400:kworker.0 00000100:00000001:0.0:1787827052.789743:0:2383:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1787827052.789746:0:2377:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:0.0:1787827052.789747:0:2383:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446630094000655168 : -113979708896448 : ffff985607f08b40) 00000100:00000001:1.0:1787827052.789747:0:2384:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1787827052.789754:0:2384:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1787827052.789758:0:2384:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1787827052.789763:0:2384:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000040:0.0:1787827052.789764:0:2383:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff985607f08b40 x1874669228035200/t0(0) o400->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827068 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1787827052.789764:0:2377:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff985607f0a940 x1874669228034944/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827068 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 02000000:00000010:1.0:1787827052.789769:0:2384:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff985610db9a00. 02000000:00000001:1.0:1787827052.789777:0:2384:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1787827052.789797:0:2384:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000001:0.0:1787827052.789807:0:2383:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000400:00000010:1.0:1787827052.789807:0:2384:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff985602ecf9f8. 00000100:00000001:3.0:1787827052.789810:0:2377:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1787827052.789815:0:2383:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.203.114@tcp 00000400:00000010:1.0:1787827052.789818:0:2384:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985606618d48. 00000400:00000010:0.0:1787827052.789821:0:2383:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985610d30770. 00000100:00000040:3.0:1787827052.789821:0:2377:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff985607f0a940 x1874669228034944/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827068 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1787827052.789825:0:2384:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1874669228035072, portal 10 00000100:00000200:0.0:1787827052.789829:0:2383:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 28, xid 1874669228035200, offset 0 00000100:00000001:1.0:1787827052.789834:0:2384:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000400:00000200:0.0:1787827052.789840:0:2383:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.203.114@tcp 00000100:00000001:3.0:1787827052.789842:0:2377:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827052.789845:0:2377:0:(events.c:68:request_out_callback()) Process leaving 00000400:00000200:0.0:1787827052.789848:0:2383:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000001:1.0:1787827052.789850:0:2384:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446630094000660928 : -113979708890688 : ffff985607f0a1c0) 00000400:00000200:0.0:1787827052.789854:0:2383:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.203.114@tcp local destination 00000400:00000200:0.0:1787827052.789861:0:2383:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1787827052.789866:0:2383:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.203.14@tcp 00000100:00000040:1.0:1787827052.789866:0:2384:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff985607f0a1c0 x1874669228035072/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827068 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000400:00000200:0.0:1787827052.789872:0:2383:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.203.14@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1787827052.789879:0:2383:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.203.114@tcp 00000400:00000200:0.0:1787827052.789888:0:2383:0:(lib-move.c:1745:lnet_handle_send()) 192.168.203.14@tcp NI seq info: [21912:21912:255:4294967295] 192.168.203.114@tcp LPNI seq info [21912:21912:7:4294967295] 00000800:00000200:3.0:1787827052.789892:0:2377:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1787827052.789901:0:2377:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff985607d95100. 00000400:00000200:0.0:1787827052.789909:0:2383:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:3.0:1787827052.789912:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:1.0:1787827052.789920:0:2384:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000400:00000200:3.0:1787827052.789930:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000100:00000040:1.0:1787827052.789933:0:2384:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.203.114@tcp 00000400:00000200:0.0:1787827052.789940:0:2383:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp:) -> 192.168.203.114@tcp(192.168.203.114@tcp:192.168.203.114@tcp) : PUT try# 0 00000400:00000010:1.0:1787827052.789940:0:2384:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985606618dd0. 00000100:00000200:1.0:1787827052.789949:0:2384:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1874669228035072, offset 0 00000400:00000200:3.0:1787827052.789953:0:2377:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985606618110 00000800:00000200:0.0:1787827052.789955:0:2383:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.203.114@tcp 00000400:00000010:3.0:1787827052.789958:0:2377:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985606618110. 00000800:00000010:0.0:1787827052.789962:0:2383:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff98561324cb00. 00000100:00000001:3.0:1787827052.789964:0:2377:0:(events.c:34:request_out_callback()) Process entered 00000400:00000200:1.0:1787827052.789971:0:2384:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.203.114@tcp 00000800:00000200:0.0:1787827052.789974:0:2383:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff985606751480] -> 12345-192.168.203.114@tcp (5) 00000100:00000200:3.0:1787827052.789976:0:2377:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff985607f08f00 x1874669228034816/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827068 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1787827052.789989:0:2384:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000800:00000200:0.0:1787827052.789990:0:2383:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.203.114@tcp ip 192.168.203.114:988 00000800:00000200:0.0:1787827052.789999:0:2383:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff98561324cb00 type 1, nob 320 niov 1 nkiov 1 00000400:00000200:1.0:1787827052.790002:0:2384:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.203.114@tcp local destination 00000100:00000001:0.0:1787827052.790005:0:2383:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827052.790008:0:2377:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1787827052.790015:0:2383:0:(niobuf.c:635:kick_cpu_latency()) lustre-OST0000-osc-ffff985602bc9800: using a timeout of 14625 usecs (15 jiffies) 00000400:00000200:1.0:1787827052.790016:0:2384:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1787827052.790030:0:2384:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.203.14@tcp 00000400:00000200:0.0:1787827052.790031:0:2383:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000001:0.0:1787827052.790040:0:2383:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff985602bc9800: PM QoS new deadline estimation for cpu 2 is 4297870864 00000400:00000200:1.0:1787827052.790044:0:2384:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.203.14@tcp lpni_is_preferred = 0 00000100:00000001:0.0:1787827052.790045:0:2383:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff985602bc9800: PM QoS request active for cpu 2, keep current deadline 4297880828 00000100:00000001:0.0:1787827052.790052:0:2383:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff985602bc9800: PM QoS new deadline estimation for cpu 3 is 4297870864 00000100:00000040:3.0:1787827052.790053:0:2377:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff985607f08f00 x1874669228034816/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827068 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787827052.790057:0:2383:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff985602bc9800: PM QoS request active for cpu 3, keep current deadline 4297880828 00000400:00000200:1.0:1787827052.790057:0:2384:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.203.114@tcp 00000100:00000001:0.0:1787827052.790066:0:2383:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1787827052.790068:0:2377:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1787827052.790073:0:2384:0:(lib-move.c:1745:lnet_handle_send()) 192.168.203.14@tcp NI seq info: [21913:21913:254:4294967295] 192.168.203.114@tcp LPNI seq info [21913:21913:6:4294967295] 00000100:00000001:3.0:1787827052.790074:0:2377:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:0.0:1787827052.790075:0:2383:0:(client.c:1941:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787827052.790085:0:2383:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1787827052.790090:0:2384:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000001:0.0:1787827052.790092:0:2383:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1787827052.790102:0:2383:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1787827052.790105:0:2383:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1787827052.790114:0:2383:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:1.0:1787827052.790116:0:2384:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp:) -> 192.168.203.114@tcp(192.168.203.114@tcp:192.168.203.114@tcp) : PUT try# 0 00000100:00000001:0.0:1787827052.790117:0:2383:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000800:00000200:3.0:1787827052.790120:0:2377:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:0.0:1787827052.790122:0:2383:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:3.0:1787827052.790126:0:2377:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff98561324cb00. 00000100:00000001:0.0:1787827052.790127:0:2383:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1787827052.790129:0:2384:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.203.114@tcp 00000400:00000200:3.0:1787827052.790135:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000800:00000010:1.0:1787827052.790137:0:2384:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff985607d95800. 00000400:00000200:3.0:1787827052.790144:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000800:00000200:1.0:1787827052.790149:0:2384:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff985606751480] -> 12345-192.168.203.114@tcp (5) 00000400:00000200:3.0:1787827052.790153:0:2377:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985610d30770 00000800:00000200:1.0:1787827052.790159:0:2384:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.203.114@tcp ip 192.168.203.114:988 00000400:00000010:3.0:1787827052.790159:0:2377:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985610d30770. 00000100:00000001:3.0:1787827052.790167:0:2377:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1787827052.790174:0:2377:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff985607f08b40 x1874669228035200/t0(0) o400->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827068 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000800:00000200:1.0:1787827052.790180:0:2384:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff985607d95800 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1787827052.790184:0:2377:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000001:1.0:1787827052.790187:0:2384:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1787827052.790193:0:2384:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff985602bc9800: using a timeout of 11916 usecs (12 jiffies) 00000400:00000200:1.0:1787827052.790197:0:2384:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000040:3.0:1787827052.790200:0:2377:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff985607f08b40 x1874669228035200/t0(0) o400->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827068 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827052.790201:0:2384:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff985602bc9800: PM QoS new deadline estimation for cpu 2 is 4297870861 00000100:00000001:1.0:1787827052.790211:0:2384:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff985602bc9800: PM QoS request active for cpu 2, keep current deadline 4297880828 00000100:00000001:3.0:1787827052.790216:0:2377:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827052.790219:0:2384:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff985602bc9800: PM QoS new deadline estimation for cpu 3 is 4297870861 00000100:00000001:3.0:1787827052.790219:0:2377:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:1.0:1787827052.790225:0:2384:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff985602bc9800: PM QoS request active for cpu 3, keep current deadline 4297880828 00000100:00000001:1.0:1787827052.790228:0:2384:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1787827052.790231:0:2384:0:(client.c:1941:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1787827052.790232:0:2377:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:1.0:1787827052.790236:0:2384:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:3.0:1787827052.790236:0:2377:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff985607d95800. 00000400:00000200:3.0:1787827052.790245:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:1.0:1787827052.790246:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1787827052.790254:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000400:00000200:3.0:1787827052.790256:0:2377:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985606618dd0 00000100:00000001:1.0:1787827052.790257:0:2384:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000400:00000010:3.0:1787827052.790259:0:2377:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985606618dd0. 00000100:00000001:1.0:1787827052.790263:0:2384:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1787827052.790267:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1787827052.790270:0:2384:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1787827052.790271:0:2377:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:1.0:1787827052.790273:0:2384:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827052.790278:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:3.0:1787827052.790284:0:2377:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff985607f0a1c0 x1874669228035072/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827068 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827052.790294:0:2377:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1787827052.790298:0:2377:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff985607f0a1c0 x1874669228035072/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827068 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827052.790306:0:2377:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827052.790309:0:2377:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1787827052.793060:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827052.793074:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1787827052.793080:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827052.793082:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827052.793091:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827052.793098:0:2378:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp) <- 192.168.203.114@tcp : PUT - for me 00000400:00000200:2.0:1787827052.793105:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827052.793110:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827052.793115:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827052.793119:0:2378:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.203.114@tcp of length 224 into portal 25 MB=0x6a90071688b00 00000400:00000200:2.0:1787827052.793124:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827052.793129:0:2378:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.203.114@tcp of length 224/224 into md 0x323ed [1] + 224 00000800:00000001:2.0:1787827052.793137:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827052.793141:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827052.793144:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1787827052.793149:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000100:00000001:2.0:1787827052.793155:0:2378:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1787827052.793175:0:2378:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff985607f08f00 x1874669228034816/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827068 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1787827052.793188:0:2378:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff985607f08f00 x1874669228034816/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827068 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827052.793206:0:2378:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1787827052.793215:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827052.793219:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1787827052.793221:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827052.793223:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827052.793226:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827052.793230:0:2378:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp) <- 192.168.203.114@tcp : PUT - for me 00000400:00000200:2.0:1787827052.793234:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827052.793237:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827052.793240:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827052.793243:0:2378:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.203.114@tcp of length 224 into portal 10 MB=0x6a90071688b80 00000400:00000200:2.0:1787827052.793246:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827052.793251:0:2378:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.203.114@tcp of length 224/224 into md 0x323e5 [1] + 224 00000800:00000001:2.0:1787827052.793254:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827052.793258:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827052.793260:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1787827052.793263:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000100:00000001:2.0:1787827052.793266:0:2378:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1787827052.793270:0:2378:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff985607f0a940 x1874669228034944/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827068 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1787827052.793279:0:2378:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff985607f0a940 x1874669228034944/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827068 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827052.793299:0:2378:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1787827052.793303:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827052.793307:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1787827052.793308:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827052.793311:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827052.793314:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827052.793317:0:2378:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp) <- 192.168.203.114@tcp : PUT - for me 00000400:00000200:2.0:1787827052.793321:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827052.793323:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827052.793325:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827052.793327:0:2378:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.203.114@tcp of length 224 into portal 10 MB=0x6a90071688c00 00000400:00000200:2.0:1787827052.793330:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827052.793334:0:2378:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.203.114@tcp of length 224/224 into md 0x323fd [1] + 224 00000800:00000001:2.0:1787827052.793337:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827052.793340:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827052.793341:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1787827052.793350:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000100:00000001:2.0:1787827052.793353:0:2378:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1787827052.793358:0:2378:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff985607f0a1c0 x1874669228035072/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827068 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787827052.793595:0:2383:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1787827052.793600:0:2383:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1787827052.793607:0:2383:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1787827052.793608:0:2383:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1787827052.793611:0:2383:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9856094ac370. 00000400:00000200:0.0:1787827052.793615:0:2383:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985610d30b28 00000400:00000010:0.0:1787827052.793618:0:2383:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985610d30b28. 00000100:00000001:0.0:1787827052.793632:0:2383:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1787827052.793638:0:2383:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff985607f0a940 x1874669228034944/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827068 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1787827052.793648:0:2383:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff985607f0a940 x1874669228034944/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827068 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827052.793651:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1787827052.793655:0:2384:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1787827052.793657:0:2383:0:(events.c:163:reply_in_callback()) Process leaving 00000400:00000001:1.0:1787827052.793660:0:2384:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1787827052.793664:0:2384:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000001:0.0:1787827052.793668:0:2383:0:(client.c:3001:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1787827052.793672:0:2383:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:0.0:1787827052.793677:0:2383:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000400:00000010:1.0:1787827052.793677:0:2384:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff985602ecf898. 00000100:00000001:0.0:1787827052.793681:0:2383:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000400:00000200:1.0:1787827052.793683:0:2384:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985606618cc0 00000100:00000001:0.0:1787827052.793684:0:2383:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1787827052.793688:0:2384:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985606618cc0. 02000000:00000001:0.0:1787827052.793714:0:2383:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827052.793722:0:2384:0:(events.c:78:reply_in_callback()) Process entered 00000100:00001000:0.0:1787827052.793724:0:2383:0:(import.c:2149:obd_at_measure()) add 5 to ffff985611182460 time=130 v=5 (5 5 5 5) 00000100:00000200:1.0:1787827052.793729:0:2384:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff985607f08f00 x1874669228034816/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827068 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00001000:0.0:1787827052.793731:0:2383:0:(import.c:2149:obd_at_measure()) add 1 to ffff985611182418 time=66 v=5 (1 0 0 0) 00000100:00000001:0.0:1787827052.793738:0:2383:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1787827052.793741:0:2383:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827052.793747:0:2383:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000200:1.0:1787827052.793749:0:2384:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff985607f08f00 x1874669228034816/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827068 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00000001:0.0:1787827052.793752:0:2383:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787827052.793758:0:2383:0:(client.c:3109:ptlrpc_free_committed()) Process entered 00000100:00000001:1.0:1787827052.793761:0:2384:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000040:0.0:1787827052.793762:0:2383:0:(client.c:3121:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: skip recheck: last_committed 219043334508 00000100:00000001:1.0:1787827052.793764:0:2384:0:(client.c:3001:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1787827052.793767:0:2383:0:(client.c:3122:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1787827052.793769:0:2383:0:(client.c:1797:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827052.793769:0:2384:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:1.0:1787827052.793772:0:2384:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1787827052.793775:0:2384:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000040:0.0:1787827052.793776:0:2383:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff985607f0a940 x1874669228034944/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827068 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827052.793778:0:2384:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1787827052.793783:0:2384:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787827052.793788:0:2383:0:(client.c:2354:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00001000:1.0:1787827052.793790:0:2384:0:(import.c:2149:obd_at_measure()) add 5 to ffff98561118e460 time=52 v=5 (5 5 5 5) 00000100:00000001:0.0:1787827052.793791:0:2383:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1787827052.793793:0:2383:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:1.0:1787827052.793798:0:2384:0:(import.c:2149:obd_at_measure()) add 1 to ffff98561118e418 time=67 v=5 (1 0 0 0) 00000100:00000040:0.0:1787827052.793799:0:2383:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff985607f0a940 x1874669228034944/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827068 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827052.793805:0:2384:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1787827052.793807:0:2384:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1787827052.793812:0:2384:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1787827052.793815:0:2384:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1787827052.793816:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607f0a940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669228034944:192.168.203.114@tcp:400:kworker.0 00000100:00000001:1.0:1787827052.793818:0:2384:0:(client.c:1797:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787827052.793825:0:2383:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1787827052.793827:0:2384:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff985607f08f00 x1874669228034816/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827068 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787827052.793830:0:2383:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1787827052.793837:0:2383:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607f0a940 x1874669228034944/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827068 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787827052.793844:0:2383:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1787827052.793846:0:2383:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1787827052.793851:0:2383:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff98560540ac00. 00000100:00000001:1.0:1787827052.793851:0:2384:0:(client.c:2354:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1787827052.793856:0:2384:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 02000000:00000001:0.0:1787827052.793858:0:2383:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000001:1.0:1787827052.793859:0:2384:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000020:00000001:0.0:1787827052.793861:0:2383:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:0.0:1787827052.793864:0:2383:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=3 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:0.0:1787827052.793867:0:2383:0:(genops.c:1230:class_import_put()) Process leaving 00000100:00000040:1.0:1787827052.793868:0:2384:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff985607f08f00 x1874669228034816/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827068 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 02000000:00000010:0.0:1787827052.793869:0:2383:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff98561324c400. 02000000:00000001:0.0:1787827052.793874:0:2383:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1787827052.793876:0:2383:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1787827052.793878:0:2383:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607f0a940. 00000100:00000001:0.0:1787827052.793883:0:2383:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1787827052.793885:0:2383:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00100000:1.0:1787827052.793887:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607f08f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:3c946890-15ca-497a-b838-fe1cba61f4dd:2384:1874669228034816:192.168.203.114@tcp:400:kworker.0 00000100:00000001:0.0:1787827052.793888:0:2383:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787827052.793890:0:2383:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1787827052.793892:0:2383:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1787827052.793897:0:2384:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787827052.793900:0:2383:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827052.793901:0:2384:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1787827052.793903:0:2383:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1787827052.793908:0:2384:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607f08f00 x1874669228034816/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827068 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827052.793915:0:2384:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1787827052.793918:0:2384:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1787827052.793920:0:2384:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff985610db8c00. 02000000:00000001:1.0:1787827052.793923:0:2384:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1787827052.793925:0:2384:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:1.0:1787827052.793926:0:2384:0:(genops.c:1223:class_import_put()) import ffff98561118e000 refcount=3 obd=MGC192.168.203.114@tcp 00000020:00000001:1.0:1787827052.793928:0:2384:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:1.0:1787827052.793930:0:2384:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff98561324cf00. 02000000:00000001:1.0:1787827052.793932:0:2384:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1787827052.793934:0:2384:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1787827052.793935:0:2384:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607f08f00. 00000100:00000001:1.0:1787827052.793938:0:2384:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1787827052.793939:0:2384:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787827052.793941:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827052.793943:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1787827052.793944:0:2384:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1787827052.793945:0:2384:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827052.793947:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1787827052.794084:0:2378:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff985607f0a1c0 x1874669228035072/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827068 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827052.794099:0:2378:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1787827052.794106:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827052.794111:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1787827052.794113:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827052.794115:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827052.794119:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827052.794123:0:2378:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp) <- 192.168.203.114@tcp : PUT - for me 00000400:00000200:2.0:1787827052.794128:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827052.794131:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827052.794134:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827052.794137:0:2378:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.203.114@tcp of length 224 into portal 4 MB=0x6a90071688c80 00000400:00000200:2.0:1787827052.794141:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827052.794145:0:2378:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 4 from 12345-192.168.203.114@tcp of length 224/224 into md 0x323f5 [1] + 224 00000800:00000001:2.0:1787827052.794150:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827052.794156:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827052.794158:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1787827052.794170:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000100:00000001:2.0:1787827052.794173:0:2378:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1787827052.794177:0:2378:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff985607f08b40 x1874669228035200/t0(0) o400->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827068 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1787827052.794186:0:2378:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff985607f08b40 x1874669228035200/t0(0) o400->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827068 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827052.794203:0:2378:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1787827052.794208:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827052.794211:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787827052.794258:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1787827052.794261:0:2384:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1787827052.794264:0:2384:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1787827052.794266:0:2384:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000001:0.0:1787827052.794267:0:2383:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000010:1.0:1787827052.794268:0:2384:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff985602ecf9f8. 00000100:00000001:0.0:1787827052.794270:0:2383:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000400:00000200:1.0:1787827052.794273:0:2384:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985606618d48 00000400:00000010:1.0:1787827052.794276:0:2384:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985606618d48. 00000100:00000001:1.0:1787827052.794278:0:2384:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000001:0.0:1787827052.794282:0:2383:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000200:1.0:1787827052.794283:0:2384:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff985607f0a1c0 x1874669228035072/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827068 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000400:00000001:0.0:1787827052.794286:0:2383:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1787827052.794290:0:2383:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9856094acf20. 00000400:00000200:0.0:1787827052.794296:0:2383:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985610d306e8 00000100:00000200:1.0:1787827052.794297:0:2384:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff985607f0a1c0 x1874669228035072/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827068 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000400:00000010:0.0:1787827052.794300:0:2383:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985610d306e8. 00000100:00000001:0.0:1787827052.794305:0:2383:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:1.0:1787827052.794310:0:2384:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000200:0.0:1787827052.794314:0:2383:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff985607f08b40 x1874669228035200/t0(0) o400->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827068 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827052.794314:0:2384:0:(client.c:3001:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787827052.794317:0:2384:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:1.0:1787827052.794320:0:2384:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1787827052.794323:0:2384:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1787827052.794325:0:2384:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1787827052.794330:0:2384:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1787827052.794331:0:2383:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff985607f08b40 x1874669228035200/t0(0) o400->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827068 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00001000:1.0:1787827052.794338:0:2384:0:(import.c:2149:obd_at_measure()) add 5 to ffff985611185c60 time=52 v=5 (5 5 5 5) 00000100:00000001:0.0:1787827052.794339:0:2383:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1787827052.794342:0:2383:0:(client.c:3001:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1787827052.794346:0:2383:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:0.0:1787827052.794347:0:2383:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1787827052.794349:0:2383:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1787827052.794350:0:2383:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1787827052.794352:0:2383:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1787827052.794356:0:2384:0:(import.c:2149:obd_at_measure()) add 1 to ffff985611185c18 time=52 v=5 (1 1 1 1) 00000100:00001000:0.0:1787827052.794358:0:2383:0:(import.c:2149:obd_at_measure()) add 5 to ffff9856110b0460 time=130 v=5 (5 5 5 5) 00000100:00001000:0.0:1787827052.794363:0:2383:0:(import.c:2149:obd_at_measure()) add 1 to ffff9856110b0418 time=16 v=5 (1 0 0 0) 00000100:00000001:0.0:1787827052.794367:0:2383:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1787827052.794368:0:2383:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827052.794370:0:2383:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1787827052.794373:0:2383:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787827052.794375:0:2383:0:(client.c:3109:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1787827052.794376:0:2383:0:(client.c:3121:ptlrpc_free_committed()) lustre-OST0000-osc-ffff985602bc9800: skip recheck: last_committed 8589935703 00000100:00000001:0.0:1787827052.794378:0:2383:0:(client.c:3122:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1787827052.794379:0:2383:0:(client.c:1797:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787827052.794384:0:2383:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff985607f08b40 x1874669228035200/t0(0) o400->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827068 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787827052.794390:0:2383:0:(client.c:2354:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1787827052.794392:0:2383:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1787827052.794393:0:2383:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1787827052.794397:0:2383:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff985607f08b40 x1874669228035200/t0(0) o400->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827068 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827052.794403:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607f08b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669228035200:192.168.203.114@tcp:400:kworker.0 00000100:00000001:0.0:1787827052.794407:0:2383:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1787827052.794409:0:2383:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1787827052.794412:0:2383:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607f08b40 x1874669228035200/t0(0) o400->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827068 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787827052.794419:0:2383:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1787827052.794420:0:2383:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1787827052.794422:0:2383:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff98560540bc00. 02000000:00000001:0.0:1787827052.794426:0:2383:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1787827052.794428:0:2383:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:0.0:1787827052.794429:0:2383:0:(genops.c:1223:class_import_put()) import ffff9856110b0000 refcount=2 obd=lustre-OST0000-osc-ffff985602bc9800 00000020:00000001:0.0:1787827052.794432:0:2383:0:(genops.c:1230:class_import_put()) Process leaving 00000100:00000001:1.0:1787827052.794433:0:2384:0:(client.c:1495:ptlrpc_check_status()) Process entered 02000000:00000010:0.0:1787827052.794434:0:2383:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff98561324ca00. 00000100:00000001:1.0:1787827052.794436:0:2384:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1787827052.794438:0:2383:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 00010000:00000001:1.0:1787827052.794440:0:2384:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 02000000:00000001:0.0:1787827052.794441:0:2383:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1787827052.794445:0:2383:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607f08b40. 00010000:00000001:1.0:1787827052.794445:0:2384:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787827052.794451:0:2383:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1787827052.794451:0:2384:0:(client.c:3109:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1787827052.794454:0:2384:0:(client.c:3121:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff985602bc9800: skip recheck: last_committed 0 00000100:00000001:1.0:1787827052.794457:0:2384:0:(client.c:3122:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1787827052.794458:0:2384:0:(client.c:1797:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787827052.794462:0:2383:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1787827052.794465:0:2384:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff985607f0a1c0 x1874669228035072/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827068 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787827052.794469:0:2383:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787827052.794476:0:2384:0:(client.c:2354:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1787827052.794478:0:2384:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1787827052.794479:0:2383:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1787827052.794480:0:2384:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1787827052.794483:0:2383:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1787827052.794486:0:2384:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff985607f0a1c0 x1874669228035072/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827068 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787827052.794488:0:2383:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1787827052.794490:0:2383:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1787827052.794499:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607f0a1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669228035072:192.168.203.114@tcp:400:kworker.0 00000100:00000001:1.0:1787827052.794504:0:2384:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787827052.794506:0:2384:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1787827052.794510:0:2384:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607f0a1c0 x1874669228035072/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827068 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827052.794516:0:2384:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1787827052.794517:0:2384:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1787827052.794520:0:2384:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff985610db9a00. 02000000:00000001:1.0:1787827052.794523:0:2384:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1787827052.794524:0:2384:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:1.0:1787827052.794525:0:2384:0:(genops.c:1223:class_import_put()) import ffff985611185800 refcount=3 obd=lustre-MDT0001-mdc-ffff985602bc9800 00000020:00000001:1.0:1787827052.794527:0:2384:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:1.0:1787827052.794529:0:2384:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff98561324c200. 02000000:00000001:1.0:1787827052.794531:0:2384:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1787827052.794532:0:2384:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1787827052.794534:0:2384:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607f0a1c0. 00000100:00000001:1.0:1787827052.794537:0:2384:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1787827052.794538:0:2384:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787827052.794539:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787827052.794543:0:2384:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1787827052.794544:0:2384:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827052.794546:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1787827052.794548:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827053.364810:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827053.364816:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827053.364822:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827053.364824:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827053.364827:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827053.364828:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1787827053.364879:0:15163:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1787827053.364883:0:15163:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1787827053.364885:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:0.0:1787827054.388810:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827054.388817:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827054.388823:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827054.388824:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827054.388827:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827054.388828:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1787827054.388934:0:33341:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1787827054.388938:0:33341:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1787827054.388941:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:0.0:1787827055.412806:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827055.412813:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827055.412817:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827055.412818:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827055.412821:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827055.412822:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1787827055.412852:0:15164:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1787827055.412855:0:15164:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1787827055.412858:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:0.0:1787827056.436811:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827056.436817:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827056.436823:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827056.436826:0:49842:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:0.0:1787827056.436828:0:49842:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:0.0:1787827056.436831:0:49842:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827056.436834:0:49842:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:0.0:1787827056.436836:0:49842:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827056.436838:0:49842:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827056.436841:0:49842:0:(ldlm_pool.c:496:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827056.436844:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827056.436846:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1787827056.436867:0:33342:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1787827056.436870:0:33342:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1787827056.436873:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:0.0:1787827057.460808:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827057.460815:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827057.460820:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827057.460821:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827057.460824:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827057.460825:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1787827057.460876:0:79887:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1787827057.460880:0:79887:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1787827057.460883:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1787827057.908814:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1787827057.908825:0:38:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1787827057.908835:0:38:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 904 at ffff985607f08b40. 00000020:00000040:0.0:1787827057.908841:0:38:0:(genops.c:1210:class_import_get()) import ffff98561118e000 refcount=4 obd=MGC192.168.203.114@tcp 00000100:00000001:0.0:1787827057.908846:0:38:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1787827057.908849:0:38:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1787827057.908853:0:38:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1787827057.908861:0:38:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff98561324ca00. 00000100:00000001:0.0:1787827057.908869:0:38:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787827057.908881:0:38:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS req@ffff985607f08b40 x1874669228035328/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1787827057.908898:0:38:0:(jobid.c:939:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1787827057.908902:0:38:0:(jobid.c:979:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787827057.908909:0:38:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000008e93a775] to pc [ptlrpcd_00_01+1] req@ffff985607f08b40 x1874669228035328/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787827057.908942:0:38:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787827057.908945:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1787827057.908948:0:38:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1787827057.908952:0:38:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 904 at ffff985607f0a940. 00000020:00000040:0.0:1787827057.908954:0:38:0:(genops.c:1210:class_import_get()) import ffff985611182000 refcount=4 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000100:00000001:0.0:1787827057.908956:0:38:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1787827057.908958:0:38:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1787827057.908959:0:38:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1787827057.908972:0:38:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff98561324c400. 00000100:00000001:1.0:1787827057.908979:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1787827057.908980:0:38:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827057.908985:0:2384:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000040:0.0:1787827057.908988:0:38:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID req@ffff985607f0a940 x1874669228035456/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1787827057.908989:0:2384:0:(client.c:1813:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1787827057.909000:0:2384:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff985607f08b40 x1874669228035328/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787827057.909001:0:38:0:(jobid.c:939:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1787827057.909005:0:38:0:(jobid.c:979:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787827057.909015:0:38:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000083e565bf] to pc [ptlrpcd_00_00+0] req@ffff985607f0a940 x1874669228035456/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827057.909016:0:2384:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1787827057.909019:0:2384:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1787827057.909024:0:2384:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1787827057.909027:0:2384:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1787827057.909031:0:38:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1787827057.909032:0:2384:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787827057.909037:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1787827057.909043:0:38:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00100000:1.0:1787827057.909045:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607f08b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:3c946890-15ca-497a-b838-fe1cba61f4dd:2384:1874669228035328:192.168.203.114@tcp:400:kworker.0 00000100:00000010:0.0:1787827057.909048:0:38:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 904 at ffff985607f0bc00. 00000100:00000001:1.0:1787827057.909052:0:2384:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000020:00000040:0.0:1787827057.909053:0:38:0:(genops.c:1210:class_import_get()) import ffff985611185800 refcount=4 obd=lustre-MDT0001-mdc-ffff985602bc9800 00000100:00000001:0.0:1787827057.909057:0:38:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1787827057.909059:0:38:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1787827057.909059:0:2384:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1787827057.909064:0:38:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1787827057.909064:0:2384:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1787827057.909069:0:2384:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1787827057.909070:0:38:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff98561324c200. 00000100:00000001:0.0:1787827057.909075:0:38:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1787827057.909075:0:2384:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff985610db9a00. 02000000:00000001:1.0:1787827057.909081:0:2384:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787827057.909085:0:38:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID req@ffff985607f0bc00 x1874669228035584/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000400:00000200:1.0:1787827057.909091:0:2384:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000010:1.0:1787827057.909098:0:2384:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff985602ecf9f8. 00000100:00000001:0.0:1787827057.909100:0:38:0:(jobid.c:939:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1787827057.909104:0:38:0:(jobid.c:979:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1787827057.909106:0:2384:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985606618d48. 00000100:00000040:0.0:1787827057.909109:0:38:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000006d54edf2] to pc [ptlrpcd_00_01+1] req@ffff985607f0bc00 x1874669228035584/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1787827057.909114:0:2384:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1874669228035328, portal 25 00000100:00000001:1.0:1787827057.909119:0:2384:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1787827057.909123:0:2384:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446630094000655168 : -113979708896448 : ffff985607f08b40) 00000100:00000001:0.0:1787827057.909124:0:38:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787827057.909129:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-OST0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:1.0:1787827057.909132:0:2384:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff985607f08b40 x1874669228035328/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827073 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787827057.909134:0:38:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000001:0.0:1787827057.909138:0:38:0:(import.c:2066:ptlrpc_disconnect_and_idle_import()) Process entered 00000100:00000001:0.0:1787827057.909149:0:38:0:(import.c:1762:ptlrpc_disconnect_prep_req()) Process entered 00000100:00000001:1.0:1787827057.909153:0:2384:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1787827057.909157:0:2384:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.203.114@tcp 00000100:00000010:0.0:1787827057.909158:0:38:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 904 at ffff985607f08000. 00000020:00000040:0.0:1787827057.909162:0:38:0:(genops.c:1210:class_import_get()) import ffff9856110b0000 refcount=3 obd=lustre-OST0000-osc-ffff985602bc9800 00000400:00000010:1.0:1787827057.909162:0:2384:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985606618cc0. 00000100:00000001:0.0:1787827057.909168:0:38:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 00000100:00000200:1.0:1787827057.909168:0:2384:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1874669228035328, offset 0 02000000:00000001:0.0:1787827057.909172:0:38:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1787827057.909175:0:38:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1787827057.909179:0:2384:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.203.114@tcp 02000000:00000010:0.0:1787827057.909182:0:38:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff98561324cf00. 00000100:00000001:0.0:1787827057.909186:0:38:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787827057.909190:0:38:0:(import.c:1800:ptlrpc_disconnect_prep_req()) Process leaving (rc=18446630094000652288 : -113979708899328 : ffff985607f08000) 00000400:00000200:1.0:1787827057.909191:0:2384:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00080000:0.0:1787827057.909195:0:38:0:(import.c:65:import_set_state_nolock()) ffff9856110b0000 lustre-OST0000_UUID: changing import state from FULL to CONNECTING 00000100:00080000:0.0:1787827057.909200:0:38:0:(import.c:2101:ptlrpc_disconnect_and_idle_import()) lustre-OST0000-osc-ffff985602bc9800: disconnect after 21s idle 00000400:00000200:1.0:1787827057.909201:0:2384:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.203.114@tcp local destination 00000100:00000001:0.0:1787827057.909204:0:38:0:(jobid.c:939:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1787827057.909207:0:38:0:(jobid.c:979:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1787827057.909209:0:2384:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000040:0.0:1787827057.909215:0:38:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000008b959a28] to pc [ptlrpcd_rcv+-1] req@ffff985607f08000 x1874669228035712/t0(0) o9->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1787827057.909216:0:2384:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.203.14@tcp 00000400:00000200:1.0:1787827057.909222:0:2384:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.203.14@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1787827057.909229:0:2384:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.203.114@tcp 00000400:00000200:1.0:1787827057.909235:0:2384:0:(lib-move.c:1745:lnet_handle_send()) 192.168.203.14@tcp NI seq info: [21914:21914:256:4294967295] 192.168.203.114@tcp LPNI seq info [21914:21914:8:4294967295] 00000400:00000200:1.0:1787827057.909240:0:2384:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000001:0.0:1787827057.909244:0:38:0:(import.c:2105:ptlrpc_disconnect_and_idle_import()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1787827057.909249:0:38:0:(pinger.c:123:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787827057.909252:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000400:00000200:1.0:1787827057.909253:0:2384:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp:) -> 192.168.203.114@tcp(192.168.203.114@tcp:192.168.203.114@tcp) : PUT try# 0 00000100:00000040:0.0:1787827057.909261:0:38:0:(pinger.c:247:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000800:00000200:1.0:1787827057.909266:0:2384:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.203.114@tcp 00000100:00000040:0.0:1787827057.909269:0:38:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (3213) 00000800:00000010:1.0:1787827057.909273:0:2384:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff985607d95800. 00000800:00000200:1.0:1787827057.909281:0:2384:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff985606751480] -> 12345-192.168.203.114@tcp (5) 00000100:00000001:0.0:1787827057.909286:0:2383:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000200:1.0:1787827057.909290:0:2384:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.203.114@tcp ip 192.168.203.114:988 00000100:00000001:0.0:1787827057.909292:0:2383:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1787827057.909293:0:2382:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1787827057.909295:0:2383:0:(client.c:1813:ptlrpc_send_new_req()) Process entered 00000800:00000200:1.0:1787827057.909298:0:2384:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff985607d95800 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1787827057.909298:0:2382:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1787827057.909302:0:2382:0:(client.c:1813:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1787827057.909304:0:2383:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff985607f0a940 x1874669228035456/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1787827057.909312:0:2382:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff985607f08000 x1874669228035712/t0(0) o9->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787827057.909315:0:2383:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1787827057.909318:0:2384:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787827057.909319:0:2383:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1787827057.909326:0:2383:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1787827057.909330:0:2383:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000400:00000200:1.0:1787827057.909330:0:2384:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000001:3.0:1787827057.909331:0:2382:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1787827057.909335:0:2384:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.203.114@tcp: PM QoS new deadline estimation for cpu 2 is 4297885948 00000100:00000001:3.0:1787827057.909335:0:2382:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1787827057.909336:0:2383:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1787827057.909340:0:2382:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1787827057.909344:0:2382:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00100000:0.0:1787827057.909345:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607f0a940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669228035456:192.168.203.114@tcp:400:kworker.0 00000100:00000001:1.0:1787827057.909346:0:2384:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.203.114@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4297880828 02000000:00000001:3.0:1787827057.909349:0:2382:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827057.909353:0:2384:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.203.114@tcp: PM QoS new deadline estimation for cpu 3 is 4297885948 00000100:00000001:0.0:1787827057.909356:0:2383:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:1.0:1787827057.909359:0:2384:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.203.114@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4297880828 00000100:00100000:3.0:1787827057.909360:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607f08000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228035712:192.168.203.114@tcp:9:kworker.0 02000000:00000001:0.0:1787827057.909362:0:2383:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1787827057.909365:0:2383:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827057.909367:0:2382:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:1.0:1787827057.909368:0:2384:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1787827057.909369:0:2383:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:1.0:1787827057.909373:0:2384:0:(client.c:1941:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1787827057.909376:0:2383:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff98560540bc00. 00000100:00000001:1.0:1787827057.909378:0:2384:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1787827057.909380:0:2383:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827057.909383:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:3.0:1787827057.909384:0:2382:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 00000400:00000200:0.0:1787827057.909389:0:2383:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 02000000:00000001:3.0:1787827057.909389:0:2382:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827057.909393:0:2384:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 02000000:00000001:3.0:1787827057.909394:0:2382:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000010:0.0:1787827057.909395:0:2383:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9856094acf20. 00000100:00000001:1.0:1787827057.909397:0:2384:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 02000000:00000010:3.0:1787827057.909401:0:2382:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff985611860600. 00000400:00000010:0.0:1787827057.909402:0:2383:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985610d306e8. 00000100:00000001:1.0:1787827057.909402:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1787827057.909405:0:2384:0:(client.c:1990:ptlrpc_check_set()) Process entered 02000000:00000001:3.0:1787827057.909406:0:2382:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1787827057.909409:0:2383:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1874669228035456, portal 10 00000100:00000001:1.0:1787827057.909409:0:2384:0:(client.c:1813:ptlrpc_send_new_req()) Process entered 00000100:00000001:0.0:1787827057.909414:0:2383:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000400:00000200:3.0:1787827057.909414:0:2382:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000001:0.0:1787827057.909416:0:2383:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446630094000662848 : -113979708888768 : ffff985607f0a940) 00000100:00000040:1.0:1787827057.909416:0:2384:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff985607f0bc00 x1874669228035584/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000400:00000010:3.0:1787827057.909420:0:2382:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff985602ad9790. 00000100:00000040:0.0:1787827057.909430:0:2383:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff985607f0a940 x1874669228035456/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827073 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827057.909431:0:2384:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1787827057.909434:0:2384:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1787827057.909437:0:2382:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985609573088. 02000000:00000001:1.0:1787827057.909439:0:2384:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1787827057.909443:0:2384:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000200:3.0:1787827057.909445:0:2382:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1874669228035712, portal 4 00000100:00000001:0.0:1787827057.909447:0:2383:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000001:1.0:1787827057.909448:0:2384:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827057.909450:0:2382:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1787827057.909454:0:2382:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446630094000652288 : -113979708899328 : ffff985607f08000) 00000800:00000200:2.0:1787827057.909455:0:2377:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000040:0.0:1787827057.909456:0:2383:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827057.909456:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607f0bc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669228035584:192.168.203.114@tcp:400:kworker.0 00000400:00000010:0.0:1787827057.909463:0:2383:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985610d30b28. 00000800:00000010:2.0:1787827057.909464:0:2377:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff985607d95800. 00000100:00000001:1.0:1787827057.909465:0:2384:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000040:3.0:1787827057.909466:0:2382:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff985607f08000 x1874669228035712/t0(0) o9->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827067 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1787827057.909471:0:2383:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1874669228035456, offset 0 02000000:00000001:1.0:1787827057.909471:0:2384:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 00000400:00000200:2.0:1787827057.909474:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 02000000:00000001:1.0:1787827057.909475:0:2384:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1787827057.909481:0:2384:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000200:0.0:1787827057.909484:0:2383:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.203.114@tcp 00000400:00000200:2.0:1787827057.909486:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 02000000:00000010:1.0:1787827057.909487:0:2384:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff985610db8c00. 02000000:00000001:1.0:1787827057.909492:0:2384:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1787827057.909493:0:2383:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827057.909494:0:2377:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985606618cc0 00000100:00000001:3.0:1787827057.909496:0:2382:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000400:00000200:1.0:1787827057.909501:0:2384:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:0.0:1787827057.909502:0:2383:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.203.114@tcp local destination 00000400:00000010:2.0:1787827057.909503:0:2377:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985606618cc0. 00000100:00000040:3.0:1787827057.909504:0:2382:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.203.114@tcp 00000400:00000010:1.0:1787827057.909507:0:2384:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff985602ecf898. 00000100:00000001:2.0:1787827057.909511:0:2377:0:(events.c:34:request_out_callback()) Process entered 00000400:00000200:0.0:1787827057.909512:0:2383:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000010:3.0:1787827057.909512:0:2382:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985609573bb0. 00000400:00000010:1.0:1787827057.909514:0:2384:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985606618000. 00000400:00000200:0.0:1787827057.909520:0:2383:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.203.14@tcp 00000100:00000200:2.0:1787827057.909520:0:2377:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff985607f08b40 x1874669228035328/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827073 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1787827057.909520:0:2382:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 28, xid 1874669228035712, offset 0 00000100:00000200:1.0:1787827057.909521:0:2384:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1874669228035584, portal 10 00000400:00000200:0.0:1787827057.909528:0:2383:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.203.14@tcp lpni_is_preferred = 0 00000100:00000001:1.0:1787827057.909528:0:2384:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1787827057.909531:0:2384:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446630094000667648 : -113979708883968 : ffff985607f0bc00) 00000400:00000200:3.0:1787827057.909536:0:2382:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.203.114@tcp 00000400:00000200:0.0:1787827057.909537:0:2383:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.203.114@tcp 00000100:00000040:1.0:1787827057.909544:0:2384:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff985607f0bc00 x1874669228035584/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827073 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827057.909546:0:2377:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000400:00000200:0.0:1787827057.909547:0:2383:0:(lib-move.c:1745:lnet_handle_send()) 192.168.203.14@tcp NI seq info: [21915:21915:255:4294967295] 192.168.203.114@tcp LPNI seq info [21915:21915:7:4294967295] 00000400:00000200:3.0:1787827057.909550:0:2382:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000040:2.0:1787827057.909555:0:2377:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff985607f08b40 x1874669228035328/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827073 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1787827057.909559:0:2382:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.203.114@tcp local destination 00000400:00000200:0.0:1787827057.909562:0:2383:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:3.0:1787827057.909570:0:2382:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:1.0:1787827057.909571:0:2384:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000400:00000200:3.0:1787827057.909577:0:2382:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.203.14@tcp 00000100:00000040:1.0:1787827057.909579:0:2384:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.203.114@tcp 00000400:00000200:3.0:1787827057.909596:0:2382:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.203.14@tcp lpni_is_preferred = 0 00000100:00000001:2.0:1787827057.909597:0:2377:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1787827057.909598:0:2384:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985606618440. 00000400:00000200:3.0:1787827057.909602:0:2382:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.203.114@tcp 00000100:00000001:2.0:1787827057.909603:0:2377:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000200:1.0:1787827057.909604:0:2384:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1874669228035584, offset 0 00000400:00000200:3.0:1787827057.909610:0:2382:0:(lib-move.c:1745:lnet_handle_send()) 192.168.203.14@tcp NI seq info: [21916:21916:255:4294967295] 192.168.203.114@tcp LPNI seq info [21916:21916:7:4294967295] 00000400:00000200:1.0:1787827057.909614:0:2384:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.203.114@tcp 00000400:00000200:3.0:1787827057.909621:0:2382:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:1.0:1787827057.909623:0:2384:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:1.0:1787827057.909631:0:2384:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.203.114@tcp local destination 00000400:00000200:1.0:1787827057.909634:0:2384:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1787827057.909637:0:2384:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.203.14@tcp 00000400:00000200:1.0:1787827057.909639:0:2384:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.203.14@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1787827057.909642:0:2384:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.203.114@tcp 00000400:00000200:1.0:1787827057.909645:0:2384:0:(lib-move.c:1745:lnet_handle_send()) 192.168.203.14@tcp NI seq info: [21917:21917:254:4294967295] 192.168.203.114@tcp LPNI seq info [21917:21917:6:4294967295] 00000400:00000200:1.0:1787827057.909649:0:2384:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:3.0:1787827057.909779:0:2382:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp:) -> 192.168.203.114@tcp(192.168.203.114@tcp:192.168.203.114@tcp) : PUT try# 0 00000800:00000200:3.0:1787827057.909786:0:2382:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.203.114@tcp 00000800:00000010:3.0:1787827057.909792:0:2382:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff985609025200. 00000400:00000200:0.0:1787827057.909794:0:2383:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp:) -> 192.168.203.114@tcp(192.168.203.114@tcp:192.168.203.114@tcp) : PUT try# 0 00000800:00000200:0.0:1787827057.909803:0:2383:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.203.114@tcp 00000800:00000010:0.0:1787827057.909808:0:2383:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff98561324cb00. 00000800:00000200:3.0:1787827057.909809:0:2382:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff985606751480] -> 12345-192.168.203.114@tcp (5) 00000800:00000200:0.0:1787827057.909817:0:2383:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff985606751480] -> 12345-192.168.203.114@tcp (5) 00000400:00000200:1.0:1787827057.909821:0:2384:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp:) -> 192.168.203.114@tcp(192.168.203.114@tcp:192.168.203.114@tcp) : PUT try# 0 00000800:00000200:3.0:1787827057.909822:0:2382:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.203.114@tcp ip 192.168.203.114:988 00000800:00000200:0.0:1787827057.909829:0:2383:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.203.114@tcp ip 192.168.203.114:988 00000800:00000200:3.0:1787827057.909830:0:2382:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff985609025200 type 1, nob 320 niov 1 nkiov 1 00000800:00000200:1.0:1787827057.909835:0:2384:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.203.114@tcp 00000800:00000200:0.0:1787827057.909836:0:2383:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff98561324cb00 type 1, nob 320 niov 1 nkiov 1 00000800:00000010:1.0:1787827057.909842:0:2384:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff985607d95100. 00000100:00000001:3.0:1787827057.909848:0:2382:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787827057.909849:0:2383:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1787827057.909849:0:2384:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff985606751480] -> 12345-192.168.203.114@tcp (5) 00000100:00000040:3.0:1787827057.909853:0:2382:0:(niobuf.c:635:kick_cpu_latency()) lustre-OST0000-osc-ffff985602bc9800: using a timeout of 14618 usecs (15 jiffies) 00000100:00000040:0.0:1787827057.909856:0:2383:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff985602bc9800: using a timeout of 38041 usecs (39 jiffies) 00000800:00000200:1.0:1787827057.909860:0:2384:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.203.114@tcp ip 192.168.203.114:988 00000400:00000200:3.0:1787827057.909861:0:2382:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000800:00000200:1.0:1787827057.909866:0:2384:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff985607d95100 type 1, nob 320 niov 1 nkiov 1 00000400:00000200:0.0:1787827057.909867:0:2383:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000001:3.0:1787827057.909867:0:2382:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff985602bc9800: PM QoS new deadline estimation for cpu 2 is 4297875984 00000100:00000001:1.0:1787827057.909873:0:2384:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787827057.909874:0:2383:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff985602bc9800: PM QoS new deadline estimation for cpu 2 is 4297876008 00000100:00000001:3.0:1787827057.909875:0:2382:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff985602bc9800: PM QoS request active for cpu 2, keep current deadline 4297885948 00000100:00000040:1.0:1787827057.909878:0:2384:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff985602bc9800: using a timeout of 11908 usecs (12 jiffies) 00000100:00000001:3.0:1787827057.909880:0:2382:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff985602bc9800: PM QoS new deadline estimation for cpu 3 is 4297875984 00000100:00000001:0.0:1787827057.909881:0:2383:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff985602bc9800: PM QoS request active for cpu 2, keep current deadline 4297885948 00000100:00000001:3.0:1787827057.909886:0:2382:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff985602bc9800: PM QoS request active for cpu 3, keep current deadline 4297885948 00000400:00000200:1.0:1787827057.909887:0:2384:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000001:0.0:1787827057.909888:0:2383:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff985602bc9800: PM QoS new deadline estimation for cpu 3 is 4297876008 00000100:00000001:1.0:1787827057.909893:0:2384:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff985602bc9800: PM QoS new deadline estimation for cpu 2 is 4297875981 00000100:00000001:3.0:1787827057.909893:0:2382:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1787827057.909894:0:2383:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff985602bc9800: PM QoS request active for cpu 3, keep current deadline 4297885948 00000100:00000001:1.0:1787827057.909898:0:2384:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff985602bc9800: PM QoS request active for cpu 2, keep current deadline 4297885948 00000100:00000001:0.0:1787827057.909900:0:2383:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1787827057.909900:0:2382:0:(client.c:1941:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827057.909902:0:2384:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff985602bc9800: PM QoS new deadline estimation for cpu 3 is 4297875981 00000100:00000001:3.0:1787827057.909906:0:2382:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787827057.909907:0:2383:0:(client.c:1941:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827057.909908:0:2384:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff985602bc9800: PM QoS request active for cpu 3, keep current deadline 4297885948 00000100:00000001:3.0:1787827057.909910:0:2382:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1787827057.909913:0:2383:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827057.909914:0:2384:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1787827057.909917:0:2382:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1787827057.909919:0:2383:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787827057.909919:0:2384:0:(client.c:1941:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1787827057.909921:0:2378:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:3.0:1787827057.909921:0:2382:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=5 : 5 : 5) 00000100:00000001:1.0:1787827057.909926:0:2384:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787827057.909927:0:2383:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000800:00000010:2.0:1787827057.909928:0:2378:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff985609025200. 00000100:00000001:3.0:1787827057.909928:0:2382:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1787827057.909929:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1787827057.909931:0:2383:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1787827057.909932:0:2382:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1787827057.909935:0:2384:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1787827057.909936:0:2383:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1787827057.909937:0:2382:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827057.909938:0:2384:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1787827057.909940:0:2383:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1787827057.909942:0:2382:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827057.909944:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:2.0:1787827057.909944:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1787827057.909945:0:2383:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827057.909948:0:2384:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1787827057.909952:0:2383:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827057.909953:0:2384:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1787827057.909955:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000100:00000001:1.0:1787827057.909959:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1787827057.909960:0:2378:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985609573bb0 00000400:00000010:2.0:1787827057.909964:0:2378:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985609573bb0. 00000100:00000001:2.0:1787827057.909967:0:2378:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1787827057.909972:0:2378:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff985607f08000 x1874669228035712/t0(0) o9->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827067 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827057.909990:0:2378:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827057.909994:0:2378:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff985607f08000 x1874669228035712/t0(0) o9->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827067 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827057.910000:0:2378:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1787827057.910002:0:2378:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:2.0:1787827057.910011:0:2378:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1787827057.910014:0:2378:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff98561324cb00. 00000400:00000200:2.0:1787827057.910016:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1787827057.910019:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000400:00000200:2.0:1787827057.910021:0:2378:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985610d30b28 00000400:00000010:2.0:1787827057.910023:0:2378:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985610d30b28. 00000100:00000001:2.0:1787827057.910025:0:2378:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1787827057.910029:0:2378:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff985607f0a940 x1874669228035456/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827073 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827057.910035:0:2378:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827057.910038:0:2378:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff985607f0a940 x1874669228035456/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827073 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827057.910044:0:2378:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1787827057.910045:0:2378:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:2.0:1787827057.910077:0:2378:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1787827057.910080:0:2378:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff985607d95100. 00000400:00000200:2.0:1787827057.910083:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1787827057.910086:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000400:00000200:2.0:1787827057.910088:0:2378:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985606618440 00000400:00000010:2.0:1787827057.910090:0:2378:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985606618440. 00000100:00000001:2.0:1787827057.910100:0:2378:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1787827057.910103:0:2378:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff985607f0bc00 x1874669228035584/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827073 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827057.910110:0:2378:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827057.910113:0:2378:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff985607f0bc00 x1874669228035584/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827073 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827057.910118:0:2378:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1787827057.910120:0:2378:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1787827057.910533:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827057.910543:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1787827057.910545:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827057.910548:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827057.910553:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827057.910558:0:2377:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp) <- 192.168.203.114@tcp : PUT - for me 00000400:00000200:2.0:1787827057.910571:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827057.910574:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827057.910578:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827057.910582:0:2377:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.203.114@tcp of length 224 into portal 25 MB=0x6a90071688d00 00000400:00000200:2.0:1787827057.910597:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827057.910602:0:2377:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.203.114@tcp of length 224/224 into md 0x32405 [1] + 224 00000800:00000001:2.0:1787827057.910607:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827057.910617:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827057.910620:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1787827057.910623:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000100:00000001:2.0:1787827057.910627:0:2377:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1787827057.910631:0:2377:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff985607f08b40 x1874669228035328/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827073 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1787827057.910640:0:2377:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff985607f08b40 x1874669228035328/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827073 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827057.910653:0:2377:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1787827057.910659:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827057.910661:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787827057.910684:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1787827057.910686:0:2384:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1787827057.910700:0:2384:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1787827057.910702:0:2384:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1787827057.910704:0:2384:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff985602ecf9f8. 00000400:00000200:1.0:1787827057.910706:0:2384:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985606618d48 00000400:00000010:1.0:1787827057.910708:0:2384:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985606618d48. 00000100:00000001:1.0:1787827057.910709:0:2384:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1787827057.910714:0:2384:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff985607f08b40 x1874669228035328/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827073 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1787827057.910723:0:2384:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff985607f08b40 x1874669228035328/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827073 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827057.910730:0:2384:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1787827057.910732:0:2384:0:(client.c:3001:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787827057.910734:0:2384:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:1.0:1787827057.910736:0:2384:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1787827057.910738:0:2384:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1787827057.910740:0:2384:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1787827057.910743:0:2384:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1787827057.910748:0:2384:0:(import.c:2149:obd_at_measure()) add 5 to ffff98561118e460 time=57 v=5 (5 5 5 5) 00000100:00001000:1.0:1787827057.910752:0:2384:0:(import.c:2149:obd_at_measure()) add 1 to ffff98561118e418 time=72 v=5 (1 0 0 0) 00000100:00000001:1.0:1787827057.910754:0:2384:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1787827057.910756:0:2384:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1787827057.910759:0:2384:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1787827057.910761:0:2384:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827057.910764:0:2384:0:(client.c:1797:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1787827057.910769:0:2384:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff985607f08b40 x1874669228035328/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827073 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827057.910776:0:2384:0:(client.c:2354:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1787827057.910779:0:2384:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1787827057.910781:0:2384:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1787827057.910784:0:2384:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff985607f08b40 x1874669228035328/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827073 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827057.910789:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607f08b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:3c946890-15ca-497a-b838-fe1cba61f4dd:2384:1874669228035328:192.168.203.114@tcp:400:kworker.0 00000100:00000001:1.0:1787827057.910795:0:2384:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827057.910796:0:2384:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1787827057.910799:0:2384:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607f08b40 x1874669228035328/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827073 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827057.910803:0:2384:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1787827057.910805:0:2384:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1787827057.910807:0:2384:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff985610db9a00. 02000000:00000001:1.0:1787827057.910809:0:2384:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1787827057.910811:0:2384:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:1.0:1787827057.910812:0:2384:0:(genops.c:1223:class_import_put()) import ffff98561118e000 refcount=3 obd=MGC192.168.203.114@tcp 00000020:00000001:1.0:1787827057.910813:0:2384:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:1.0:1787827057.910815:0:2384:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff98561324ca00. 02000000:00000001:1.0:1787827057.910817:0:2384:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1787827057.910819:0:2384:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1787827057.910820:0:2384:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607f08b40. 00000100:00000001:1.0:1787827057.910822:0:2384:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1787827057.910823:0:2384:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787827057.910825:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827057.910826:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1787827057.910827:0:2384:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1787827057.910829:0:2384:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827057.910830:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1787827057.910881:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827057.910889:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1787827057.910891:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827057.910894:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827057.910898:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827057.910902:0:2378:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp) <- 192.168.203.114@tcp : PUT - for me 00000400:00000200:2.0:1787827057.910907:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827057.910909:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827057.910912:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827057.910914:0:2378:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.203.114@tcp of length 224 into portal 10 MB=0x6a90071688d80 00000400:00000200:2.0:1787827057.910917:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827057.910921:0:2378:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.203.114@tcp of length 224/224 into md 0x3240d [1] + 224 00000800:00000001:2.0:1787827057.910925:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827057.910927:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827057.910929:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1787827057.910933:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000100:00000001:2.0:1787827057.910935:0:2378:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1787827057.910939:0:2378:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff985607f0a940 x1874669228035456/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827073 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1787827057.910952:0:2378:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff985607f0a940 x1874669228035456/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827073 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827057.910972:0:2378:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1787827057.910976:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827057.910988:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1787827057.910990:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827057.910994:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827057.910997:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827057.911001:0:2378:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp) <- 192.168.203.114@tcp : PUT - for me 00000400:00000200:2.0:1787827057.911005:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827057.911007:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827057.911010:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827057.911012:0:2378:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.203.114@tcp of length 224 into portal 10 MB=0x6a90071688e00 00000400:00000200:2.0:1787827057.911015:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827057.911018:0:2378:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.203.114@tcp of length 224/224 into md 0x32425 [1] + 224 00000800:00000001:2.0:1787827057.911021:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827057.911025:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827057.911027:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1787827057.911030:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000100:00000001:2.0:1787827057.911033:0:2378:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1787827057.911036:0:2378:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff985607f0bc00 x1874669228035584/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827073 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1787827057.911043:0:2378:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff985607f0bc00 x1874669228035584/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827073 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827057.911056:0:2378:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1787827057.911059:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827057.911062:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787827057.911094:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1787827057.911097:0:2384:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1787827057.911100:0:2384:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1787827057.911102:0:2384:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1787827057.911105:0:2384:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff985602ecf898. 00000400:00000200:1.0:1787827057.911109:0:2384:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985606618000 00000400:00000010:1.0:1787827057.911111:0:2384:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985606618000. 00000100:00000001:1.0:1787827057.911114:0:2384:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:0.0:1787827057.911119:0:2383:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000200:1.0:1787827057.911120:0:2384:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff985607f0bc00 x1874669228035584/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827073 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787827057.911123:0:2383:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1787827057.911128:0:2383:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1787827057.911130:0:2383:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1787827057.911135:0:2383:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9856094acf20. 00000100:00000200:1.0:1787827057.911136:0:2384:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff985607f0bc00 x1874669228035584/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827073 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000400:00000200:0.0:1787827057.911139:0:2383:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985610d306e8 00000400:00000010:0.0:1787827057.911143:0:2383:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985610d306e8. 00000100:00000001:0.0:1787827057.911148:0:2383:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:1.0:1787827057.911150:0:2384:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1787827057.911153:0:2384:0:(client.c:3001:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:0.0:1787827057.911156:0:2383:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff985607f0a940 x1874669228035456/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827073 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827057.911156:0:2384:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:1.0:1787827057.911159:0:2384:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000200:0.0:1787827057.911167:0:2383:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff985607f0a940 x1874669228035456/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827073 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827057.911170:0:2384:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1787827057.911173:0:2384:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787827057.911176:0:2383:0:(events.c:163:reply_in_callback()) Process leaving 02000000:00000001:1.0:1787827057.911177:0:2384:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787827057.911179:0:2383:0:(client.c:3001:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1787827057.911182:0:2383:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:0.0:1787827057.911184:0:2383:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00001000:1.0:1787827057.911184:0:2384:0:(import.c:2149:obd_at_measure()) add 5 to ffff985611185c60 time=57 v=5 (5 5 5 5) 00000100:00000001:0.0:1787827057.911187:0:2383:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1787827057.911190:0:2383:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1787827057.911191:0:2384:0:(import.c:2149:obd_at_measure()) add 1 to ffff985611185c18 time=57 v=5 (1 1 1 1) 02000000:00000001:0.0:1787827057.911194:0:2383:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827057.911195:0:2384:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1787827057.911198:0:2384:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1787827057.911200:0:2384:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00001000:0.0:1787827057.911201:0:2383:0:(import.c:2149:obd_at_measure()) add 5 to ffff985611182460 time=135 v=5 (5 5 5 5) 00010000:00000001:1.0:1787827057.911203:0:2384:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1787827057.911205:0:2383:0:(import.c:2149:obd_at_measure()) add 1 to ffff985611182418 time=71 v=5 (1 0 0 0) 00000100:00000001:1.0:1787827057.911207:0:2384:0:(client.c:3109:ptlrpc_free_committed()) Process entered 00000100:00000001:0.0:1787827057.911209:0:2383:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000040:1.0:1787827057.911211:0:2384:0:(client.c:3121:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff985602bc9800: skip recheck: last_committed 0 00000100:00000001:0.0:1787827057.911213:0:2383:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827057.911214:0:2384:0:(client.c:3122:ptlrpc_free_committed()) Process leaving 00010000:00000001:0.0:1787827057.911217:0:2383:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:1.0:1787827057.911218:0:2384:0:(client.c:1797:after_reply()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827057.911221:0:2383:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787827057.911226:0:2383:0:(client.c:3109:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1787827057.911226:0:2384:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff985607f0bc00 x1874669228035584/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827073 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1787827057.911229:0:2383:0:(client.c:3121:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: skip recheck: last_committed 219043334508 00000100:00000001:0.0:1787827057.911233:0:2383:0:(client.c:3122:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1787827057.911235:0:2383:0:(client.c:1797:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787827057.911242:0:2383:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff985607f0a940 x1874669228035456/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827073 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827057.911243:0:2384:0:(client.c:2354:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1787827057.911248:0:2384:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1787827057.911251:0:2384:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1787827057.911257:0:2383:0:(client.c:2354:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1787827057.911266:0:2383:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000040:1.0:1787827057.911266:0:2384:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff985607f0bc00 x1874669228035584/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827073 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787827057.911269:0:2383:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1787827057.911275:0:2383:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff985607f0a940 x1874669228035456/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827073 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827057.911288:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607f0bc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669228035584:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787827057.911293:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607f0a940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669228035456:192.168.203.114@tcp:400:kworker.0 00000100:00000001:1.0:1787827057.911296:0:2384:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787827057.911299:0:2384:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1787827057.911300:0:2383:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1787827057.911305:0:2384:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607f0bc00 x1874669228035584/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827073 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787827057.911306:0:2383:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1787827057.911312:0:2383:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607f0a940 x1874669228035456/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827073 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827057.911323:0:2384:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1787827057.911326:0:2384:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:0.0:1787827057.911327:0:2383:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1787827057.911330:0:2383:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1787827057.911330:0:2384:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff985610db8c00. 02000000:00000010:0.0:1787827057.911334:0:2383:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff98560540bc00. 02000000:00000001:1.0:1787827057.911336:0:2384:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1787827057.911339:0:2384:0:(genops.c:1217:class_import_put()) Process entered 02000000:00000001:0.0:1787827057.911340:0:2383:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000040:1.0:1787827057.911343:0:2384:0:(genops.c:1223:class_import_put()) import ffff985611185800 refcount=3 obd=lustre-MDT0001-mdc-ffff985602bc9800 00000020:00000001:0.0:1787827057.911344:0:2383:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:0.0:1787827057.911346:0:2383:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=3 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:0.0:1787827057.911348:0:2383:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:0.0:1787827057.911350:0:2383:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff98561324c400. 02000000:00000001:0.0:1787827057.911352:0:2383:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1787827057.911354:0:2383:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000020:00000001:1.0:1787827057.911354:0:2384:0:(genops.c:1230:class_import_put()) Process leaving 00000100:00000010:0.0:1787827057.911358:0:2383:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607f0a940. 02000000:00000010:1.0:1787827057.911358:0:2384:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff98561324c200. 00000100:00000001:0.0:1787827057.911361:0:2383:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1787827057.911363:0:2383:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1787827057.911366:0:2383:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:1.0:1787827057.911366:0:2384:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1787827057.911369:0:2384:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1787827057.911371:0:2384:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607f0bc00. 00000100:00000001:1.0:1787827057.911374:0:2384:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1787827057.911375:0:2384:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787827057.911377:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1787827057.911378:0:2383:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1787827057.911381:0:2384:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1787827057.911382:0:2383:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827057.911385:0:2384:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787827057.911386:0:2383:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1787827057.911390:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1787827057.911391:0:2383:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827057.911395:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1787827057.911597:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827057.911604:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1787827057.911606:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827057.911609:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827057.911613:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827057.911618:0:2377:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp) <- 192.168.203.114@tcp : PUT - for me 00000400:00000200:2.0:1787827057.911623:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827057.911626:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827057.911629:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827057.911633:0:2377:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.203.114@tcp of length 224 into portal 4 MB=0x6a90071688e80 00000400:00000200:2.0:1787827057.911636:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827057.911640:0:2377:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 4 from 12345-192.168.203.114@tcp of length 224/224 into md 0x32415 [1] + 224 00000800:00000001:2.0:1787827057.911644:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827057.911649:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827057.911651:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1787827057.911654:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000100:00000001:2.0:1787827057.911658:0:2377:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1787827057.911662:0:2377:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff985607f08000 x1874669228035712/t0(0) o9->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827067 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1787827057.911672:0:2377:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff985607f08000 x1874669228035712/t0(0) o9->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827067 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827057.911685:0:2377:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1787827057.911706:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827057.911709:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1787827057.911730:0:2382:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1787827057.911732:0:2382:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000400:00000001:2.0:1787827057.911736:0:2382:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1787827057.911738:0:2382:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1787827057.911740:0:2382:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff985602ad9790. 00000400:00000200:2.0:1787827057.911744:0:2382:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985609573088 00000400:00000010:2.0:1787827057.911746:0:2382:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985609573088. 00000100:00000001:2.0:1787827057.911748:0:2382:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1787827057.911752:0:2382:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff985607f08000 x1874669228035712/t0(0) o9->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827067 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1787827057.911761:0:2382:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff985607f08000 x1874669228035712/t0(0) o9->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827067 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827057.911773:0:2382:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1787827057.911775:0:2382:0:(client.c:3001:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1787827057.911777:0:2382:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:2.0:1787827057.911778:0:2382:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1787827057.911780:0:2382:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1787827057.911781:0:2382:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1787827057.911783:0:2382:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1787827057.911788:0:2382:0:(import.c:2149:obd_at_measure()) add 5 to ffff9856110b0460 time=135 v=5 (5 5 5 5) 00000100:00001000:2.0:1787827057.911791:0:2382:0:(import.c:2149:obd_at_measure()) add 1 to ffff9856110b0418 time=21 v=5 (1 0 0 0) 00000100:00000001:2.0:1787827057.911793:0:2382:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1787827057.911795:0:2382:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1787827057.911796:0:2382:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:2.0:1787827057.911799:0:2382:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1787827057.911801:0:2382:0:(client.c:3109:ptlrpc_free_committed()) Process entered 00000100:00000040:2.0:1787827057.911802:0:2382:0:(client.c:3121:ptlrpc_free_committed()) lustre-OST0000-osc-ffff985602bc9800: skip recheck: last_committed 8589935703 00000100:00000001:2.0:1787827057.911804:0:2382:0:(client.c:3122:ptlrpc_free_committed()) Process leaving 00000100:00000001:2.0:1787827057.911805:0:2382:0:(client.c:1797:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1787827057.911809:0:2382:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff985607f08000 x1874669228035712/t0(0) o9->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827067 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827057.911816:0:2382:0:(client.c:2354:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1787827057.911818:0:2382:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1787827057.911820:0:2382:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00080000:2.0:1787827057.911824:0:2382:0:(import.c:2011:ptlrpc_disconnect_idle_interpret()) @@@ inflight=1, refcount=3: rc = 0 req@ffff985607f08000 x1874669228035712/t0(0) o9->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827067 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827057.911832:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9856110b0000 lustre-OST0000_UUID: changing import state from CONNECTING to IDLE 00000100:00000040:2.0:1787827057.911836:0:2382:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff985607f08000 x1874669228035712/t0(0) o9->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827067 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827057.911844:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607f08000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228035712:192.168.203.114@tcp:9:kworker.0 00000100:00000001:2.0:1787827057.911848:0:2382:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1787827057.911849:0:2382:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827057.911853:0:2382:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607f08000 x1874669228035712/t0(0) o9->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827067 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827057.911860:0:2382:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1787827057.911861:0:2382:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1787827057.911863:0:2382:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff985611860600. 02000000:00000001:2.0:1787827057.911865:0:2382:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1787827057.911867:0:2382:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1787827057.911868:0:2382:0:(genops.c:1223:class_import_put()) import ffff9856110b0000 refcount=2 obd=lustre-OST0000-osc-ffff985602bc9800 00000020:00000001:2.0:1787827057.911870:0:2382:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1787827057.911876:0:2382:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff98561324cf00. 02000000:00000001:2.0:1787827057.911879:0:2382:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1787827057.911880:0:2382:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1787827057.911882:0:2382:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607f08000. 00000100:00000001:2.0:1787827057.911884:0:2382:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1787827057.911885:0:2382:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1787827057.911887:0:2382:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1787827057.911890:0:2382:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1787827057.911892:0:2382:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1787827057.911894:0:2382:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1787827057.911895:0:2382:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827058.484802:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827058.484809:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827058.484814:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827058.484815:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827058.484818:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827058.484821:0:49842:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:0.0:1787827058.484823:0:49842:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:0.0:1787827058.484829:0:49842:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:0.0:1787827058.484831:0:49842:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:0.0:1787827058.484833:0:49842:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827058.484836:0:49842:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:0.0:1787827058.484838:0:49842:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827058.484840:0:49842:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827058.484842:0:49842:0:(ldlm_pool.c:496:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1787827058.484903:0:15163:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1787827058.484906:0:15163:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1787827058.484910:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:0.0:1787827059.508853:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827059.508862:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827059.508870:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827059.508871:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827059.508875:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827059.508877:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1787827059.508962:0:33341:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1787827059.508969:0:33341:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1787827059.508973:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:0.0:1787827060.533258:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827060.533266:0:49842:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:0.0:1787827060.533269:0:49842:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:0.0:1787827060.533272:0:49842:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827060.533277:0:49842:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:0.0:1787827060.533279:0:49842:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827060.533281:0:49842:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827060.533283:0:49842:0:(ldlm_pool.c:496:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827060.533293:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827060.533295:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827060.533298:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827060.533300:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1787827060.533323:0:15164:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1787827060.533325:0:15164:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1787827060.533328:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:0.0:1787827061.556851:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827061.556860:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827061.556867:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827061.556869:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827061.556872:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827061.556874:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1787827061.556913:0:33342:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1787827061.556916:0:33342:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1787827061.556919:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:0.0:1787827062.581548:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827062.581565:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827062.581571:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827062.581572:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827062.581575:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827062.581576:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1787827062.581723:0:79887:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1787827062.581726:0:79887:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1787827062.581729:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:2.0:1787827063.028912:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1787827063.028923:0:38:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1787827063.028932:0:38:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 904 at ffff9856140d8f00. 00000020:00000040:2.0:1787827063.028939:0:38:0:(genops.c:1210:class_import_get()) import ffff98561118e000 refcount=4 obd=MGC192.168.203.114@tcp 00000100:00000001:2.0:1787827063.028943:0:38:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1787827063.028946:0:38:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1787827063.028950:0:38:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1787827063.028958:0:38:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff985615779d00. 00000100:00000001:2.0:1787827063.028965:0:38:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1787827063.028976:0:38:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS req@ffff9856140d8f00 x1874669228035840/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1787827063.028987:0:38:0:(jobid.c:939:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1787827063.028991:0:38:0:(jobid.c:979:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1787827063.028998:0:38:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000e9d4c698] to pc [ptlrpcd_01_00+0] req@ffff9856140d8f00 x1874669228035840/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827063.029037:0:38:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1787827063.029040:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1787827063.029044:0:38:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1787827063.029047:0:38:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 904 at ffff9856140db480. 00000020:00000040:2.0:1787827063.029050:0:38:0:(genops.c:1210:class_import_get()) import ffff985611182000 refcount=4 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000100:00000001:2.0:1787827063.029051:0:38:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1787827063.029053:0:38:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1787827063.029055:0:38:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1787827063.029058:0:38:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff985615779800. 00000100:00000001:2.0:1787827063.029061:0:38:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1787827063.029065:0:38:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID req@ffff9856140db480 x1874669228035968/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1787827063.029072:0:38:0:(jobid.c:939:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1787827063.029074:0:38:0:(jobid.c:979:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1787827063.029078:0:38:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000f7a0de5b] to pc [ptlrpcd_01_01+1] req@ffff9856140db480 x1874669228035968/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827063.029086:0:38:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1787827063.029088:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1787827063.029091:0:38:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1787827063.029093:0:38:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 904 at ffff9856140da940. 00000020:00000040:2.0:1787827063.029096:0:38:0:(genops.c:1210:class_import_get()) import ffff985611185800 refcount=4 obd=lustre-MDT0001-mdc-ffff985602bc9800 00000100:00000001:2.0:1787827063.029098:0:38:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1787827063.029099:0:38:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1787827063.029100:0:38:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1787827063.029103:0:38:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff985615779600. 00000100:00000001:2.0:1787827063.029106:0:38:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1787827063.029110:0:38:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID req@ffff9856140da940 x1874669228036096/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1787827063.029116:0:38:0:(jobid.c:939:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1787827063.029118:0:38:0:(jobid.c:979:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1787827063.029123:0:38:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000a1a886e6] to pc [ptlrpcd_01_00+0] req@ffff9856140da940 x1874669228036096/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827063.029130:0:38:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1787827063.029132:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:2.0:1787827063.029136:0:38:0:(pinger.c:247:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000040:2.0:1787827063.029139:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1787827063.029139:0:2385:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:2.0:1787827063.029145:0:38:0:(pinger.c:247:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000001:3.0:1787827063.029146:0:2385:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1787827063.029150:0:2385:0:(client.c:1813:ptlrpc_send_new_req()) Process entered 00000100:00000040:2.0:1787827063.029152:0:38:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (3218) 00000100:00000040:3.0:1787827063.029160:0:2385:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9856140d8f00 x1874669228035840/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827063.029169:0:2386:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1787827063.029169:0:2385:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1787827063.029174:0:2385:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1787827063.029178:0:2385:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1787827063.029180:0:2385:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1787827063.029182:0:2385:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1787827063.029186:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2385:1874669228035840:192.168.203.114@tcp:400:kworker.0 00000100:00000001:3.0:1787827063.029191:0:2385:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:3.0:1787827063.029196:0:2385:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1787827063.029198:0:2385:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1787827063.029200:0:2385:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:3.0:1787827063.029205:0:2385:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff985611861e00. 02000000:00000001:3.0:1787827063.029207:0:2385:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1787827063.029213:0:2385:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000010:3.0:1787827063.029217:0:2385:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff985602ad93c8. 00000400:00000010:3.0:1787827063.029223:0:2385:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9856095733b8. 00000100:00000200:3.0:1787827063.029229:0:2385:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1874669228035840, portal 25 00000100:00000001:3.0:1787827063.029231:0:2385:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1787827063.029232:0:2385:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446630094203883264 : -113979505668352 : ffff9856140d8f00) 00000100:00000001:2.0:1787827063.029233:0:2386:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1787827063.029237:0:2386:0:(client.c:1813:ptlrpc_send_new_req()) Process entered 00000100:00000040:2.0:1787827063.029242:0:2386:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9856140db480 x1874669228035968/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827063.029252:0:2386:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000040:3.0:1787827063.029253:0:2385:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9856140d8f00 x1874669228035840/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827079 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827063.029255:0:2386:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1787827063.029260:0:2386:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1787827063.029263:0:2386:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1787827063.029268:0:2385:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000001:2.0:1787827063.029269:0:2386:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1787827063.029276:0:2385:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827063.029277:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228035968:192.168.203.114@tcp:400:kworker.0 00000400:00000010:3.0:1787827063.029281:0:2385:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985609573770. 00000100:00000001:2.0:1787827063.029282:0:2386:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1787827063.029287:0:2386:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1787827063.029288:0:2386:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1787827063.029290:0:2386:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:2.0:1787827063.029295:0:2386:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff985607e33c00. 02000000:00000001:2.0:1787827063.029297:0:2386:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1787827063.029300:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000010:2.0:1787827063.029304:0:2386:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff98561080eb00. 00000400:00000010:2.0:1787827063.029308:0:2386:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985610afa3b8. 00000100:00000200:2.0:1787827063.029311:0:2386:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1874669228035968, portal 10 00000100:00000001:2.0:1787827063.029313:0:2386:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1787827063.029314:0:2386:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446630094203892864 : -113979505658752 : ffff9856140db480) 00000100:00000040:2.0:1787827063.029321:0:2386:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9856140db480 x1874669228035968/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827079 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827063.029327:0:2386:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:2.0:1787827063.029330:0:2386:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.203.114@tcp 00000400:00000010:2.0:1787827063.029332:0:2386:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985610afadd0. 00000100:00000200:2.0:1787827063.029335:0:2386:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1874669228035968, offset 0 00000400:00000200:2.0:1787827063.029352:0:2386:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.203.114@tcp 00000400:00000200:2.0:1787827063.029361:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827063.029368:0:2386:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.203.114@tcp local destination 00000400:00000200:2.0:1787827063.029373:0:2386:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:2.0:1787827063.029377:0:2386:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.203.14@tcp 00000400:00000200:2.0:1787827063.029380:0:2386:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.203.14@tcp lpni_is_preferred = 0 00000400:00000200:2.0:1787827063.029383:0:2386:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.203.114@tcp 00000400:00000200:2.0:1787827063.029386:0:2386:0:(lib-move.c:1745:lnet_handle_send()) 192.168.203.14@tcp NI seq info: [21918:21918:256:4294967295] 192.168.203.114@tcp LPNI seq info [21918:21918:8:4294967295] 00000400:00000200:2.0:1787827063.029391:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827063.029399:0:2386:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp:) -> 192.168.203.114@tcp(192.168.203.114@tcp:192.168.203.114@tcp) : PUT try# 0 00000800:00000200:2.0:1787827063.029405:0:2386:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.203.114@tcp 00000800:00000010:2.0:1787827063.029411:0:2386:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff985615779100. 00000800:00000200:2.0:1787827063.029417:0:2386:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff985606751480] -> 12345-192.168.203.114@tcp (5) 00000800:00000200:2.0:1787827063.029425:0:2386:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.203.114@tcp ip 192.168.203.114:988 00000800:00000200:2.0:1787827063.029429:0:2386:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff985615779100 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:2.0:1787827063.029440:0:2386:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1787827063.029444:0:2386:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff985602bc9800: using a timeout of 38040 usecs (39 jiffies) 00000400:00000200:2.0:1787827063.029447:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000001:2.0:1787827063.029449:0:2386:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff985602bc9800: PM QoS new deadline estimation for cpu 2 is 4297881127 00000100:00000001:2.0:1787827063.029452:0:2386:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff985602bc9800: PM QoS request active for cpu 2, keep current deadline 4297885948 00000100:00000001:2.0:1787827063.029455:0:2386:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff985602bc9800: PM QoS new deadline estimation for cpu 3 is 4297881127 00000100:00000001:2.0:1787827063.029457:0:2386:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff985602bc9800: PM QoS request active for cpu 3, keep current deadline 4297885948 00000100:00000001:2.0:1787827063.029460:0:2386:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1787827063.029462:0:2386:0:(client.c:1941:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1787827063.029465:0:2386:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1787827063.029467:0:2386:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1787827063.029473:0:2386:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1787827063.029475:0:2386:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1787827063.029477:0:2386:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1787827063.029479:0:2386:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1787827063.029480:0:2386:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1787827063.029482:0:2386:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1787827063.029579:0:2378:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1787827063.029583:0:2378:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff985615779100. 00000400:00000200:2.0:1787827063.029588:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1787827063.029593:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000400:00000200:2.0:1787827063.029597:0:2378:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985610afadd0 00000400:00000010:2.0:1787827063.029599:0:2378:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985610afadd0. 00000100:00000001:2.0:1787827063.029603:0:2378:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1787827063.029606:0:2378:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff9856140db480 x1874669228035968/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827079 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827063.029615:0:2378:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827063.029618:0:2378:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9856140db480 x1874669228035968/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827079 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827063.029625:0:2378:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1787827063.029626:0:2378:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000200:3.0:1787827063.030053:0:2385:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1874669228035840, offset 0 00000400:00000200:3.0:1787827063.030060:0:2385:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.203.114@tcp 00000400:00000200:3.0:1787827063.030063:0:2385:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:3.0:1787827063.030066:0:2385:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.203.114@tcp local destination 00000400:00000200:3.0:1787827063.030068:0:2385:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:3.0:1787827063.030070:0:2385:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.203.14@tcp 00000400:00000200:3.0:1787827063.030072:0:2385:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.203.14@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1787827063.030073:0:2385:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.203.114@tcp 00000400:00000200:3.0:1787827063.030075:0:2385:0:(lib-move.c:1745:lnet_handle_send()) 192.168.203.14@tcp NI seq info: [21919:21919:256:4294967295] 192.168.203.114@tcp LPNI seq info [21919:21919:8:4294967295] 00000400:00000200:3.0:1787827063.030078:0:2385:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:3.0:1787827063.030082:0:2385:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp:) -> 192.168.203.114@tcp(192.168.203.114@tcp:192.168.203.114@tcp) : PUT try# 0 00000800:00000200:3.0:1787827063.030085:0:2385:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.203.114@tcp 00000800:00000010:3.0:1787827063.030088:0:2385:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff985609025f00. 00000800:00000200:3.0:1787827063.030090:0:2385:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff985606751480] -> 12345-192.168.203.114@tcp (5) 00000800:00000200:3.0:1787827063.030100:0:2385:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.203.114@tcp ip 192.168.203.114:988 00000800:00000200:3.0:1787827063.030104:0:2385:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff985609025f00 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1787827063.030116:0:2385:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1787827063.030120:0:2385:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000001:3.0:1787827063.030122:0:2385:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.203.114@tcp: PM QoS new deadline estimation for cpu 2 is 4297891069 00000100:00000001:3.0:1787827063.030128:0:2385:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.203.114@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4297885948 00000100:00000001:3.0:1787827063.030132:0:2385:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.203.114@tcp: PM QoS new deadline estimation for cpu 3 is 4297891069 00000100:00000001:3.0:1787827063.030135:0:2385:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.203.114@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4297885948 00000100:00000001:3.0:1787827063.030140:0:2385:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1787827063.030142:0:2385:0:(client.c:1941:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827063.030144:0:2385:0:(client.c:1813:ptlrpc_send_new_req()) Process entered 00000100:00000040:3.0:1787827063.030150:0:2385:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9856140da940 x1874669228036096/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827063.030158:0:2385:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1787827063.030160:0:2385:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1787827063.030163:0:2385:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1787827063.030165:0:2385:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1787827063.030167:0:2385:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1787827063.030170:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669228036096:192.168.203.114@tcp:400:kworker.0 00000100:00000001:3.0:1787827063.030173:0:2385:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:3.0:1787827063.030176:0:2385:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1787827063.030177:0:2385:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1787827063.030179:0:2385:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:3.0:1787827063.030183:0:2385:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff985611860200. 02000000:00000001:3.0:1787827063.030184:0:2385:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1787827063.030188:0:2385:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000010:3.0:1787827063.030190:0:2385:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff985602ad92c0. 00000400:00000010:3.0:1787827063.030194:0:2385:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985609573198. 00000100:00000200:3.0:1787827063.030197:0:2385:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1874669228036096, portal 10 00000100:00000001:3.0:1787827063.030199:0:2385:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1787827063.030200:0:2385:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446630094203889984 : -113979505661632 : ffff9856140da940) 00000100:00000040:3.0:1787827063.030206:0:2385:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9856140da940 x1874669228036096/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827079 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827063.030213:0:2385:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1787827063.030215:0:2385:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.203.114@tcp 00000400:00000010:3.0:1787827063.030217:0:2385:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985609573b28. 00000100:00000200:3.0:1787827063.030220:0:2385:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1874669228036096, offset 0 00000400:00000200:3.0:1787827063.030231:0:2385:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.203.114@tcp 00000400:00000200:3.0:1787827063.030235:0:2385:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:3.0:1787827063.030237:0:2385:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.203.114@tcp local destination 00000400:00000200:3.0:1787827063.030239:0:2385:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:3.0:1787827063.030240:0:2385:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.203.14@tcp 00000400:00000200:3.0:1787827063.030242:0:2385:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.203.14@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1787827063.030243:0:2385:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.203.114@tcp 00000400:00000200:3.0:1787827063.030245:0:2385:0:(lib-move.c:1745:lnet_handle_send()) 192.168.203.14@tcp NI seq info: [21920:21920:255:4294967295] 192.168.203.114@tcp LPNI seq info [21920:21920:7:4294967295] 00000400:00000200:3.0:1787827063.030248:0:2385:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:3.0:1787827063.030252:0:2385:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp:) -> 192.168.203.114@tcp(192.168.203.114@tcp:192.168.203.114@tcp) : PUT try# 0 00000800:00000200:3.0:1787827063.030255:0:2385:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.203.114@tcp 00000800:00000010:3.0:1787827063.030258:0:2385:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff985609025a00. 00000800:00000200:3.0:1787827063.030260:0:2385:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff985606751480] -> 12345-192.168.203.114@tcp (5) 00000800:00000200:3.0:1787827063.030263:0:2385:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.203.114@tcp ip 192.168.203.114:988 00000800:00000200:3.0:1787827063.030265:0:2385:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff985609025a00 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1787827063.030267:0:2385:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1787827063.030269:0:2385:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff985602bc9800: using a timeout of 11895 usecs (12 jiffies) 00000400:00000200:3.0:1787827063.030271:0:2385:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000001:3.0:1787827063.030272:0:2385:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff985602bc9800: PM QoS new deadline estimation for cpu 2 is 4297881101 00000100:00000001:3.0:1787827063.030274:0:2385:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff985602bc9800: PM QoS request active for cpu 2, keep current deadline 4297891069 00000100:00000001:3.0:1787827063.030275:0:2385:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff985602bc9800: PM QoS new deadline estimation for cpu 3 is 4297881101 00000100:00000001:3.0:1787827063.030276:0:2385:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff985602bc9800: PM QoS request active for cpu 3, keep current deadline 4297891069 00000100:00000001:3.0:1787827063.030278:0:2385:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1787827063.030279:0:2385:0:(client.c:1941:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827063.030280:0:2385:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827063.030281:0:2385:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1787827063.030285:0:2385:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1787827063.030286:0:2385:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1787827063.030287:0:2385:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1787827063.030288:0:2385:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1787827063.030289:0:2385:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827063.030290:0:2385:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1787827063.030334:0:2377:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1787827063.030388:0:2377:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff985609025f00. 00000400:00000200:2.0:1787827063.030393:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1787827063.030397:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000400:00000200:2.0:1787827063.030400:0:2377:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985609573770 00000400:00000010:2.0:1787827063.030402:0:2377:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985609573770. 00000100:00000001:2.0:1787827063.030405:0:2377:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1787827063.030409:0:2377:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff9856140d8f00 x1874669228035840/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827079 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827063.030416:0:2377:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827063.030419:0:2377:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9856140d8f00 x1874669228035840/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827079 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827063.030426:0:2377:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1787827063.030427:0:2377:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:2.0:1787827063.030585:0:2377:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1787827063.030588:0:2377:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff985609025a00. 00000400:00000200:2.0:1787827063.030592:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1787827063.030597:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000400:00000200:2.0:1787827063.030600:0:2377:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985609573b28 00000400:00000010:2.0:1787827063.030602:0:2377:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985609573b28. 00000100:00000001:2.0:1787827063.030605:0:2377:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1787827063.030609:0:2377:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff9856140da940 x1874669228036096/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827079 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827063.030616:0:2377:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827063.030618:0:2377:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9856140da940 x1874669228036096/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827079 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827063.030625:0:2377:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1787827063.030627:0:2377:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1787827063.031809:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827063.031819:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1787827063.031822:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827063.031825:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827063.031831:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827063.031836:0:2378:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp) <- 192.168.203.114@tcp : PUT - for me 00000400:00000200:2.0:1787827063.031842:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827063.031845:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827063.031850:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827063.031853:0:2378:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.203.114@tcp of length 224 into portal 10 MB=0x6a90071688f80 00000400:00000200:2.0:1787827063.031857:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827063.031862:0:2378:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.203.114@tcp of length 224/224 into md 0x32435 [1] + 224 00000800:00000001:2.0:1787827063.031868:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827063.031872:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827063.031874:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1787827063.031878:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000100:00000001:2.0:1787827063.031882:0:2378:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1787827063.031887:0:2378:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff9856140db480 x1874669228035968/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827079 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1787827063.031897:0:2378:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9856140db480 x1874669228035968/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827079 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827063.031914:0:2378:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1787827063.031920:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827063.031924:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1787827063.031926:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827063.031928:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827063.031931:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827063.031934:0:2378:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp) <- 192.168.203.114@tcp : PUT - for me 00000400:00000200:2.0:1787827063.031938:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827063.031941:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827063.031944:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827063.031947:0:2378:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.203.114@tcp of length 224 into portal 25 MB=0x6a90071688f00 00000400:00000200:2.0:1787827063.031950:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827063.031953:0:2378:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.203.114@tcp of length 224/224 into md 0x3242d [1] + 224 00000800:00000001:2.0:1787827063.031957:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827063.031960:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827063.031962:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1787827063.031965:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000100:00000001:2.0:1787827063.031968:0:2378:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1787827063.031971:0:2378:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff9856140d8f00 x1874669228035840/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827079 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1787827063.031979:0:2378:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9856140d8f00 x1874669228035840/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827079 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827063.031989:0:2378:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1787827063.031992:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827063.031995:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1787827063.031998:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827063.032000:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827063.032002:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827063.032006:0:2378:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp) <- 192.168.203.114@tcp : PUT - for me 00000400:00000200:2.0:1787827063.032009:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827063.032012:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827063.032014:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827063.032017:0:2378:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.203.114@tcp of length 224 into portal 10 MB=0x6a90071689000 00000400:00000200:2.0:1787827063.032020:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827063.032023:0:2378:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.203.114@tcp of length 224/224 into md 0x3244d [1] + 224 00000800:00000001:2.0:1787827063.032026:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827063.032031:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827063.032032:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1787827063.032035:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000100:00000001:2.0:1787827063.032038:0:2378:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1787827063.032041:0:2378:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff9856140da940 x1874669228036096/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827079 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1787827063.032049:0:2378:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9856140da940 x1874669228036096/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827079 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827063.032055:0:2378:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1787827063.032059:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827063.032062:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1787827063.032151:0:2386:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1787827063.032154:0:2386:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000400:00000001:3.0:1787827063.032159:0:2386:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1787827063.032161:0:2386:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1787827063.032164:0:2386:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff98561080eb00. 00000400:00000200:3.0:1787827063.032167:0:2386:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985610afa3b8 00000400:00000010:3.0:1787827063.032169:0:2386:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985610afa3b8. 00000100:00000001:3.0:1787827063.032172:0:2386:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1787827063.032236:0:2386:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff9856140db480 x1874669228035968/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827079 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1787827063.032253:0:2386:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff9856140db480 x1874669228035968/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827079 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827063.032259:0:2386:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1787827063.032261:0:2386:0:(client.c:3001:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1787827063.032264:0:2386:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:3.0:1787827063.032266:0:2386:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1787827063.032268:0:2386:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1787827063.032271:0:2386:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1787827063.032273:0:2386:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1787827063.032281:0:2386:0:(import.c:2149:obd_at_measure()) add 5 to ffff985611182460 time=141 v=5 (5 5 5 5) 00000100:00001000:3.0:1787827063.032285:0:2386:0:(import.c:2149:obd_at_measure()) add 1 to ffff985611182418 time=77 v=5 (1 0 0 0) 00000100:00000001:3.0:1787827063.032288:0:2386:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1787827063.032290:0:2386:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1787827063.032292:0:2386:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1787827063.032295:0:2386:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827063.032298:0:2386:0:(client.c:3109:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1787827063.032301:0:2386:0:(client.c:3121:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: skip recheck: last_committed 219043334508 00000100:00000001:3.0:1787827063.032303:0:2386:0:(client.c:3122:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1787827063.032305:0:2386:0:(client.c:1797:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1787827063.032309:0:2386:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9856140db480 x1874669228035968/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827079 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827063.032315:0:2386:0:(client.c:2354:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1787827063.032318:0:2386:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1787827063.032319:0:2386:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1787827063.032323:0:2386:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9856140db480 x1874669228035968/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827079 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827063.032330:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228035968:192.168.203.114@tcp:400:kworker.0 00000100:00000001:3.0:1787827063.032335:0:2386:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1787827063.032336:0:2386:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1787827063.032339:0:2386:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9856140db480 x1874669228035968/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827079 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827063.032346:0:2386:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1787827063.032348:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1787827063.032350:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff985607e33c00. 02000000:00000001:3.0:1787827063.032353:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1787827063.032355:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:3.0:1787827063.032357:0:2386:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=3 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:3.0:1787827063.032359:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:3.0:1787827063.032361:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff985615779800. 02000000:00000001:3.0:1787827063.032365:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1787827063.032367:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1787827063.032368:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff9856140db480. 00000100:00000001:3.0:1787827063.032371:0:2386:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1787827063.032372:0:2386:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1787827063.032374:0:2386:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1787827063.032379:0:2386:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1787827063.032381:0:2386:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827063.032383:0:2386:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1787827063.032385:0:2386:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827063.032392:0:2385:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1787827063.032394:0:2385:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000400:00000001:3.0:1787827063.032397:0:2385:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1787827063.032399:0:2385:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1787827063.032400:0:2385:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff985602ad93c8. 00000400:00000200:3.0:1787827063.032404:0:2385:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9856095733b8 00000400:00000010:3.0:1787827063.032406:0:2385:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9856095733b8. 00000100:00000001:3.0:1787827063.032408:0:2385:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1787827063.032412:0:2385:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff9856140d8f00 x1874669228035840/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827079 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1787827063.032421:0:2385:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff9856140d8f00 x1874669228035840/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827079 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827063.032427:0:2385:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1787827063.032428:0:2385:0:(client.c:3001:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1787827063.032430:0:2385:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:3.0:1787827063.032431:0:2385:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1787827063.032432:0:2385:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1787827063.032434:0:2385:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1787827063.032436:0:2385:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1787827063.032438:0:2385:0:(import.c:2149:obd_at_measure()) add 5 to ffff98561118e460 time=63 v=5 (5 5 5 5) 00000100:00001000:3.0:1787827063.032441:0:2385:0:(import.c:2149:obd_at_measure()) add 1 to ffff98561118e418 time=78 v=5 (1 0 0 0) 00000100:00000001:3.0:1787827063.032443:0:2385:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1787827063.032444:0:2385:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1787827063.032446:0:2385:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1787827063.032447:0:2385:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827063.032449:0:2385:0:(client.c:1797:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1787827063.032453:0:2385:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9856140d8f00 x1874669228035840/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827079 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827063.032459:0:2385:0:(client.c:2354:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1787827063.032462:0:2385:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1787827063.032463:0:2385:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1787827063.032467:0:2385:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9856140d8f00 x1874669228035840/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827079 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827063.032474:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2385:1874669228035840:192.168.203.114@tcp:400:kworker.0 00000400:00000001:3.0:1787827063.032479:0:2385:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1787827063.032480:0:2385:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1787827063.032482:0:2385:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff985602ad92c0. 00000400:00000200:3.0:1787827063.032484:0:2385:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985609573198 00000400:00000010:3.0:1787827063.032486:0:2385:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985609573198. 00000100:00000001:3.0:1787827063.032488:0:2385:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1787827063.032491:0:2385:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff9856140da940 x1874669228036096/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827079 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1787827063.032499:0:2385:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff9856140da940 x1874669228036096/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827079 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827063.032505:0:2385:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1787827063.032506:0:2385:0:(client.c:3001:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1787827063.032508:0:2385:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:3.0:1787827063.032509:0:2385:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1787827063.032510:0:2385:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1787827063.032512:0:2385:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1787827063.032514:0:2385:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1787827063.032518:0:2385:0:(import.c:2149:obd_at_measure()) add 5 to ffff985611185c60 time=63 v=5 (5 5 5 5) 00000100:00001000:3.0:1787827063.032520:0:2385:0:(import.c:2149:obd_at_measure()) add 1 to ffff985611185c18 time=63 v=5 (1 1 1 1) 00000100:00000001:3.0:1787827063.032522:0:2385:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1787827063.032524:0:2385:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1787827063.032525:0:2385:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1787827063.032527:0:2385:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827063.032529:0:2385:0:(client.c:3109:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1787827063.032530:0:2385:0:(client.c:3121:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff985602bc9800: skip recheck: last_committed 0 00000100:00000001:3.0:1787827063.032541:0:2385:0:(client.c:3122:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1787827063.032543:0:2385:0:(client.c:1797:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1787827063.032555:0:2385:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9856140da940 x1874669228036096/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827079 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827063.032563:0:2385:0:(client.c:2354:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1787827063.032565:0:2385:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1787827063.032566:0:2385:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1787827063.032570:0:2385:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9856140da940 x1874669228036096/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827079 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827063.032578:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669228036096:192.168.203.114@tcp:400:kworker.0 00000100:00000001:3.0:1787827063.032582:0:2385:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1787827063.032583:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1787827063.032586:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9856140d8f00 x1874669228035840/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827079 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827063.032592:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1787827063.032593:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1787827063.032595:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff985611861e00. 02000000:00000001:3.0:1787827063.032598:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1787827063.032599:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:3.0:1787827063.032600:0:2385:0:(genops.c:1223:class_import_put()) import ffff98561118e000 refcount=3 obd=MGC192.168.203.114@tcp 00000020:00000001:3.0:1787827063.032602:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:3.0:1787827063.032604:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff985615779d00. 02000000:00000001:3.0:1787827063.032606:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1787827063.032608:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1787827063.032609:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff9856140d8f00. 00000100:00000001:3.0:1787827063.032612:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1787827063.032613:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1787827063.032615:0:2385:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1787827063.032618:0:2385:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9856140da940 x1874669228036096/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827079 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827063.032624:0:2385:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1787827063.032625:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1787827063.032626:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff985611860200. 02000000:00000001:3.0:1787827063.032628:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1787827063.032630:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:3.0:1787827063.032631:0:2385:0:(genops.c:1223:class_import_put()) import ffff985611185800 refcount=3 obd=lustre-MDT0001-mdc-ffff985602bc9800 00000020:00000001:3.0:1787827063.032632:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:3.0:1787827063.032634:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff985615779600. 02000000:00000001:3.0:1787827063.032635:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1787827063.032636:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1787827063.032638:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff9856140da940. 00000100:00000001:3.0:1787827063.032640:0:2385:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1787827063.032642:0:2385:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1787827063.032643:0:2385:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1787827063.032646:0:2385:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1787827063.032647:0:2385:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827063.032649:0:2385:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1787827063.032650:0:2385:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827063.604984:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827063.604994:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827063.605002:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827063.605003:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827063.605007:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827063.605008:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1787827063.605103:0:15163:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1787827063.605109:0:15163:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1787827063.605114:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:0.0:1787827064.628833:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827064.628838:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827064.628841:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827064.628842:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827064.628845:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827064.628845:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1787827064.628901:0:33341:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1787827064.628905:0:33341:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1787827064.628908:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:0.0:1787827065.652832:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827065.652839:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827065.652844:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827065.652845:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827065.652848:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827065.652849:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1787827065.652871:0:15164:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1787827065.652874:0:15164:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1787827065.652877:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:0.0:1787827066.676931:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827066.676937:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827066.676943:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827066.676946:0:49842:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:0.0:1787827066.676948:0:49842:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:0.0:1787827066.676951:0:49842:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827066.676954:0:49842:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:0.0:1787827066.676955:0:49842:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827066.676958:0:49842:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827066.676960:0:49842:0:(ldlm_pool.c:496:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827066.676964:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827066.676965:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1787827066.676992:0:33342:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1787827066.676995:0:33342:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1787827066.676998:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:0.0:1787827067.700809:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827067.700815:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827067.700821:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827067.700822:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827067.700826:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827067.700827:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1787827067.700930:0:79887:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1787827067.700934:0:79887:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1787827067.700948:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:1.0:1787827068.148828:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1787827068.148839:0:37:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1787827068.148849:0:37:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 904 at ffff985607e161c0. 00000020:00000040:1.0:1787827068.148855:0:37:0:(genops.c:1210:class_import_get()) import ffff98561118e000 refcount=4 obd=MGC192.168.203.114@tcp 00000100:00000001:1.0:1787827068.148859:0:37:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1787827068.148869:0:37:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1787827068.148875:0:37:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1787827068.148884:0:37:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff985607d95600. 00000100:00000001:1.0:1787827068.148890:0:37:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1787827068.148901:0:37:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS req@ffff985607e161c0 x1874669228036224/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1787827068.148912:0:37:0:(jobid.c:939:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1787827068.148916:0:37:0:(jobid.c:979:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1787827068.148922:0:37:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000c2ab42e7] to pc [ptlrpcd_00_00+0] req@ffff985607e161c0 x1874669228036224/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827068.148956:0:37:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1787827068.148960:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1787827068.148963:0:37:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1787827068.148966:0:37:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 904 at ffff985607e17c00. 00000020:00000040:1.0:1787827068.148969:0:37:0:(genops.c:1210:class_import_get()) import ffff985611182000 refcount=4 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000100:00000001:1.0:1787827068.148971:0:37:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1787827068.148972:0:37:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1787827068.148974:0:37:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1787827068.148978:0:37:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff985607d95d00. 00000100:00000001:1.0:1787827068.148981:0:37:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1787827068.148995:0:37:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID req@ffff985607e17c00 x1874669228036352/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1787827068.149001:0:37:0:(jobid.c:939:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1787827068.149002:0:37:0:(jobid.c:979:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1787827068.149005:0:37:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000846571c6] to pc [ptlrpcd_00_01+1] req@ffff985607e17c00 x1874669228036352/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827068.149011:0:37:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1787827068.149012:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1787827068.149014:0:37:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1787827068.149016:0:37:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 904 at ffff985607e17840. 00000020:00000040:1.0:1787827068.149018:0:37:0:(genops.c:1210:class_import_get()) import ffff985611185800 refcount=4 obd=lustre-MDT0001-mdc-ffff985602bc9800 00000100:00000001:1.0:1787827068.149020:0:37:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1787827068.149020:0:37:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1787827068.149022:0:37:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1787827068.149024:0:37:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff985607d95500. 00000100:00000001:1.0:1787827068.149026:0:37:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1787827068.149028:0:37:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID req@ffff985607e17840 x1874669228036480/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1787827068.149032:0:37:0:(jobid.c:939:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1787827068.149033:0:37:0:(jobid.c:979:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1787827068.149035:0:37:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000f55263cd] to pc [ptlrpcd_00_00+0] req@ffff985607e17840 x1874669228036480/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827068.149048:0:37:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1787827068.149050:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:1.0:1787827068.149053:0:37:0:(pinger.c:247:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000040:1.0:1787827068.149055:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:1.0:1787827068.149058:0:37:0:(pinger.c:247:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000040:1.0:1787827068.149062:0:37:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (3223) 00000100:00000001:1.0:1787827068.149074:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1787827068.149077:0:2384:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1787827068.149079:0:2384:0:(client.c:1813:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1787827068.149084:0:2384:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff985607e17c00 x1874669228036352/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827068.149099:0:2384:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1787827068.149102:0:2384:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1787827068.149105:0:2384:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1787827068.149108:0:2384:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1787827068.149111:0:2384:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1787827068.149114:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607e17c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669228036352:192.168.203.114@tcp:400:kworker.0 00000100:00000001:1.0:1787827068.149118:0:2384:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1787827068.149123:0:2384:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1787827068.149126:0:2384:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1787827068.149128:0:2384:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1787827068.149132:0:2384:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff985610db8c00. 02000000:00000001:1.0:1787827068.149134:0:2384:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1787827068.149140:0:2384:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000010:1.0:1787827068.149145:0:2384:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff985602ecf898. 00000400:00000010:1.0:1787827068.149151:0:2384:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985606618000. 00000100:00000200:1.0:1787827068.149157:0:2384:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1874669228036352, portal 10 00000100:00000001:1.0:1787827068.149160:0:2384:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1787827068.149161:0:2384:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446630093999668224 : -113979709883392 : ffff985607e17c00) 00000100:00000040:1.0:1787827068.149169:0:2384:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff985607e17c00 x1874669228036352/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827084 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827068.149175:0:2384:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1787827068.149177:0:2384:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.203.114@tcp 00000400:00000010:1.0:1787827068.149179:0:2384:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985606618d48. 00000100:00000200:1.0:1787827068.149182:0:2384:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1874669228036352, offset 0 00000400:00000200:1.0:1787827068.149187:0:2384:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.203.114@tcp 00000400:00000200:1.0:1787827068.149194:0:2384:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:1.0:1787827068.149199:0:2384:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.203.114@tcp local destination 00000400:00000200:1.0:1787827068.149203:0:2384:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1787827068.149206:0:2384:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.203.14@tcp 00000400:00000200:1.0:1787827068.149209:0:2384:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.203.14@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1787827068.149211:0:2384:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.203.114@tcp 00000400:00000200:1.0:1787827068.149214:0:2384:0:(lib-move.c:1745:lnet_handle_send()) 192.168.203.14@tcp NI seq info: [21921:21921:256:4294967295] 192.168.203.114@tcp LPNI seq info [21921:21921:8:4294967295] 00000400:00000200:1.0:1787827068.149217:0:2384:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:1.0:1787827068.149224:0:2384:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp:) -> 192.168.203.114@tcp(192.168.203.114@tcp:192.168.203.114@tcp) : PUT try# 0 00000800:00000200:1.0:1787827068.149228:0:2384:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.203.114@tcp 00000800:00000010:1.0:1787827068.149231:0:2384:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff985607d95300. 00000800:00000200:1.0:1787827068.149236:0:2384:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff985606751480] -> 12345-192.168.203.114@tcp (5) 00000800:00000200:1.0:1787827068.149242:0:2384:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.203.114@tcp ip 192.168.203.114:988 00000800:00000200:1.0:1787827068.149245:0:2384:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff985607d95300 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1787827068.149259:0:2384:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1787827068.149263:0:2384:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff985602bc9800: using a timeout of 38037 usecs (39 jiffies) 00000400:00000200:1.0:1787827068.149265:0:2384:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000001:1.0:1787827068.149267:0:2384:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff985602bc9800: PM QoS new deadline estimation for cpu 2 is 4297886247 00000100:00000001:1.0:1787827068.149269:0:2384:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff985602bc9800: PM QoS request active for cpu 2, keep current deadline 4297891069 00000100:00000001:1.0:1787827068.149271:0:2384:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff985602bc9800: PM QoS new deadline estimation for cpu 3 is 4297886247 00000100:00000001:1.0:1787827068.149272:0:2384:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff985602bc9800: PM QoS request active for cpu 3, keep current deadline 4297891069 00000100:00000001:1.0:1787827068.149274:0:2384:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1787827068.149277:0:2384:0:(client.c:1941:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827068.149279:0:2384:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827068.149281:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787827068.149287:0:2384:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1787827068.149289:0:2384:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1787827068.149290:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1787827068.149291:0:2384:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1787827068.149293:0:2384:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1787827068.149295:0:2384:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00000001:1.0:1787827068.149296:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787827068.149298:0:2384:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1787827068.149300:0:2384:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787827068.149301:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1787827068.149303:0:2384:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1787827068.149304:0:2384:0:(client.c:1813:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1787827068.149308:0:2384:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff985607e161c0 x1874669228036224/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827068.149314:0:2384:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1787827068.149316:0:2384:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1787827068.149318:0:2384:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1787827068.149320:0:2384:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1787827068.149322:0:2384:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1787827068.149325:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607e161c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:3c946890-15ca-497a-b838-fe1cba61f4dd:2384:1874669228036224:192.168.203.114@tcp:400:kworker.0 00000100:00000001:1.0:1787827068.149328:0:2384:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1787827068.149331:0:2384:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1787827068.149333:0:2384:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1787827068.149335:0:2384:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1787827068.149338:0:2384:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff985610db9a00. 02000000:00000001:1.0:1787827068.149340:0:2384:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1787827068.149350:0:2384:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000010:1.0:1787827068.149353:0:2384:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff985602ecf9f8. 00000400:00000010:1.0:1787827068.149357:0:2384:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9856066183b8. 00000100:00000200:1.0:1787827068.149360:0:2384:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1874669228036224, portal 25 00000100:00000001:1.0:1787827068.149362:0:2384:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1787827068.149363:0:2384:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446630093999661504 : -113979709890112 : ffff985607e161c0) 00000100:00000040:1.0:1787827068.149368:0:2384:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff985607e161c0 x1874669228036224/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827084 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827068.149374:0:2384:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1787827068.149377:0:2384:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.203.114@tcp 00000800:00000200:2.0:1787827068.149379:0:2377:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000400:00000010:1.0:1787827068.149380:0:2384:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985606618770. 00000100:00000200:1.0:1787827068.149386:0:2384:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1874669228036224, offset 0 00000800:00000010:2.0:1787827068.149387:0:2377:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff985607d95300. 00000400:00000200:1.0:1787827068.149392:0:2384:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.203.114@tcp 00000400:00000200:2.0:1787827068.149394:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1787827068.149398:0:2384:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827068.149403:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000400:00000200:1.0:1787827068.149407:0:2384:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.203.114@tcp local destination 00000400:00000200:2.0:1787827068.149411:0:2377:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985606618d48 00000400:00000010:2.0:1787827068.149413:0:2377:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985606618d48. 00000100:00000001:2.0:1787827068.149418:0:2377:0:(events.c:34:request_out_callback()) Process entered 00000400:00000200:1.0:1787827068.149419:0:2384:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1787827068.149423:0:2384:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.203.14@tcp 00000400:00000200:1.0:1787827068.149427:0:2384:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.203.14@tcp lpni_is_preferred = 0 00000100:00000200:2.0:1787827068.149427:0:2377:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff985607e17c00 x1874669228036352/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827084 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1787827068.149432:0:2384:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.203.114@tcp 00000400:00000200:1.0:1787827068.149438:0:2384:0:(lib-move.c:1745:lnet_handle_send()) 192.168.203.14@tcp NI seq info: [21922:21922:255:4294967295] 192.168.203.114@tcp LPNI seq info [21922:21922:7:4294967295] 00000400:00000200:1.0:1787827068.149448:0:2384:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000001:2.0:1787827068.149450:0:2377:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1787827068.149458:0:2377:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff985607e17c00 x1874669228036352/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827084 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1787827068.149459:0:2384:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp:) -> 192.168.203.114@tcp(192.168.203.114@tcp:192.168.203.114@tcp) : PUT try# 0 00000800:00000200:1.0:1787827068.149471:0:2384:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.203.114@tcp 00000100:00000001:2.0:1787827068.149474:0:2377:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:1.0:1787827068.149478:0:2384:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff985607d95a00. 00000100:00000001:2.0:1787827068.149478:0:2377:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:1.0:1787827068.149485:0:2384:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff985606751480] -> 12345-192.168.203.114@tcp (5) 00000800:00000200:1.0:1787827068.149491:0:2384:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.203.114@tcp ip 192.168.203.114:988 00000800:00000200:1.0:1787827068.149494:0:2384:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff985607d95a00 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1787827068.149505:0:2384:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1787827068.149508:0:2384:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000001:1.0:1787827068.149510:0:2384:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.203.114@tcp: PM QoS new deadline estimation for cpu 2 is 4297896188 00000100:00000001:1.0:1787827068.149527:0:2384:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.203.114@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4297891069 00000100:00000001:1.0:1787827068.149531:0:2384:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.203.114@tcp: PM QoS new deadline estimation for cpu 3 is 4297896188 00000100:00000001:1.0:1787827068.149537:0:2384:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.203.114@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4297891069 00000100:00000001:1.0:1787827068.149540:0:2384:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1787827068.149542:0:2384:0:(client.c:1941:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827068.149544:0:2384:0:(client.c:1813:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1787827068.149548:0:2384:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff985607e17840 x1874669228036480/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827068.149555:0:2384:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1787827068.149557:0:2384:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1787827068.149563:0:2384:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1787827068.149565:0:2384:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1787827068.149567:0:2384:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1787827068.149570:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607e17840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669228036480:192.168.203.114@tcp:400:kworker.0 00000100:00000001:1.0:1787827068.149573:0:2384:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1787827068.149576:0:2384:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1787827068.149577:0:2384:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1787827068.149578:0:2384:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1787827068.149582:0:2384:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff985610db9400. 02000000:00000001:1.0:1787827068.149583:0:2384:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1787827068.149587:0:2384:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000010:1.0:1787827068.149590:0:2384:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff985602ecfa50. 00000400:00000010:1.0:1787827068.149605:0:2384:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985606618908. 00000100:00000200:1.0:1787827068.149609:0:2384:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1874669228036480, portal 10 00000100:00000001:1.0:1787827068.149611:0:2384:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1787827068.149612:0:2384:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446630093999667264 : -113979709884352 : ffff985607e17840) 00000100:00000040:1.0:1787827068.149618:0:2384:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff985607e17840 x1874669228036480/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827084 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827068.149624:0:2384:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1787827068.149626:0:2384:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.203.114@tcp 00000400:00000010:1.0:1787827068.149628:0:2384:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985606618330. 00000100:00000200:1.0:1787827068.149631:0:2384:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1874669228036480, offset 0 00000400:00000200:1.0:1787827068.149635:0:2384:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.203.114@tcp 00000400:00000200:1.0:1787827068.149638:0:2384:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:1.0:1787827068.149659:0:2384:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.203.114@tcp local destination 00000400:00000200:1.0:1787827068.149662:0:2384:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1787827068.149665:0:2384:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.203.14@tcp 00000400:00000200:1.0:1787827068.149668:0:2384:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.203.14@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1787827068.149672:0:2384:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.203.114@tcp 00000400:00000200:1.0:1787827068.149676:0:2384:0:(lib-move.c:1745:lnet_handle_send()) 192.168.203.14@tcp NI seq info: [21923:21923:255:4294967295] 192.168.203.114@tcp LPNI seq info [21923:21923:7:4294967295] 00000400:00000200:1.0:1787827068.149681:0:2384:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:1.0:1787827068.149687:0:2384:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp:) -> 192.168.203.114@tcp(192.168.203.114@tcp:192.168.203.114@tcp) : PUT try# 0 00000800:00000200:1.0:1787827068.149737:0:2384:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.203.114@tcp 00000800:00000010:1.0:1787827068.149742:0:2384:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff985607d95200. 00000800:00000200:1.0:1787827068.149749:0:2384:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff985606751480] -> 12345-192.168.203.114@tcp (5) 00000800:00000200:1.0:1787827068.149754:0:2384:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.203.114@tcp ip 192.168.203.114:988 00000800:00000200:1.0:1787827068.149757:0:2384:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff985607d95200 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1787827068.149760:0:2384:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1787827068.149763:0:2384:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff985602bc9800: using a timeout of 11882 usecs (12 jiffies) 00000400:00000200:1.0:1787827068.149767:0:2384:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000001:1.0:1787827068.149769:0:2384:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff985602bc9800: PM QoS new deadline estimation for cpu 2 is 4297886221 00000100:00000001:1.0:1787827068.149772:0:2384:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff985602bc9800: PM QoS request active for cpu 2, keep current deadline 4297896188 00000100:00000001:1.0:1787827068.149774:0:2384:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff985602bc9800: PM QoS new deadline estimation for cpu 3 is 4297886221 00000100:00000001:1.0:1787827068.149778:0:2384:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff985602bc9800: PM QoS request active for cpu 3, keep current deadline 4297896188 00000800:00000200:3.0:1787827068.149780:0:2378:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:1.0:1787827068.149782:0:2384:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1787827068.149790:0:2384:0:(client.c:1941:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:3.0:1787827068.149791:0:2378:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff985607d95a00. 00000400:00000200:3.0:1787827068.149800:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:1.0:1787827068.149801:0:2384:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827068.149811:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1787827068.149817:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000400:00000200:3.0:1787827068.149825:0:2378:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985606618770 00000400:00000010:3.0:1787827068.149828:0:2378:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985606618770. 00000100:00000001:3.0:1787827068.149835:0:2378:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1787827068.149844:0:2378:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff985607e161c0 x1874669228036224/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827084 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827068.149856:0:2378:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1787827068.149860:0:2378:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff985607e161c0 x1874669228036224/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827084 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827068.149866:0:2378:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827068.149868:0:2378:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1787827068.150321:0:2378:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1787827068.150324:0:2378:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff985607d95200. 00000400:00000200:3.0:1787827068.150328:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1787827068.150333:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000400:00000200:3.0:1787827068.150337:0:2378:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985606618330 00000400:00000010:3.0:1787827068.150340:0:2378:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985606618330. 00000100:00000001:3.0:1787827068.150343:0:2378:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1787827068.150347:0:2378:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff985607e17840 x1874669228036480/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827084 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827068.150357:0:2378:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1787827068.150360:0:2378:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff985607e17840 x1874669228036480/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827084 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827068.150367:0:2378:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827068.150369:0:2378:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:0.0:1787827068.150757:0:2383:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1787827068.150763:0:2383:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1787827068.150839:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827068.150848:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1787827068.150850:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827068.150852:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827068.150856:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827068.150867:0:2377:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp) <- 192.168.203.114@tcp : PUT - for me 00000400:00000200:2.0:1787827068.150875:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827068.150879:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827068.150883:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827068.150886:0:2377:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.203.114@tcp of length 224 into portal 10 MB=0x6a90071689100 00000400:00000200:2.0:1787827068.150891:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827068.150896:0:2377:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.203.114@tcp of length 224/224 into md 0x3245d [1] + 224 00000800:00000001:2.0:1787827068.150902:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827068.150906:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827068.150908:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1787827068.150912:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000100:00000001:2.0:1787827068.150917:0:2377:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1787827068.150922:0:2377:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff985607e17c00 x1874669228036352/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827084 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1787827068.150929:0:2377:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff985607e17c00 x1874669228036352/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827084 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827068.150943:0:2377:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1787827068.150950:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827068.150957:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1787827068.150959:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827068.150962:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827068.150965:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827068.150969:0:2377:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp) <- 192.168.203.114@tcp : PUT - for me 00000400:00000200:2.0:1787827068.150975:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827068.150978:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827068.150982:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000001:1.0:1787827068.150984:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:2.0:1787827068.150986:0:2377:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.203.114@tcp of length 224 into portal 25 MB=0x6a90071689080 00000100:00000001:1.0:1787827068.150988:0:2384:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000400:00000200:2.0:1787827068.150992:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000001:1.0:1787827068.150995:0:2384:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1787827068.150998:0:2384:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1787827068.151001:0:2384:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff985602ecf898. 00000400:00000200:1.0:1787827068.151005:0:2384:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985606618000 00000400:00000010:1.0:1787827068.151016:0:2384:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985606618000. 00000100:00000001:1.0:1787827068.151019:0:2384:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000200:2.0:1787827068.151025:0:2377:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.203.114@tcp of length 224/224 into md 0x32465 [1] + 224 00000800:00000001:2.0:1787827068.151032:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000200:1.0:1787827068.151037:0:2384:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff985607e17c00 x1874669228036352/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827084 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1787827068.151040:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827068.151045:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1787827068.151053:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000100:00000200:1.0:1787827068.151057:0:2384:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff985607e17c00 x1874669228036352/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827084 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827068.151057:0:2377:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1787827068.151064:0:2377:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff985607e161c0 x1874669228036224/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827084 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827068.151080:0:2384:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1787827068.151083:0:2384:0:(client.c:3001:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787827068.151086:0:2384:0:(client.c:1593:after_reply()) Process entered 00000100:00000040:2.0:1787827068.151088:0:2377:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff985607e161c0 x1874669228036224/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827084 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1787827068.151090:0:2384:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1787827068.151093:0:2384:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1787827068.151094:0:2384:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1787827068.151096:0:2384:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1787827068.151103:0:2384:0:(import.c:2149:obd_at_measure()) add 5 to ffff985611182460 time=146 v=5 (5 5 5 5) 00000100:00000001:2.0:1787827068.151106:0:2377:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00001000:1.0:1787827068.151107:0:2384:0:(import.c:2149:obd_at_measure()) add 1 to ffff985611182418 time=82 v=5 (1 0 0 0) 00000100:00000001:1.0:1787827068.151111:0:2384:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1787827068.151112:0:2384:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1787827068.151114:0:2384:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000800:00000001:2.0:1787827068.151115:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00010000:00000001:1.0:1787827068.151117:0:2384:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827068.151121:0:2384:0:(client.c:3109:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1787827068.151123:0:2384:0:(client.c:3121:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: skip recheck: last_committed 219043334508 00000800:00000001:2.0:1787827068.151123:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787827068.151126:0:2384:0:(client.c:3122:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1787827068.151128:0:2384:0:(client.c:1797:after_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1787827068.151128:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827068.151134:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1787827068.151135:0:2384:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff985607e17c00 x1874669228036352/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827084 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1787827068.151143:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000001:1.0:1787827068.151147:0:2384:0:(client.c:2354:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1787827068.151153:0:2384:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000400:00000200:2.0:1787827068.151153:0:2377:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp) <- 192.168.203.114@tcp : PUT - for me 00000100:00000001:1.0:1787827068.151157:0:2384:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827068.151161:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000040:1.0:1787827068.151165:0:2384:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff985607e17c00 x1874669228036352/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827084 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1787827068.151168:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827068.151175:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827068.151181:0:2377:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.203.114@tcp of length 224 into portal 10 MB=0x6a90071689180 00000100:00100000:1.0:1787827068.151184:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607e17c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669228036352:192.168.203.114@tcp:400:kworker.0 00000400:00000200:2.0:1787827068.151185:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827068.151192:0:2377:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.203.114@tcp of length 224/224 into md 0x3246d [1] + 224 00000100:00000001:1.0:1787827068.151194:0:2384:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1787827068.151196:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1787827068.151197:0:2384:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1787827068.151202:0:2384:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607e17c00 x1874669228036352/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827084 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1787827068.151204:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827068.151210:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:1.0:1787827068.151211:0:2384:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1787827068.151215:0:2384:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 00000400:00000200:2.0:1787827068.151218:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 02000000:00000010:1.0:1787827068.151219:0:2384:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff985610db8c00. 00000100:00000001:2.0:1787827068.151226:0:2377:0:(events.c:78:reply_in_callback()) Process entered 02000000:00000001:1.0:1787827068.151228:0:2384:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000200:2.0:1787827068.151233:0:2377:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff985607e17840 x1874669228036480/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827084 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000020:00000001:1.0:1787827068.151238:0:2384:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:1.0:1787827068.151241:0:2384:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=3 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:1.0:1787827068.151247:0:2384:0:(genops.c:1230:class_import_put()) Process leaving 00000100:00000040:2.0:1787827068.151250:0:2377:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff985607e17840 x1874669228036480/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827084 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 02000000:00000010:1.0:1787827068.151251:0:2384:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff985607d95d00. 02000000:00000001:1.0:1787827068.151257:0:2384:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1787827068.151259:0:2384:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1787827068.151263:0:2384:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607e17c00. 00000100:00000001:2.0:1787827068.151263:0:2377:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1787827068.151269:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1787827068.151271:0:2384:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000800:00000001:2.0:1787827068.151275:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787827068.151276:0:2384:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787827068.151281:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827068.151283:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1787827068.151285:0:2384:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1787827068.151287:0:2384:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1787827068.151289:0:2384:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1787827068.151291:0:2384:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff985602ecf9f8. 00000400:00000200:1.0:1787827068.151294:0:2384:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9856066183b8 00000400:00000010:1.0:1787827068.151296:0:2384:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9856066183b8. 00000100:00000001:1.0:1787827068.151298:0:2384:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1787827068.151302:0:2384:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff985607e161c0 x1874669228036224/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827084 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1787827068.151310:0:2384:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff985607e161c0 x1874669228036224/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827084 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827068.151316:0:2384:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1787827068.151317:0:2384:0:(client.c:3001:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787827068.151319:0:2384:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:1.0:1787827068.151320:0:2384:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1787827068.151321:0:2384:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1787827068.151323:0:2384:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1787827068.151324:0:2384:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1787827068.151327:0:2384:0:(import.c:2149:obd_at_measure()) add 5 to ffff98561118e460 time=68 v=5 (5 5 5 5) 00000100:00001000:1.0:1787827068.151330:0:2384:0:(import.c:2149:obd_at_measure()) add 1 to ffff98561118e418 time=83 v=5 (1 0 0 0) 00000100:00000001:1.0:1787827068.151332:0:2384:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1787827068.151333:0:2384:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1787827068.151335:0:2384:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1787827068.151337:0:2384:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827068.151339:0:2384:0:(client.c:1797:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1787827068.151343:0:2384:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff985607e161c0 x1874669228036224/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827084 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827068.151359:0:2384:0:(client.c:2354:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1787827068.151362:0:2384:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1787827068.151363:0:2384:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1787827068.151366:0:2384:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff985607e161c0 x1874669228036224/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827084 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827068.151374:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607e161c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:3c946890-15ca-497a-b838-fe1cba61f4dd:2384:1874669228036224:192.168.203.114@tcp:400:kworker.0 00000400:00000001:1.0:1787827068.151379:0:2384:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1787827068.151381:0:2384:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1787827068.151383:0:2384:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff985602ecfa50. 00000400:00000200:1.0:1787827068.151385:0:2384:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985606618908 00000400:00000010:1.0:1787827068.151387:0:2384:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985606618908. 00000100:00000001:1.0:1787827068.151389:0:2384:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1787827068.151392:0:2384:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff985607e17840 x1874669228036480/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827084 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1787827068.151397:0:2384:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff985607e17840 x1874669228036480/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827084 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827068.151401:0:2384:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1787827068.151402:0:2384:0:(client.c:3001:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787827068.151403:0:2384:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:1.0:1787827068.151403:0:2384:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1787827068.151404:0:2384:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1787827068.151405:0:2384:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1787827068.151406:0:2384:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1787827068.151409:0:2384:0:(import.c:2149:obd_at_measure()) add 5 to ffff985611185c60 time=68 v=5 (5 5 5 5) 00000100:00001000:1.0:1787827068.151410:0:2384:0:(import.c:2149:obd_at_measure()) add 1 to ffff985611185c18 time=68 v=5 (1 1 1 1) 00000100:00000001:1.0:1787827068.151412:0:2384:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1787827068.151412:0:2384:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1787827068.151413:0:2384:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1787827068.151415:0:2384:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827068.151417:0:2384:0:(client.c:3109:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1787827068.151418:0:2384:0:(client.c:3121:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff985602bc9800: skip recheck: last_committed 0 00000100:00000001:1.0:1787827068.151419:0:2384:0:(client.c:3122:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1787827068.151420:0:2384:0:(client.c:1797:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1787827068.151422:0:2384:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff985607e17840 x1874669228036480/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827084 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827068.151425:0:2384:0:(client.c:2354:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1787827068.151426:0:2384:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1787827068.151427:0:2384:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1787827068.151429:0:2384:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff985607e17840 x1874669228036480/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827084 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827068.151434:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607e17840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669228036480:192.168.203.114@tcp:400:kworker.0 00000100:00000001:1.0:1787827068.151436:0:2384:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787827068.151437:0:2384:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1787827068.151439:0:2384:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607e161c0 x1874669228036224/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827084 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827068.151442:0:2384:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1787827068.151443:0:2384:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1787827068.151444:0:2384:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff985610db9a00. 02000000:00000001:1.0:1787827068.151446:0:2384:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1787827068.151447:0:2384:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:1.0:1787827068.151449:0:2384:0:(genops.c:1223:class_import_put()) import ffff98561118e000 refcount=3 obd=MGC192.168.203.114@tcp 00000020:00000001:1.0:1787827068.151450:0:2384:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:1.0:1787827068.151452:0:2384:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff985607d95600. 02000000:00000001:1.0:1787827068.151454:0:2384:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1787827068.151455:0:2384:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1787827068.151456:0:2384:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607e161c0. 00000100:00000001:1.0:1787827068.151459:0:2384:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1787827068.151460:0:2384:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787827068.151461:0:2384:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1787827068.151464:0:2384:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607e17840 x1874669228036480/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827084 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827068.151474:0:2384:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1787827068.151475:0:2384:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1787827068.151476:0:2384:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff985610db9400. 02000000:00000001:1.0:1787827068.151478:0:2384:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1787827068.151479:0:2384:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:1.0:1787827068.151480:0:2384:0:(genops.c:1223:class_import_put()) import ffff985611185800 refcount=3 obd=lustre-MDT0001-mdc-ffff985602bc9800 00000020:00000001:1.0:1787827068.151482:0:2384:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:1.0:1787827068.151483:0:2384:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff985607d95500. 02000000:00000001:1.0:1787827068.151485:0:2384:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1787827068.151486:0:2384:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1787827068.151487:0:2384:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607e17840. 00000100:00000001:1.0:1787827068.151489:0:2384:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1787827068.151490:0:2384:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787827068.151492:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787827068.151497:0:2384:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1787827068.151498:0:2384:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827068.151500:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1787827068.151502:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827068.724821:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827068.724828:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827068.724833:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827068.724834:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827068.724837:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827068.724840:0:49842:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:0.0:1787827068.724842:0:49842:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:0.0:1787827068.724848:0:49842:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:0.0:1787827068.724851:0:49842:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:0.0:1787827068.724852:0:49842:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827068.724854:0:49842:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:0.0:1787827068.724856:0:49842:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827068.724858:0:49842:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827068.724860:0:49842:0:(ldlm_pool.c:496:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1787827068.724926:0:15163:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1787827068.724930:0:15163:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1787827068.724934:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:0.0:1787827069.748953:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827069.748960:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827069.748966:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827069.748968:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827069.748971:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827069.748973:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1787827069.749041:0:33341:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1787827069.749045:0:33341:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1787827069.749048:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:0.0:1787827070.772803:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827070.772810:0:49842:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:0.0:1787827070.772812:0:49842:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:0.0:1787827070.772815:0:49842:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827070.772818:0:49842:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:0.0:1787827070.772820:0:49842:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827070.772821:0:49842:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827070.772823:0:49842:0:(ldlm_pool.c:496:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827070.772825:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827070.772826:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827070.772829:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827070.772829:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1787827070.772849:0:15164:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1787827070.772851:0:15164:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1787827070.772853:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:0.0:1787827071.796787:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827071.796792:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827071.796796:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827071.796797:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827071.796799:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827071.796800:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1787827071.796816:0:33342:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1787827071.796818:0:33342:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1787827071.796819:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:0.0:1787827072.820789:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827072.820795:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827072.820798:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827072.820799:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827072.820802:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827072.820803:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1787827072.820862:0:79887:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1787827072.820866:0:79887:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1787827072.820869:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1787827073.268835:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1787827073.268845:0:38:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1787827073.268856:0:38:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 904 at ffff985607f0a940. 00000020:00000040:0.0:1787827073.268861:0:38:0:(genops.c:1210:class_import_get()) import ffff98561118e000 refcount=4 obd=MGC192.168.203.114@tcp 00000100:00000001:0.0:1787827073.268865:0:38:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1787827073.268867:0:38:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1787827073.268872:0:38:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1787827073.268878:0:38:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff98561324c400. 00000100:00000001:0.0:1787827073.268883:0:38:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787827073.268894:0:38:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS req@ffff985607f0a940 x1874669228036608/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1787827073.268903:0:38:0:(jobid.c:939:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1787827073.268907:0:38:0:(jobid.c:979:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787827073.268912:0:38:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000083e565bf] to pc [ptlrpcd_00_01+1] req@ffff985607f0a940 x1874669228036608/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787827073.268939:0:38:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787827073.268942:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1787827073.268944:0:38:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1787827073.268948:0:38:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 904 at ffff985607f08780. 00000020:00000040:0.0:1787827073.268950:0:38:0:(genops.c:1210:class_import_get()) import ffff985611182000 refcount=4 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000100:00000001:0.0:1787827073.268953:0:38:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1787827073.268954:0:38:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1787827073.268955:0:38:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1787827073.268958:0:38:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff98561324c000. 00000100:00000001:0.0:1787827073.268962:0:38:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827073.268962:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:0.0:1787827073.268967:0:38:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID req@ffff985607f08780 x1874669228036736/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1787827073.268968:0:2384:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1787827073.268971:0:2384:0:(client.c:1813:ptlrpc_send_new_req()) Process entered 00000100:00000001:0.0:1787827073.268979:0:38:0:(jobid.c:939:lustre_get_jobid()) Process entered 00000100:00000040:1.0:1787827073.268980:0:2384:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff985607f0a940 x1874669228036608/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787827073.268982:0:38:0:(jobid.c:979:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787827073.268991:0:38:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000fa05cf99] to pc [ptlrpcd_00_00+0] req@ffff985607f08780 x1874669228036736/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827073.268995:0:2384:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1787827073.268998:0:2384:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1787827073.269003:0:2384:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:0.0:1787827073.269005:0:38:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1787827073.269007:0:2384:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000040:0.0:1787827073.269009:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000001:1.0:1787827073.269012:0:2384:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787827073.269014:0:38:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1787827073.269018:0:38:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 904 at ffff985607f0b840. 00000100:00100000:1.0:1787827073.269019:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607f0a940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:3c946890-15ca-497a-b838-fe1cba61f4dd:2384:1874669228036608:192.168.203.114@tcp:400:kworker.0 00000020:00000040:0.0:1787827073.269024:0:38:0:(genops.c:1210:class_import_get()) import ffff985611185800 refcount=4 obd=lustre-MDT0001-mdc-ffff985602bc9800 00000100:00000001:1.0:1787827073.269025:0:2384:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:0.0:1787827073.269028:0:38:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1787827073.269030:0:38:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1787827073.269031:0:2384:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1787827073.269032:0:38:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1787827073.269034:0:2384:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1787827073.269037:0:2384:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1787827073.269039:0:38:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff98561324cf00. 02000000:00000010:1.0:1787827073.269041:0:2384:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff985610db9400. 00000100:00000001:0.0:1787827073.269044:0:38:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1787827073.269044:0:2384:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1787827073.269050:0:2384:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000040:0.0:1787827073.269052:0:38:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID req@ffff985607f0b840 x1874669228036864/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000400:00000010:1.0:1787827073.269054:0:2384:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff985602ecfa50. 00000100:00000001:0.0:1787827073.269063:0:38:0:(jobid.c:939:lustre_get_jobid()) Process entered 00000400:00000010:1.0:1787827073.269063:0:2384:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985606618908. 00000100:00000001:0.0:1787827073.269068:0:38:0:(jobid.c:979:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:1.0:1787827073.269071:0:2384:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1874669228036608, portal 25 00000100:00000040:0.0:1787827073.269073:0:38:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000002bed56f1] to pc [ptlrpcd_00_01+1] req@ffff985607f0b840 x1874669228036864/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827073.269075:0:2384:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1787827073.269078:0:2384:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446630094000662848 : -113979708888768 : ffff985607f0a940) 00000100:00000001:0.0:1787827073.269087:0:38:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1787827073.269089:0:2384:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff985607f0a940 x1874669228036608/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827089 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1787827073.269091:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:0.0:1787827073.269100:0:38:0:(pinger.c:247:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000001:1.0:1787827073.269106:0:2384:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1787827073.269107:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:0.0:1787827073.269112:0:38:0:(pinger.c:247:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000040:1.0:1787827073.269113:0:2384:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.203.114@tcp 00000400:00000010:1.0:1787827073.269118:0:2384:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9856066183b8. 00000100:00000040:0.0:1787827073.269119:0:38:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (3229) 00000100:00000200:1.0:1787827073.269123:0:2384:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1874669228036608, offset 0 00000100:00000001:0.0:1787827073.269138:0:2383:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1787827073.269141:0:2383:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000400:00000200:1.0:1787827073.269143:0:2384:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.203.114@tcp 00000100:00000001:0.0:1787827073.269144:0:2383:0:(client.c:1813:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1787827073.269149:0:2383:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff985607f08780 x1874669228036736/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787827073.269155:0:2383:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000400:00000200:1.0:1787827073.269155:0:2384:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000001:0.0:1787827073.269157:0:2383:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1787827073.269160:0:2383:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1787827073.269162:0:2383:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000400:00000200:1.0:1787827073.269164:0:2384:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.203.114@tcp local destination 02000000:00000001:0.0:1787827073.269165:0:2383:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1787827073.269171:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607f08780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669228036736:192.168.203.114@tcp:400:kworker.0 00000400:00000200:1.0:1787827073.269172:0:2384:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:0.0:1787827073.269176:0:2383:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000200:1.0:1787827073.269178:0:2384:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.203.14@tcp 02000000:00000001:0.0:1787827073.269180:0:2383:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1787827073.269182:0:2383:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1787827073.269182:0:2384:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.203.14@tcp lpni_is_preferred = 0 02000000:00000001:0.0:1787827073.269185:0:2383:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000200:1.0:1787827073.269188:0:2384:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.203.114@tcp 02000000:00000010:0.0:1787827073.269190:0:2383:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff98560540bc00. 00000400:00000200:1.0:1787827073.269192:0:2384:0:(lib-move.c:1745:lnet_handle_send()) 192.168.203.14@tcp NI seq info: [21924:21924:256:4294967295] 192.168.203.114@tcp LPNI seq info [21924:21924:8:4294967295] 02000000:00000001:0.0:1787827073.269193:0:2383:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1787827073.269200:0:2384:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:0.0:1787827073.269201:0:2383:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000010:0.0:1787827073.269204:0:2383:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9856094acf20. 00000400:00000010:0.0:1787827073.269208:0:2383:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985610d306e8. 00000400:00000200:1.0:1787827073.269212:0:2384:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp:) -> 192.168.203.114@tcp(192.168.203.114@tcp:192.168.203.114@tcp) : PUT try# 0 00000100:00000200:0.0:1787827073.269213:0:2383:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1874669228036736, portal 10 00000100:00000001:0.0:1787827073.269217:0:2383:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1787827073.269219:0:2383:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446630094000654208 : -113979708897408 : ffff985607f08780) 00000800:00000200:1.0:1787827073.269220:0:2384:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.203.114@tcp 00000100:00000040:0.0:1787827073.269226:0:2383:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff985607f08780 x1874669228036736/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827089 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000800:00000010:1.0:1787827073.269226:0:2384:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff985607d95500. 00000100:00000001:0.0:1787827073.269232:0:2383:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000800:00000200:1.0:1787827073.269235:0:2384:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff985606751480] -> 12345-192.168.203.114@tcp (5) 00000100:00000040:0.0:1787827073.269238:0:2383:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.203.114@tcp 00000400:00000010:0.0:1787827073.269241:0:2383:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985610d30198. 00000100:00000200:0.0:1787827073.269243:0:2383:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1874669228036736, offset 0 00000800:00000200:1.0:1787827073.269244:0:2384:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.203.114@tcp ip 192.168.203.114:988 00000400:00000200:0.0:1787827073.269249:0:2383:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.203.114@tcp 00000800:00000200:1.0:1787827073.269250:0:2384:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff985607d95500 type 1, nob 320 niov 1 nkiov 1 00000400:00000200:0.0:1787827073.269254:0:2383:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:0.0:1787827073.269258:0:2383:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.203.114@tcp local destination 00000400:00000200:0.0:1787827073.269260:0:2383:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1787827073.269262:0:2383:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.203.14@tcp 00000400:00000200:0.0:1787827073.269264:0:2383:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.203.14@tcp lpni_is_preferred = 0 00000100:00000001:1.0:1787827073.269264:0:2384:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1787827073.269269:0:2383:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.203.114@tcp 00000400:00000200:1.0:1787827073.269271:0:2384:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:0.0:1787827073.269274:0:2383:0:(lib-move.c:1745:lnet_handle_send()) 192.168.203.14@tcp NI seq info: [21925:21925:255:4294967295] 192.168.203.114@tcp LPNI seq info [21925:21925:7:4294967295] 00000100:00000001:1.0:1787827073.269276:0:2384:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.203.114@tcp: PM QoS new deadline estimation for cpu 2 is 4297901308 00000400:00000200:0.0:1787827073.269281:0:2383:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000001:1.0:1787827073.269284:0:2384:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.203.114@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4297896188 00000100:00000001:1.0:1787827073.269290:0:2384:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.203.114@tcp: PM QoS new deadline estimation for cpu 3 is 4297901308 00000400:00000200:0.0:1787827073.269295:0:2383:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp:) -> 192.168.203.114@tcp(192.168.203.114@tcp:192.168.203.114@tcp) : PUT try# 0 00000100:00000001:1.0:1787827073.269295:0:2384:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.203.114@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4297896188 00000100:00000001:1.0:1787827073.269300:0:2384:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1787827073.269303:0:2384:0:(client.c:1941:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1787827073.269306:0:2383:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.203.114@tcp 00000100:00000001:1.0:1787827073.269307:0:2384:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827073.269311:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000800:00000010:0.0:1787827073.269314:0:2383:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff98561324c200. 00000100:00000001:1.0:1787827073.269317:0:2384:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1787827073.269319:0:2384:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000800:00000200:0.0:1787827073.269321:0:2383:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff985606751480] -> 12345-192.168.203.114@tcp (5) 00000100:00000001:1.0:1787827073.269322:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1787827073.269324:0:2384:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1787827073.269327:0:2384:0:(client.c:1813:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1787827073.269331:0:2384:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff985607f0b840 x1874669228036864/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000800:00000200:0.0:1787827073.269332:0:2383:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.203.114@tcp ip 192.168.203.114:988 00000800:00000200:0.0:1787827073.269338:0:2383:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff98561324c200 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1787827073.269340:0:2384:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1787827073.269342:0:2384:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787827073.269345:0:2383:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1787827073.269345:0:2384:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1787827073.269347:0:2384:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1787827073.269350:0:2384:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787827073.269351:0:2383:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff985602bc9800: using a timeout of 38035 usecs (39 jiffies) 00000100:00100000:1.0:1787827073.269353:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607f0b840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669228036864:192.168.203.114@tcp:400:kworker.0 00000100:00000001:1.0:1787827073.269357:0:2384:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000200:0.0:1787827073.269360:0:2383:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 02000000:00000001:1.0:1787827073.269360:0:2384:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1787827073.269363:0:2384:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787827073.269365:0:2383:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff985602bc9800: PM QoS new deadline estimation for cpu 2 is 4297891367 02000000:00000001:1.0:1787827073.269365:0:2384:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:0.0:1787827073.269370:0:2383:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff985602bc9800: PM QoS request active for cpu 2, keep current deadline 4297901308 02000000:00000010:1.0:1787827073.269370:0:2384:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff985610db9a00. 02000000:00000001:1.0:1787827073.269372:0:2384:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787827073.269376:0:2383:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff985602bc9800: PM QoS new deadline estimation for cpu 3 is 4297891367 00000400:00000200:1.0:1787827073.269377:0:2384:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000001:0.0:1787827073.269382:0:2383:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff985602bc9800: PM QoS request active for cpu 3, keep current deadline 4297901308 00000400:00000010:1.0:1787827073.269383:0:2384:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff985602ecf9f8. 00000100:00000001:0.0:1787827073.269387:0:2383:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000010:1.0:1787827073.269388:0:2384:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985606618000. 00000100:00000001:0.0:1787827073.269392:0:2383:0:(client.c:1941:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1787827073.269392:0:2378:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000200:1.0:1787827073.269393:0:2384:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1874669228036864, portal 10 00000100:00000001:1.0:1787827073.269398:0:2384:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1787827073.269399:0:2383:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:3.0:1787827073.269400:0:2378:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff985607d95500. 00000100:00000001:1.0:1787827073.269401:0:2384:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446630094000666688 : -113979708884928 : ffff985607f0b840) 00000100:00000001:0.0:1787827073.269405:0:2383:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1787827073.269410:0:2384:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff985607f0b840 x1874669228036864/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827089 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1787827073.269410:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1787827073.269411:0:2383:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1787827073.269414:0:2383:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1787827073.269419:0:2383:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1787827073.269422:0:2383:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000400:00000200:3.0:1787827073.269423:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000100:00000001:0.0:1787827073.269426:0:2383:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827073.269427:0:2384:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:0.0:1787827073.269432:0:2383:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1787827073.269432:0:2384:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.203.114@tcp 00000400:00000200:3.0:1787827073.269432:0:2378:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9856066183b8 00000400:00000010:1.0:1787827073.269435:0:2384:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff985606618bb0. 00000400:00000010:3.0:1787827073.269438:0:2378:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9856066183b8. 00000100:00000001:3.0:1787827073.269448:0:2378:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1787827073.269449:0:2384:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1874669228036864, offset 0 00000400:00000200:1.0:1787827073.269455:0:2384:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.203.114@tcp 00000100:00000200:3.0:1787827073.269456:0:2378:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff985607f0a940 x1874669228036608/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827089 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1787827073.269461:0:2384:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:1.0:1787827073.269468:0:2384:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.203.114@tcp local destination 00000400:00000200:1.0:1787827073.269473:0:2384:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:3.0:1787827073.269474:0:2378:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000400:00000200:1.0:1787827073.269488:0:2384:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.203.14@tcp 00000400:00000200:1.0:1787827073.269491:0:2384:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.203.14@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1787827073.269494:0:2384:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.203.114@tcp 00000400:00000200:1.0:1787827073.269496:0:2384:0:(lib-move.c:1745:lnet_handle_send()) 192.168.203.14@tcp NI seq info: [21926:21926:254:4294967295] 192.168.203.114@tcp LPNI seq info [21926:21926:6:4294967295] 00000400:00000200:1.0:1787827073.269500:0:2384:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:1.0:1787827073.269506:0:2384:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp:) -> 192.168.203.114@tcp(192.168.203.114@tcp:192.168.203.114@tcp) : PUT try# 0 00000800:00000200:1.0:1787827073.269511:0:2384:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.203.114@tcp 00000800:00000010:1.0:1787827073.269514:0:2384:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff985607d95600. 00000800:00000200:1.0:1787827073.269518:0:2384:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff985606751480] -> 12345-192.168.203.114@tcp (5) 00000800:00000200:1.0:1787827073.269523:0:2384:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.203.114@tcp ip 192.168.203.114:988 00000800:00000200:1.0:1787827073.269525:0:2384:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff985607d95600 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1787827073.269533:0:2384:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1787827073.269536:0:2384:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff985602bc9800: using a timeout of 11870 usecs (12 jiffies) 00000400:00000200:1.0:1787827073.269540:0:2384:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000001:1.0:1787827073.269542:0:2384:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff985602bc9800: PM QoS new deadline estimation for cpu 2 is 4297891340 00000100:00000001:1.0:1787827073.269546:0:2384:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff985602bc9800: PM QoS request active for cpu 2, keep current deadline 4297901308 00000100:00000040:3.0:1787827073.269549:0:2378:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff985607f0a940 x1874669228036608/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827089 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827073.269550:0:2384:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff985602bc9800: PM QoS new deadline estimation for cpu 3 is 4297891340 00000100:00000001:1.0:1787827073.269554:0:2384:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff985602bc9800: PM QoS request active for cpu 3, keep current deadline 4297901308 00000100:00000001:1.0:1787827073.269562:0:2384:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1787827073.269567:0:2384:0:(client.c:1941:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827073.269569:0:2378:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827073.269571:0:2384:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827073.269572:0:2378:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:1.0:1787827073.269576:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787827073.269580:0:2384:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1787827073.269582:0:2384:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1787827073.269584:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1787827073.269586:0:2384:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1787827073.269587:0:2384:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827073.269590:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1787827073.269593:0:2378:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1787827073.269597:0:2378:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff98561324c200. 00000400:00000200:3.0:1787827073.269600:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1787827073.269658:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000400:00000200:3.0:1787827073.269661:0:2378:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985610d30198 00000400:00000010:3.0:1787827073.269662:0:2378:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985610d30198. 00000100:00000001:3.0:1787827073.269665:0:2378:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1787827073.269668:0:2378:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff985607f08780 x1874669228036736/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827089 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827073.269673:0:2378:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1787827073.269675:0:2378:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff985607f08780 x1874669228036736/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827089 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827073.269679:0:2378:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827073.269681:0:2378:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1787827073.269734:0:2378:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1787827073.269736:0:2378:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff985607d95600. 00000400:00000200:3.0:1787827073.269739:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1787827073.269743:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000400:00000200:3.0:1787827073.269746:0:2378:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985606618bb0 00000400:00000010:3.0:1787827073.269748:0:2378:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985606618bb0. 00000100:00000001:3.0:1787827073.269750:0:2378:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1787827073.269754:0:2378:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff985607f0b840 x1874669228036864/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827089 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827073.269759:0:2378:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1787827073.269763:0:2378:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff985607f0b840 x1874669228036864/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827089 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827073.269769:0:2378:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787827073.269770:0:2378:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1787827073.270378:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827073.270389:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1787827073.270392:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827073.270395:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827073.270400:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827073.270406:0:2377:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp) <- 192.168.203.114@tcp : PUT - for me 00000400:00000200:2.0:1787827073.270411:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827073.270415:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827073.270419:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827073.270422:0:2377:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.203.114@tcp of length 224 into portal 25 MB=0x6a90071689200 00000400:00000200:2.0:1787827073.270426:0:2377:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:2.0:1787827073.270431:0:2377:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.203.114@tcp of length 224/224 into md 0x32475 [1] + 224 00000800:00000001:2.0:1787827073.270437:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827073.270442:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1787827073.270445:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1787827073.270448:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000100:00000001:2.0:1787827073.270452:0:2377:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1787827073.270458:0:2377:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff985607f0a940 x1874669228036608/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827089 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1787827073.270468:0:2377:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff985607f0a940 x1874669228036608/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827089 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787827073.270498:0:2377:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1787827073.270504:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1787827073.270507:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787827073.270534:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1787827073.270537:0:2384:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1787827073.270546:0:2384:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1787827073.270547:0:2384:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1787827073.270550:0:2384:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff985602ecfa50. 00000400:00000200:1.0:1787827073.270554:0:2384:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985606618908 00000400:00000010:1.0:1787827073.270556:0:2384:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985606618908. 00000100:00000001:1.0:1787827073.270566:0:2384:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1787827073.270571:0:2384:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff985607f0a940 x1874669228036608/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827089 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1787827073.270581:0:2384:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff985607f0a940 x1874669228036608/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827089 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827073.270638:0:2384:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1787827073.270640:0:2384:0:(client.c:3001:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787827073.270642:0:2384:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:1.0:1787827073.270644:0:2384:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1787827073.270646:0:2384:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1787827073.270647:0:2384:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1787827073.270649:0:2384:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1787827073.270654:0:2384:0:(import.c:2149:obd_at_measure()) add 5 to ffff98561118e460 time=73 v=5 (5 5 5 5) 00000100:00001000:1.0:1787827073.270657:0:2384:0:(import.c:2149:obd_at_measure()) add 1 to ffff98561118e418 time=88 v=5 (1 0 0 0) 00000100:00000001:1.0:1787827073.270664:0:2384:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1787827073.270665:0:2384:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1787827073.270667:0:2384:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1787827073.270669:0:2384:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827073.270670:0:2384:0:(client.c:1797:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1787827073.270674:0:2384:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff985607f0a940 x1874669228036608/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827089 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827073.270679:0:2384:0:(client.c:2354:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1787827073.270681:0:2384:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1787827073.270683:0:2384:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1787827073.270685:0:2384:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff985607f0a940 x1874669228036608/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827089 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827073.270705:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607f0a940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:3c946890-15ca-497a-b838-fe1cba61f4dd:2384:1874669228036608:192.168.203.114@tcp:400:kworker.0 00000100:00000001:1.0:1787827073.270709:0:2384:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827073.270714:0:2384:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1787827073.270717:0:2384:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607f0a940 x1874669228036608/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827089 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827073.270724:0:2384:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1787827073.270726:0:2384:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1787827073.270729:0:2384:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff985610db9400. 02000000:00000001:1.0:1787827073.270732:0:2384:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1787827073.270735:0:2384:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:1.0:1787827073.270748:0:2384:0:(genops.c:1223:class_import_put()) import ffff98561118e000 refcount=3 obd=MGC192.168.203.114@tcp 00000020:00000001:1.0:1787827073.270751:0:2384:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:1.0:1787827073.270754:0:2384:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff98561324c400. 02000000:00000001:1.0:1787827073.270757:0:2384:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1787827073.270758:0:2384:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1787827073.270760:0:2384:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607f0a940. 00000100:00000001:1.0:1787827073.270763:0:2384:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1787827073.270765:0:2384:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787827073.270767:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827073.270770:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1787827073.270771:0:2384:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1787827073.270776:0:2384:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827073.270778:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1787827073.270843:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1787827073.270847:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1787827073.270849:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1787827073.270850:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1787827073.270853:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:3.0:1787827073.270856:0:2378:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp) <- 192.168.203.114@tcp : PUT - for me 00000400:00000200:3.0:1787827073.270860:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:3.0:1787827073.270862:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:3.0:1787827073.270864:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:3.0:1787827073.270866:0:2378:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.203.114@tcp of length 224 into portal 10 MB=0x6a90071689300 00000400:00000200:3.0:1787827073.270868:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:3.0:1787827073.270871:0:2378:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.203.114@tcp of length 224/224 into md 0x32485 [1] + 224 00000800:00000001:3.0:1787827073.270874:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1787827073.270876:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1787827073.270877:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1787827073.270880:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000100:00000001:3.0:1787827073.270883:0:2378:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1787827073.270887:0:2378:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff985607f0b840 x1874669228036864/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827089 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1787827073.270895:0:2378:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff985607f0b840 x1874669228036864/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827089 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787827073.270947:0:2378:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1787827073.270953:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1787827073.270957:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1787827073.270959:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1787827073.270962:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1787827073.270966:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000200:3.0:1787827073.270971:0:2378:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.203.14@tcp(192.168.203.14@tcp) <- 192.168.203.114@tcp : PUT - for me 00000100:00000001:1.0:1787827073.270975:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1787827073.270978:0:2384:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000400:00000200:3.0:1787827073.270979:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000001:1.0:1787827073.270981:0:2384:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1787827073.270983:0:2384:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000200:3.0:1787827073.270986:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000400:00000010:1.0:1787827073.270987:0:2384:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff985602ecf9f8. 00000400:00000200:1.0:1787827073.270991:0:2384:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985606618000 00000400:00000010:1.0:1787827073.270994:0:2384:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985606618000. 00000400:00000200:3.0:1787827073.270995:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000001:1.0:1787827073.270996:0:2384:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1787827073.271003:0:2384:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff985607f0b840 x1874669228036864/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827089 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1787827073.271004:0:2378:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.203.114@tcp of length 224 into portal 10 MB=0x6a90071689280 00000400:00000200:3.0:1787827073.271015:0:2378:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.203.114@tcp to cpt 1 00000100:00000200:1.0:1787827073.271017:0:2384:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff985607f0b840 x1874669228036864/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827089 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827073.271023:0:2384:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1787827073.271024:0:2384:0:(client.c:3001:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787827073.271025:0:2384:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:1.0:1787827073.271026:0:2384:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1787827073.271027:0:2384:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1787827073.271029:0:2384:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1787827073.271031:0:2384:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1787827073.271036:0:2384:0:(import.c:2149:obd_at_measure()) add 5 to ffff985611185c60 time=73 v=5 (5 5 5 5) 00000100:00001000:1.0:1787827073.271040:0:2384:0:(import.c:2149:obd_at_measure()) add 1 to ffff985611185c18 time=73 v=5 (1 1 1 1) 00000100:00000001:1.0:1787827073.271042:0:2384:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1787827073.271043:0:2384:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1787827073.271045:0:2384:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1787827073.271047:0:2384:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827073.271049:0:2384:0:(client.c:3109:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1787827073.271051:0:2384:0:(client.c:3121:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff985602bc9800: skip recheck: last_committed 0 00000100:00000001:1.0:1787827073.271054:0:2384:0:(client.c:3122:ptlrpc_free_committed()) Process leaving 00000400:00000200:3.0:1787827073.271056:0:2378:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.203.114@tcp of length 224/224 into md 0x3247d [1] + 224 00000100:00000001:1.0:1787827073.271057:0:2384:0:(client.c:1797:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1787827073.271065:0:2384:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff985607f0b840 x1874669228036864/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827089 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000800:00000001:3.0:1787827073.271066:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1787827073.271073:0:2384:0:(client.c:2354:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000800:00000001:3.0:1787827073.271074:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787827073.271077:0:2384:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000400:00000200:3.0:1787827073.271079:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:1.0:1787827073.271080:0:2384:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1787827073.271087:0:2384:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff985607f0b840 x1874669228036864/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827089 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1787827073.271095:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.203.14@tcp->192.168.203.114@tcp: PUT: OK 00000100:00100000:1.0:1787827073.271102:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607f0b840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669228036864:192.168.203.114@tcp:400:kworker.0 00000100:00000001:3.0:1787827073.271103:0:2378:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:1.0:1787827073.271109:0:2384:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:3.0:1787827073.271111:0:2378:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff985607f08780 x1874669228036736/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827089 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827073.271112:0:2384:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1787827073.271119:0:2384:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607f0b840 x1874669228036864/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827089 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1787827073.271130:0:2378:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff985607f08780 x1874669228036736/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827089 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787827073.271143:0:2384:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1787827073.271145:0:2384:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1787827073.271147:0:2384:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff985610db9a00. 02000000:00000001:1.0:1787827073.271150:0:2384:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1787827073.271152:0:2384:0:(genops.c:1217:class_import_put()) Process entered 00000100:00000001:3.0:1787827073.271153:0:2378:0:(events.c:163:reply_in_callback()) Process leaving 00000020:00000040:1.0:1787827073.271154:0:2384:0:(genops.c:1223:class_import_put()) import ffff985611185800 refcount=3 obd=lustre-MDT0001-mdc-ffff985602bc9800 00000020:00000001:1.0:1787827073.271158:0:2384:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:1.0:1787827073.271160:0:2384:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff98561324cf00. 00000800:00000001:3.0:1787827073.271161:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 02000000:00000001:1.0:1787827073.271163:0:2384:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1787827073.271166:0:2384:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000800:00000001:3.0:1787827073.271166:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000010:1.0:1787827073.271170:0:2384:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607f0b840. 00000100:00000001:1.0:1787827073.271178:0:2384:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1787827073.271180:0:2384:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787827073.271182:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787827073.271187:0:2384:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1787827073.271189:0:2384:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787827073.271191:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1787827073.271193:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787827073.271272:0:2383:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1787827073.271277:0:2383:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1787827073.271285:0:2383:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1787827073.271287:0:2383:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1787827073.271291:0:2383:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9856094acf20. 00000400:00000200:0.0:1787827073.271298:0:2383:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff985610d306e8 00000400:00000010:0.0:1787827073.271301:0:2383:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff985610d306e8. 00000100:00000001:0.0:1787827073.271305:0:2383:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1787827073.271315:0:2383:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff985607f08780 x1874669228036736/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827089 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1787827073.271327:0:2383:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff985607f08780 x1874669228036736/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827089 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787827073.271340:0:2383:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1787827073.271342:0:2383:0:(client.c:3001:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1787827073.271345:0:2383:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:0.0:1787827073.271348:0:2383:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1787827073.271350:0:2383:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1787827073.271352:0:2383:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1787827073.271355:0:2383:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1787827073.271362:0:2383:0:(import.c:2149:obd_at_measure()) add 5 to ffff985611182460 time=151 v=5 (5 5 5 5) 00000100:00001000:0.0:1787827073.271366:0:2383:0:(import.c:2149:obd_at_measure()) add 1 to ffff985611182418 time=87 v=5 (1 0 0 0) 00000100:00000001:0.0:1787827073.271369:0:2383:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1787827073.271371:0:2383:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827073.271374:0:2383:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1787827073.271377:0:2383:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787827073.271380:0:2383:0:(client.c:3109:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1787827073.271381:0:2383:0:(client.c:3121:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: skip recheck: last_committed 219043334508 00000100:00000001:0.0:1787827073.271383:0:2383:0:(client.c:3122:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1787827073.271384:0:2383:0:(client.c:1797:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787827073.271388:0:2383:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff985607f08780 x1874669228036736/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827089 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787827073.271393:0:2383:0:(client.c:2354:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1787827073.271395:0:2383:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1787827073.271396:0:2383:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1787827073.271399:0:2383:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff985607f08780 x1874669228036736/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827089 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827073.271403:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607f08780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669228036736:192.168.203.114@tcp:400:kworker.0 00000100:00000001:0.0:1787827073.271406:0:2383:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1787827073.271408:0:2383:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1787827073.271410:0:2383:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff985607f08780 x1874669228036736/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827089 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787827073.271414:0:2383:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1787827073.271417:0:2383:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1787827073.271419:0:2383:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff98560540bc00. 02000000:00000001:0.0:1787827073.271423:0:2383:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1787827073.271425:0:2383:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:0.0:1787827073.271427:0:2383:0:(genops.c:1223:class_import_put()) import ffff985611182000 refcount=3 obd=lustre-MDT0000-mdc-ffff985602bc9800 00000020:00000001:0.0:1787827073.271429:0:2383:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:0.0:1787827073.271430:0:2383:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff98561324c000. 02000000:00000001:0.0:1787827073.271432:0:2383:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1787827073.271434:0:2383:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1787827073.271435:0:2383:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 904 at ffff985607f08780. 00000100:00000001:0.0:1787827073.271438:0:2383:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1787827073.271439:0:2383:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1787827073.271441:0:2383:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1787827073.271447:0:2383:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1787827073.271449:0:2383:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787827073.271451:0:2383:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1787827073.271453:0:2383:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827073.844774:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827073.844782:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827073.844789:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827073.844790:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827073.844795:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827073.844796:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1787827073.844855:0:15163:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1787827073.844859:0:15163:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1787827073.844862:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:0.0:1787827074.868813:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827074.868820:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827074.868826:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827074.868827:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827074.868831:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827074.868832:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1787827074.868893:0:33341:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1787827074.868897:0:33341:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1787827074.868900:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:0.0:1787827075.892807:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827075.892812:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827075.892817:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827075.892818:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827075.892820:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827075.892821:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1787827075.892838:0:15164:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1787827075.892840:0:15164:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1787827075.892842:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:0.0:1787827076.916812:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827076.916819:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827076.916826:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827076.916829:0:49842:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:0.0:1787827076.916832:0:49842:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:0.0:1787827076.916834:0:49842:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827076.916838:0:49842:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:0.0:1787827076.916840:0:49842:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827076.916842:0:49842:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827076.916844:0:49842:0:(ldlm_pool.c:496:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787827076.916848:0:49842:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1787827076.916849:0:49842:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1787827076.916872:0:33342:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1787827076.916875:0:33342:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1787827076.916878:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000001:3.0:1787827077.430552:0:88825:0:(debug.c:305:cfs_str2mask()) Process entered 00010000:00010000:3.0:1787827077.940860:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827078.388831:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607e161c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2383:1874669228036992:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827078.388841:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607e17c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669228037120:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787827078.388960:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607e14000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669228037248:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787827078.390025:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607e161c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2383:1874669228036992:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827078.390099:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607e17c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669228037120:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787827078.390112:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607e14000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669228037248:192.168.203.114@tcp:400:kworker.0 00010000:00010000:1.0:1787827079.988799:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200000:0.0:1787827080.421306:0:87899:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:0.0:1787827080.421343:0:87899:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827080.421418:0:87899:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7ee551 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 87899 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827080.421440:0:87899:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7ee551 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 87899 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827080.421451:0:87899:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7ee551 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 87899 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787827080.421477:0:87899:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7840 pname:cluuid:pid:xid:nid:opc:job bash:7b4e8535-9db7-485d-abf6-bda966b63320:87899:1874669228190976:192.168.203.114@tcp:101:bash.0 00000100:00100000:0.0:1787827080.421583:0:87899:0:(client.c:2715:ptlrpc_set_wait()) set ffff98560869dc00 going to sleep for 11 seconds 00000100:00100000:0.0:1787827080.423231:0:87899:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7840 pname:cluuid:pid:xid:nid:opc:job bash:7b4e8535-9db7-485d-abf6-bda966b63320:87899:1874669228190976:192.168.203.114@tcp:101:bash.0 00010000:00010000:0.0:1787827080.423267:0:87899:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7ee551 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e3c6b expref: -99 pid: 87899 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827080.423281:0:87899:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7ee551 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e3c6b expref: -99 pid: 87899 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827080.423296:0:87899:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7ee551 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e3c6b expref: -99 pid: 87899 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827080.423308:0:87899:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7ee551 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e3c6b expref: -99 pid: 87899 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827080.423322:0:87899:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7ee551 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e3c6b expref: -99 pid: 87899 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827080.423331:0:87899:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7ee551 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e3c6b expref: -99 pid: 87899 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827080.423338:0:87899:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7ee551 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e3c6b expref: -99 pid: 87899 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787827080.423354:0:87899:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff985607df7840 x1874669228190976/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 576/688 e 0 to 0 dl 1787827096 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'bash.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787827080.423368:0:87899:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7ee551 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e3c6b expref: -99 pid: 87899 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827080.423387:0:87899:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:0.0:1787827080.423394:0:87899:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:0.0:1787827080.423445:0:87899:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 69632 00000080:00010000:0.0:1787827080.423454:0:87899:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee551 00000080:00010000:0.0:1787827080.423461:0:87899:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:0.0:1787827080.423464:0:87899:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee551 from it ffffab3ac1c97bb0 00010000:00010000:0.0:1787827080.423469:0:87899:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7ee551 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e3c6b expref: -99 pid: 87899 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827080.423479:0:87899:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7ee551 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e3c6b expref: -99 pid: 87899 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787827080.423508:0:87899:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 44 00000080:00200008:0.0:1787827080.423521:0:87899:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 304000 00000080:00200008:0.0:1787827080.423544:0:87899:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 2047, flags 0, foreign 0 00000002:00010000:0.0:1787827080.423574:0:87899:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: readdir flags 0 00010000:00010000:0.0:1787827080.423581:0:87899:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7ee551 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e3c6b expref: -99 pid: 87899 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787827080.423589:0:87899:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7ee551 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e3c6b expref: -99 pid: 87899 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000100:00100000:0.0:1787827080.424237:0:87899:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7840 pname:cluuid:pid:xid:nid:opc:job bash:7b4e8535-9db7-485d-abf6-bda966b63320:87899:1874669228191104:192.168.203.114@tcp:37:bash.0 00000100:00100000:0.0:1787827080.424328:0:87899:0:(client.c:2715:ptlrpc_set_wait()) set ffff98560869dc00 going to sleep for 11 seconds 00000100:00100000:0.0:1787827080.427568:0:87899:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7840 pname:cluuid:pid:xid:nid:opc:job bash:7b4e8535-9db7-485d-abf6-bda966b63320:87899:1874669228191104:192.168.203.114@tcp:37:bash.0 00010000:00010000:0.0:1787827080.428419:0:87899:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7ee551 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e3c6b expref: -99 pid: 87899 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787827080.428454:0:87899:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7ee551 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e3c6b expref: -99 pid: 87899 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1787827080.428497:0:87899:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:0.0:1787827080.428525:0:87899:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827080.428545:0:87899:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7ee551 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e3c6b expref: -99 pid: 87899 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1787827080.428559:0:87899:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7ee551 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e3c6b expref: -99 pid: 87899 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1787827080.428567:0:87899:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827080.428574:0:87899:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:0.0:1787827080.428578:0:87899:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee551 00000080:00010000:0.0:1787827080.428584:0:87899:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee551 from it ffffab3ac1c97cd8 00010000:00010000:0.0:1787827080.428587:0:87899:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7ee551 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e3c6b expref: -99 pid: 87899 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1787827080.428593:0:87899:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7ee551 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e3c6b expref: -99 pid: 87899 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 1 00000080:00200000:0.0:1787827080.428948:0:87899:0:(dir.c:332:ll_iterate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) pos/size0/69632 32bit_api 0 00000080:00200000:0.0:1787827080.429006:0:87899:0:(dir.c:332:ll_iterate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) pos/size18446744073709551614/69632 32bit_api 0 00000080:00200008:0.0:1787827080.429027:0:87899:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 304000 00000080:00200000:2.0:1787827080.433953:0:89076:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:2.0:1787827080.434160:0:89076:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827080.434186:0:89076:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7ee551 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e3c6b expref: -99 pid: 87899 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 1 00010000:00010000:2.0:1787827080.434206:0:89076:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7ee551 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e3c6b expref: -99 pid: 87899 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 1 00000080:00200000:2.0:1787827080.434214:0:89076:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827080.434223:0:89076:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:2.0:1787827080.434228:0:89076:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee551 00000080:00010000:2.0:1787827080.434267:0:89076:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee551 from it ffffab3ac1cafb58 00010000:00010000:2.0:1787827080.434271:0:89076:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7ee551 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e3c6b expref: -99 pid: 87899 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 1 00010000:00010000:2.0:1787827080.434279:0:89076:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7ee551 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e3c6b expref: -99 pid: 87899 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00000080:00200000:2.0:1787827080.434295:0:89076:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:2.0:1787827080.434307:0:89076:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f42.replay-single-*, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), flags=8 00000080:00200000:2.0:1787827080.434311:0:89076:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f42.replay-single-*, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=getattr 00000002:00010000:2.0:1787827080.434323:0:89076:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f42.replay-single-*,[0x0:0x0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 01 00010000:00010000:2.0:1787827080.434369:0:89076:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9800/0xb3f83dad1b7ee558 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 89076 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827080.434375:0:89076:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9800/0xb3f83dad1b7ee558 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 89076 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827080.434386:0:89076:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9800/0xb3f83dad1b7ee558 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 89076 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787827080.434412:0:89076:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job checkstat:7b4e8535-9db7-485d-abf6-bda966b63320:89076:1874669228191232:192.168.203.114@tcp:101:checkstat.0 00000100:00100000:2.0:1787827080.434521:0:89076:0:(client.c:2715:ptlrpc_set_wait()) set ffff985605437b40 going to sleep for 11 seconds 00000100:00100000:0.0:1787827080.436752:0:89076:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job checkstat:7b4e8535-9db7-485d-abf6-bda966b63320:89076:1874669228191232:192.168.203.114@tcp:101:checkstat.0 00010000:00010000:0.0:1787827080.436783:0:89076:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9800/0xb3f83dad1b7ee558 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 89076 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827080.436798:0:89076:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9800/0xb3f83dad1b7ee558 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 89076 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827080.436807:0:89076:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9800/0xb3f83dad1b7ee558 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 89076 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827080.436815:0:89076:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9800/0xb3f83dad1b7ee558 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 89076 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827080.436824:0:89076:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9800/0xb3f83dad1b7ee558 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 89076 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827080.436831:0:89076:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff985607da9800 already unused, calling callback (ffffffffc1d08990) 00010000:00010000:0.0:1787827080.436837:0:89076:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9800/0xb3f83dad1b7ee558 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 89076 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827080.436845:0:89076:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9800/0xb3f83dad1b7ee558 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 89076 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827080.436853:0:89076:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9800/0xb3f83dad1b7ee558 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x20e400000000 pid: 89076 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827080.436862:0:89076:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787827080.436870:0:89076:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9800/0xb3f83dad1b7ee558 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 89076 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827080.436878:0:89076:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607da9800/0xb3f83dad1b7ee558 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 89076 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787827080.436902:0:89076:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9856140d9680 x1874669228191232/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 592/584 e 0 to 0 dl 1787827096 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'checkstat.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787827080.436917:0:89076:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00010000:00010000:0.0:1787827080.436992:0:89076:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7ee551 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e3c6b expref: -99 pid: 87899 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00010000:00010000:0.0:1787827080.437006:0:89076:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7ee551 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e3c6b expref: -99 pid: 87899 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00000080:00010000:0.0:1787827080.437016:0:89076:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee551 from it ffffab3ac1caf930 00010000:00010000:0.0:1787827080.437030:0:89076:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7ee551 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e3c6b expref: -99 pid: 87899 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00010000:00010000:0.0:1787827080.437043:0:89076:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7ee551 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e3c6b expref: -99 pid: 87899 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00010000:00010000:1.0:1787827081.012842:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827082.036802:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:2.0:1787827082.663500:0:89639:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == replay-single test 43: mds osc import failure during recovery; don't LBUG ========================================================== 06:38:02 (1787827082) 00010000:00010000:3.0:1787827083.060831:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200008:0.0:1787827083.158953:0:89691:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:0.0:1787827083.158963:0:89691:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:0.0:1787827083.158988:0:89691:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827083.159007:0:89691:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7ee551 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e3c6b expref: -99 pid: 87899 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00010000:00010000:0.0:1787827083.159021:0:89691:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7ee551 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e3c6b expref: -99 pid: 87899 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00000080:00200000:0.0:1787827083.159030:0:89691:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827083.159038:0:89691:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:0.0:1787827083.159042:0:89691:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee551 00000080:00010000:0.0:1787827083.159050:0:89691:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee551 from it ffffab3ac1897cb8 00010000:00010000:0.0:1787827083.159054:0:89691:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7ee551 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e3c6b expref: -99 pid: 87899 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00010000:00010000:0.0:1787827083.159063:0:89691:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7ee551 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e3c6b expref: -99 pid: 87899 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00000080:00200008:0.0:1787827083.159586:0:89691:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 0, flags 0, foreign 0 00000080:00200000:0.0:1787827083.159591:0:89691:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:0.0:1787827083.159600:0:89691:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827083.159608:0:89691:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7ee551 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e3c6b expref: -99 pid: 87899 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00010000:00010000:0.0:1787827083.159617:0:89691:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7ee551 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e3c6b expref: -99 pid: 87899 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00000080:00200000:0.0:1787827083.159624:0:89691:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827083.159629:0:89691:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:0.0:1787827083.159632:0:89691:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee551 00000080:00010000:0.0:1787827083.159636:0:89691:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee551 from it ffffab3ac1897cb8 00010000:00010000:0.0:1787827083.159639:0:89691:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7ee551 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e3c6b expref: -99 pid: 87899 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00010000:00010000:0.0:1787827083.159646:0:89691:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7ee551 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e3c6b expref: -99 pid: 87899 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 1 00000080:00200000:0.0:1787827083.159745:0:89691:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:0.0:1787827083.159754:0:89691:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827083.159760:0:89691:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7ee551 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e3c6b expref: -99 pid: 87899 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 1 00010000:00010000:0.0:1787827083.159768:0:89691:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7ee551 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e3c6b expref: -99 pid: 87899 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 1 00000080:00200000:0.0:1787827083.159774:0:89691:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827083.159777:0:89691:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:0.0:1787827083.159780:0:89691:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee551 00000080:00010000:0.0:1787827083.159783:0:89691:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee551 from it ffffab3ac1897bb0 00010000:00010000:0.0:1787827083.159786:0:89691:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7ee551 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e3c6b expref: -99 pid: 87899 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 1 00010000:00010000:0.0:1787827083.159792:0:89691:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7ee551 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e3c6b expref: -99 pid: 87899 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 1 00000080:00200000:0.0:1787827083.159801:0:89691:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 44 00000080:00200008:0.0:1787827083.159807:0:89691:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 100000 00000080:00200080:0.0:1787827083.159882:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00800000:00000080:0.0:1787827083.159896:0:89691:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00800000:00000004:0.0:1787827083.159903:0:89691:0:(obd_class.h:1091:obd_statfs()) lustre-MDT0000-mdc-ffff985602bc9800: age 3165, max_age 3232 00000100:00100000:0.0:1787827083.159950:0:89691:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6580 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:89691:1874669228191360:192.168.203.114@tcp:41:lfs.0 00000100:00100000:0.0:1787827083.160022:0:89691:0:(client.c:2715:ptlrpc_set_wait()) set ffff9856057da240 going to sleep for 11 seconds 00000100:00100000:0.0:1787827083.161955:0:89691:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 219043334908 gen 9 00000100:00100000:0.0:1787827083.162878:0:89691:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6580 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:89691:1874669228191360:192.168.203.114@tcp:41:lfs.0 00800000:00000004:0.0:1787827083.162890:0:89691:0:(obd_class.h:1117:obd_statfs()) lustre-MDT0000-mdc-ffff985602bc9800: update ffff985606a337f0 cache blocks 317309/353529 objects 1023688/1024000 00000080:00200080:0.0:1787827083.162951:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00800000:00000080:0.0:1787827083.162964:0:89691:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00800000:00000004:0.0:1787827083.162971:0:89691:0:(obd_class.h:1091:obd_statfs()) lustre-MDT0001-mdc-ffff985602bc9800: age 3165, max_age 3232 00000100:00100000:0.0:1787827083.162997:0:89691:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6580 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:89691:1874669228191488:192.168.203.114@tcp:41:lfs.0 00000100:00100000:0.0:1787827083.163062:0:89691:0:(client.c:2715:ptlrpc_set_wait()) set ffff9856057da240 going to sleep for 11 seconds 00000100:00100000:0.0:1787827083.164315:0:89691:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6580 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:89691:1874669228191488:192.168.203.114@tcp:41:lfs.0 00800000:00000004:0.0:1787827083.164332:0:89691:0:(obd_class.h:1117:obd_statfs()) lustre-MDT0001-mdc-ffff985602bc9800: update ffff98560952b7f0 cache blocks 317401/353529 objects 1023737/1024000 00000080:00200080:0.0:1787827083.164377:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00800000:00000080:0.0:1787827083.164389:0:89691:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.164396:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00800000:00000080:0.0:1787827083.164401:0:89691:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.164406:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00800000:00000080:0.0:1787827083.164426:0:89691:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.164432:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00800000:00000080:0.0:1787827083.164437:0:89691:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.164441:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00800000:00000080:0.0:1787827083.164446:0:89691:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.164451:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00800000:00000080:0.0:1787827083.164456:0:89691:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.164460:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00800000:00000080:0.0:1787827083.164465:0:89691:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.164469:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00800000:00000080:0.0:1787827083.164473:0:89691:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.164476:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00800000:00000080:0.0:1787827083.164481:0:89691:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.164485:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00800000:00000080:0.0:1787827083.164488:0:89691:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.164492:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00800000:00000080:0.0:1787827083.164495:0:89691:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.164499:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00800000:00000080:0.0:1787827083.164503:0:89691:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.164506:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00800000:00000080:0.0:1787827083.164510:0:89691:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.164514:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00800000:00000080:0.0:1787827083.164517:0:89691:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.164521:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00800000:00000080:0.0:1787827083.164525:0:89691:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.164529:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00800000:00000080:0.0:1787827083.164533:0:89691:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.164537:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00800000:00000080:0.0:1787827083.164540:0:89691:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.164544:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00800000:00000080:0.0:1787827083.164548:0:89691:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.164552:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00800000:00000080:0.0:1787827083.164556:0:89691:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.164559:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00800000:00000080:0.0:1787827083.164563:0:89691:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.164567:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00800000:00000080:0.0:1787827083.164570:0:89691:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.164575:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00800000:00000080:0.0:1787827083.164578:0:89691:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.164583:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00800000:00000080:0.0:1787827083.164586:0:89691:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.164589:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00800000:00000080:0.0:1787827083.164593:0:89691:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.164596:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00800000:00000080:0.0:1787827083.164600:0:89691:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.164604:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00800000:00000080:0.0:1787827083.164607:0:89691:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.164611:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00800000:00000080:0.0:1787827083.164614:0:89691:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.164618:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00800000:00000080:0.0:1787827083.164622:0:89691:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.164626:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00800000:00000080:0.0:1787827083.164629:0:89691:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.164633:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00800000:00000080:0.0:1787827083.164636:0:89691:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.164640:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00800000:00000080:0.0:1787827083.164643:0:89691:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.164648:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00800000:00000080:0.0:1787827083.164651:0:89691:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.164656:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00800000:00000080:0.0:1787827083.164665:0:89691:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.164671:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00800000:00000080:0.0:1787827083.164676:0:89691:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.164680:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00800000:00000080:0.0:1787827083.164683:0:89691:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.164688:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00800000:00000080:0.0:1787827083.164728:0:89691:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.164736:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00800000:00000080:0.0:1787827083.164740:0:89691:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.164746:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00800000:00000080:0.0:1787827083.164750:0:89691:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.164754:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00800000:00000080:0.0:1787827083.164758:0:89691:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.164762:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00800000:00000080:0.0:1787827083.164765:0:89691:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.164769:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00800000:00000080:0.0:1787827083.164771:0:89691:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.164775:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00800000:00000080:0.0:1787827083.164778:0:89691:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.164782:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00800000:00000080:0.0:1787827083.164785:0:89691:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.164788:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00800000:00000080:0.0:1787827083.164791:0:89691:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.164795:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00800000:00000080:0.0:1787827083.164799:0:89691:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.164803:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00800000:00000080:0.0:1787827083.164806:0:89691:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.164811:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00800000:00000080:0.0:1787827083.164814:0:89691:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.164818:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00800000:00000080:0.0:1787827083.164821:0:89691:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.164825:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00800000:00000080:0.0:1787827083.164829:0:89691:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.164832:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00800000:00000080:0.0:1787827083.164836:0:89691:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.164840:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00800000:00000080:0.0:1787827083.164843:0:89691:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.164847:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00800000:00000080:0.0:1787827083.164851:0:89691:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.164855:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00800000:00000080:0.0:1787827083.164858:0:89691:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.164862:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00800000:00000080:0.0:1787827083.164866:0:89691:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.164870:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00800000:00000080:0.0:1787827083.164873:0:89691:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.164877:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00800000:00000080:0.0:1787827083.164880:0:89691:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.164884:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00800000:00000080:0.0:1787827083.164888:0:89691:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.164892:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00800000:00000080:0.0:1787827083.164896:0:89691:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.164900:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00800000:00000080:0.0:1787827083.164903:0:89691:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.164907:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00800000:00000080:0.0:1787827083.164910:0:89691:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.164914:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00800000:00000080:0.0:1787827083.164917:0:89691:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.164921:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00800000:00000080:0.0:1787827083.164924:0:89691:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.164928:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00800000:00000080:0.0:1787827083.164931:0:89691:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.164935:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00020000:00000080:0.0:1787827083.164942:0:89691:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00020000:00000004:0.0:1787827083.164950:0:89691:0:(obd_class.h:1091:obd_statfs()) lustre-OST0000-osc-ffff985602bc9800: age 3165, max_age 3232 00000100:00080000:0.0:1787827083.164961:0:89691:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3187 00000100:00080000:0.0:1787827083.164972:0:89691:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff985602bc9800: import ffff9856110b0000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787827083.164982:0:89691:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683216 ocd_grant: 8388608, lost: 0. 00000100:00100000:0.0:1787827083.165002:0:89691:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df70c0 x1874669228191616/t0(0) o8->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827083.165045:0:89691:0:(client.c:1845:ptlrpc_send_new_req()) @@@ req waiting for recovery: (FULL != CONNECTING) req@ffff985607df6580 x1874669228191744/t0(0) o13->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:7/4 lens 224/368 e 0 to 0 dl 0 ref 2 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827083.165054:0:89691:0:(client.c:2715:ptlrpc_set_wait()) set ffff9856057da240 going to sleep for 0 seconds 00000100:00100000:3.0:1787827083.165076:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228191616:192.168.203.114@tcp:8:lfs.0 00000100:00100000:0.0:1787827083.167644:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff985602bc9800: committing for last_committed 8589935703 gen 6 00000100:00080000:0.0:1787827083.167657:0:2382:0:(import.c:1132:ptlrpc_connect_interpret()) lustre-OST0000-osc-ffff985602bc9800: connect to target with instance 3 00000100:00080000:0.0:1787827083.167665:0:2382:0:(import.c:956:ptlrpc_connect_set_flags()) lustre-OST0000-osc-ffff985602bc9800: Resetting ns_connect_flags to server flags: 0xa0427af2e3440038 00000100:00080000:0.0:1787827083.167677:0:2382:0:(import.c:1213:ptlrpc_connect_interpret()) connected to replayable target: lustre-OST0000_UUID 00000100:00080000:0.0:1787827083.167680:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9856110b0000 lustre-OST0000_UUID: changing import state from CONNECTING to FULL 00000080:00000004:0.0:1787827083.167689:0:2382:0:(lcommon_misc.c:94:cl_ocd_update()) Changing connect_flags: 0xa0427af2e3440038 -> 0xa0427af2e3440038 00000080:00080000:0.0:1787827083.167715:0:2382:0:(lcommon_misc.c:56:cl_init_ea_size()) updating def/max_easize: 72/65536 00000100:00080000:0.0:1787827083.167728:0:2382:0:(recover.c:191:ptlrpc_wake_delayed()) @@@ waking (set 000000003466f79f): req@ffff985607df6580 x1874669228191744/t0(0) o13->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:7/4 lens 224/368 e 0 to 0 dl 0 ref 2 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827083.167755:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228191616:192.168.203.114@tcp:8:lfs.0 00000100:00100000:3.0:1787827083.167913:0:89691:0:(client.c:2715:ptlrpc_set_wait()) set ffff9856057da240 going to sleep for 11 seconds 00000100:00100000:0.0:1787827083.169454:0:89691:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6580 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:89691:1874669228191744:192.168.203.114@tcp:13:lfs.0 00020000:00000004:0.0:1787827083.169475:0:89691:0:(obd_class.h:1117:obd_statfs()) lustre-OST0000-osc-ffff985602bc9800: update ffff985609528450 cache blocks 901356/958279 objects 261512/262144 00000080:00200080:0.0:1787827083.169542:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00020000:00000080:0.0:1787827083.169561:0:89691:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00020000:00000004:0.0:1787827083.169573:0:89691:0:(obd_class.h:1091:obd_statfs()) lustre-OST0001-osc-ffff985602bc9800: age 3165, max_age 3232 00000100:00080000:0.0:1787827083.169583:0:89691:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 922 00000100:00080000:0.0:1787827083.169591:0:89691:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff985602bc9800: import ffff9856110b1000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787827083.169600:0:89691:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683216 ocd_grant: 491413504, lost: 0. 00000100:00100000:0.0:1787827083.169621:0:89691:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df70c0 x1874669228191872/t0(0) o8->lustre-OST0001-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827083.169669:0:89691:0:(client.c:1845:ptlrpc_send_new_req()) @@@ req waiting for recovery: (FULL != CONNECTING) req@ffff985607df6580 x1874669228192000/t0(0) o13->lustre-OST0001-osc-ffff985602bc9800@192.168.203.114@tcp:7/4 lens 224/368 e 0 to 0 dl 0 ref 2 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827083.169678:0:89691:0:(client.c:2715:ptlrpc_set_wait()) set ffff9856057da240 going to sleep for 0 seconds 00000100:00100000:3.0:1787827083.170202:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228191872:192.168.203.114@tcp:8:lfs.0 00000100:00100000:0.0:1787827083.171820:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0001-osc-ffff985602bc9800: committing for last_committed 4294967636 gen 6 00000100:00080000:0.0:1787827083.171831:0:2382:0:(import.c:1132:ptlrpc_connect_interpret()) lustre-OST0001-osc-ffff985602bc9800: connect to target with instance 1 00000100:00080000:0.0:1787827083.171839:0:2382:0:(import.c:956:ptlrpc_connect_set_flags()) lustre-OST0001-osc-ffff985602bc9800: Resetting ns_connect_flags to server flags: 0xa0427af2e3440038 00000100:00080000:0.0:1787827083.171849:0:2382:0:(import.c:1213:ptlrpc_connect_interpret()) connected to replayable target: lustre-OST0001_UUID 00000100:00080000:0.0:1787827083.171853:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9856110b1000 lustre-OST0001_UUID: changing import state from CONNECTING to FULL 00000080:00000004:0.0:1787827083.171861:0:2382:0:(lcommon_misc.c:94:cl_ocd_update()) Changing connect_flags: 0xa0427af2e3440038 -> 0xa0427af2e3440038 00000080:00080000:0.0:1787827083.171866:0:2382:0:(lcommon_misc.c:56:cl_init_ea_size()) updating def/max_easize: 72/65536 00000100:00080000:0.0:1787827083.171877:0:2382:0:(recover.c:191:ptlrpc_wake_delayed()) @@@ waking (set 000000003466f79f): req@ffff985607df6580 x1874669228192000/t0(0) o13->lustre-OST0001-osc-ffff985602bc9800@192.168.203.114@tcp:7/4 lens 224/368 e 0 to 0 dl 0 ref 2 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827083.171900:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228191872:192.168.203.114@tcp:8:lfs.0 00000100:00100000:3.0:1787827083.171980:0:89691:0:(client.c:2715:ptlrpc_set_wait()) set ffff9856057da240 going to sleep for 11 seconds 00000100:00100000:0.0:1787827083.172814:0:89691:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6580 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:89691:1874669228192000:192.168.203.114@tcp:13:lfs.0 00020000:00000004:0.0:1787827083.172829:0:89691:0:(obd_class.h:1117:obd_statfs()) lustre-OST0001-osc-ffff985602bc9800: update ffff98560952d1c0 cache blocks 901363/958279 objects 261551/262144 00000080:00200080:0.0:1787827083.172881:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00020000:00000080:0.0:1787827083.172894:0:89691:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.172901:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00020000:00000080:0.0:1787827083.172906:0:89691:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.172910:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00020000:00000080:0.0:1787827083.172914:0:89691:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.172918:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00020000:00000080:0.0:1787827083.172921:0:89691:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.172926:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00020000:00000080:0.0:1787827083.172930:0:89691:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.172934:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00020000:00000080:0.0:1787827083.172937:0:89691:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.172955:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00020000:00000080:0.0:1787827083.172960:0:89691:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.172965:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00020000:00000080:0.0:1787827083.172969:0:89691:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.172974:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00020000:00000080:0.0:1787827083.172978:0:89691:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.172982:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00020000:00000080:0.0:1787827083.172985:0:89691:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.172989:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00020000:00000080:0.0:1787827083.172992:0:89691:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.172996:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00020000:00000080:0.0:1787827083.172999:0:89691:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.173003:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00020000:00000080:0.0:1787827083.173006:0:89691:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.173009:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00020000:00000080:0.0:1787827083.173012:0:89691:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.173024:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00020000:00000080:0.0:1787827083.173028:0:89691:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.173032:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00020000:00000080:0.0:1787827083.173040:0:89691:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.173044:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00020000:00000080:0.0:1787827083.173048:0:89691:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.173051:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00020000:00000080:0.0:1787827083.173055:0:89691:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.173058:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00020000:00000080:0.0:1787827083.173062:0:89691:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.173066:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00020000:00000080:0.0:1787827083.173069:0:89691:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.173073:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00020000:00000080:0.0:1787827083.173076:0:89691:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.173080:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00020000:00000080:0.0:1787827083.173084:0:89691:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.173088:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00020000:00000080:0.0:1787827083.173092:0:89691:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.173095:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00020000:00000080:0.0:1787827083.173099:0:89691:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.173102:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00020000:00000080:0.0:1787827083.173106:0:89691:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.173110:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00020000:00000080:0.0:1787827083.173113:0:89691:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.173117:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00020000:00000080:0.0:1787827083.173121:0:89691:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.173124:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00020000:00000080:0.0:1787827083.173128:0:89691:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.173131:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00020000:00000080:0.0:1787827083.173135:0:89691:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.173139:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00020000:00000080:0.0:1787827083.173142:0:89691:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.173146:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00020000:00000080:0.0:1787827083.173148:0:89691:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.173152:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00020000:00000080:0.0:1787827083.173155:0:89691:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.173159:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00020000:00000080:0.0:1787827083.173162:0:89691:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.173166:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00020000:00000080:0.0:1787827083.173169:0:89691:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.173173:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00020000:00000080:0.0:1787827083.173176:0:89691:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.173180:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00020000:00000080:0.0:1787827083.173183:0:89691:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.173187:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00020000:00000080:0.0:1787827083.173190:0:89691:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.173194:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00020000:00000080:0.0:1787827083.173198:0:89691:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.173201:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00020000:00000080:0.0:1787827083.173204:0:89691:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.173208:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00020000:00000080:0.0:1787827083.173211:0:89691:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.173215:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00020000:00000080:0.0:1787827083.173218:0:89691:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.173222:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00020000:00000080:0.0:1787827083.173225:0:89691:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.173229:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00020000:00000080:0.0:1787827083.173232:0:89691:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.173237:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00020000:00000080:0.0:1787827083.173240:0:89691:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.173243:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00020000:00000080:0.0:1787827083.173246:0:89691:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.173250:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00020000:00000080:0.0:1787827083.173254:0:89691:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.173257:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00020000:00000080:0.0:1787827083.173262:0:89691:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.173265:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00020000:00000080:0.0:1787827083.173269:0:89691:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.173272:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00020000:00000080:0.0:1787827083.173276:0:89691:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.173279:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00020000:00000080:0.0:1787827083.173283:0:89691:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.173286:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00020000:00000080:0.0:1787827083.173290:0:89691:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.173294:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00020000:00000080:0.0:1787827083.173297:0:89691:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.173301:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00020000:00000080:0.0:1787827083.173311:0:89691:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.173316:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00020000:00000080:0.0:1787827083.173320:0:89691:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.173325:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00020000:00000080:0.0:1787827083.173328:0:89691:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.173332:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00020000:00000080:0.0:1787827083.173335:0:89691:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.173338:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00020000:00000080:0.0:1787827083.173341:0:89691:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.173345:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00020000:00000080:0.0:1787827083.173348:0:89691:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.173352:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00020000:00000080:0.0:1787827083.173355:0:89691:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.173359:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00020000:00000080:0.0:1787827083.173362:0:89691:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.173366:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00020000:00000080:0.0:1787827083.173370:0:89691:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.173373:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00020000:00000080:0.0:1787827083.173377:0:89691:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200080:0.0:1787827083.173381:0:89691:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffde6568860 00020000:00000080:0.0:1787827083.173384:0:89691:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608614c00 uarg=0000000000000000 00000080:00200008:0.0:1787827083.173397:0:89691:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 100000 00000080:00200000:3.0:1787827083.191110:0:89698:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:3.0:1787827083.191133:0:89698:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787827083.191149:0:89698:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7ee551 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e3c6b expref: -99 pid: 87899 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 1 00010000:00010000:3.0:1787827083.191163:0:89698:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7ee551 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e3c6b expref: -99 pid: 87899 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 1 00000080:00200000:3.0:1787827083.191171:0:89698:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787827083.191178:0:89698:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:3.0:1787827083.191181:0:89698:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee551 00000080:00010000:3.0:1787827083.191188:0:89698:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee551 from it ffffab3ac1c77bd0 00010000:00010000:3.0:1787827083.191192:0:89698:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7ee551 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e3c6b expref: -99 pid: 87899 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 1 00010000:00010000:3.0:1787827083.191202:0:89698:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7ee551 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e3c6b expref: -99 pid: 87899 timeout: 0 lvb_type: 0 lru_score: 9 lru_type: 1 00000080:00200000:3.0:1787827083.191216:0:89698:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:3.0:1787827083.191226:0:89698:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=fsa-oleg314-client.virtnet, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), flags=1536 00000080:00200000:3.0:1787827083.191228:0:89698:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:3.0:1787827083.191233:0:89698:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787827083.191238:0:89698:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7ee551 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e3c6b expref: -99 pid: 87899 timeout: 0 lvb_type: 0 lru_score: 9 lru_type: 1 00010000:00010000:3.0:1787827083.191243:0:89698:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7ee551 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e3c6b expref: -99 pid: 87899 timeout: 0 lvb_type: 0 lru_score: 9 lru_type: 1 00000080:00200000:3.0:1787827083.191247:0:89698:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787827083.191250:0:89698:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:3.0:1787827083.191252:0:89698:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee551 00000080:00010000:3.0:1787827083.191254:0:89698:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee551 from it ffffab3ac1c77cd0 00010000:00010000:3.0:1787827083.191256:0:89698:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7ee551 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e3c6b expref: -99 pid: 87899 timeout: 0 lvb_type: 0 lru_score: 9 lru_type: 1 00010000:00010000:3.0:1787827083.191261:0:89698:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7ee551 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e3c6b expref: -99 pid: 87899 timeout: 0 lvb_type: 0 lru_score: 10 lru_type: 1 00000080:00200000:3.0:1787827083.191265:0:89698:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 3 00000080:00200000:3.0:1787827083.191268:0:89698:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:3.0:1787827083.191270:0:89698:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787827083.191273:0:89698:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7ee551 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e3c6b expref: -99 pid: 87899 timeout: 0 lvb_type: 0 lru_score: 10 lru_type: 1 00010000:00010000:3.0:1787827083.191277:0:89698:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7ee551 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e3c6b expref: -99 pid: 87899 timeout: 0 lvb_type: 0 lru_score: 10 lru_type: 1 00000080:00200000:3.0:1787827083.191281:0:89698:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787827083.191283:0:89698:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:3.0:1787827083.191285:0:89698:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee551 00000080:00010000:3.0:1787827083.191286:0:89698:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee551 from it ffffab3ac1c77db0 00010000:00010000:3.0:1787827083.191288:0:89698:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7ee551 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e3c6b expref: -99 pid: 87899 timeout: 0 lvb_type: 0 lru_score: 10 lru_type: 1 00010000:00010000:3.0:1787827083.191292:0:89698:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7ee551 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e3c6b expref: -99 pid: 87899 timeout: 0 lvb_type: 0 lru_score: 11 lru_type: 1 00000080:00200000:3.0:1787827083.191296:0:89698:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 3 00000080:00200000:3.0:1787827083.191299:0:89698:0:(namei.c:2129:ll_create_nd()) VFS Op:name=fsa-oleg314-client.virtnet, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), flags=33188, excl=1 00000080:00200000:3.0:1787827083.191306:0:89698:0:(namei.c:2084:ll_mknod()) VFS Op:name=fsa-oleg314-client.virtnet, dir=[0x200000007:0x1:0x0](ffff98561158e3c8) mode 100644 dev 0 00010000:00010000:3.0:1787827083.191324:0:89698:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7ee551 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x5aa6d504d91e3c6b expref: -99 pid: 87899 timeout: 0 lvb_type: 0 lru_score: 11 lru_type: 1 00010000:00010000:3.0:1787827083.191387:0:89698:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7ee551 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91e3c6b expref: -99 pid: 87899 timeout: 0 lvb_type: 0 lru_score: 11 lru_type: 1 00010000:00010000:3.0:1787827083.191394:0:89698:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787827083.191397:0:89698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e07600/0xb3f83dad1b7ee551 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91e3c6b expref: -99 pid: 87899 timeout: 0 lvb_type: 0 lru_score: 11 lru_type: 1 00000020:00100000:3.0:1787827083.191432:0:89698:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:3.0:1787827083.191445:0:89698:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985610f821c0 x1874669228192256/t0(0) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 528/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827083.191468:0:89698:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f821c0 pname:cluuid:pid:xid:nid:opc:job mcreate:7b4e8535-9db7-485d-abf6-bda966b63320:89698:1874669228192256:192.168.203.114@tcp:36:mcreate.0 00000100:00100000:3.0:1787827083.191529:0:89698:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607ff09c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827083.193269:0:89698:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f821c0 pname:cluuid:pid:xid:nid:opc:job mcreate:7b4e8535-9db7-485d-abf6-bda966b63320:89698:1874669228192256:192.168.203.114@tcp:36:mcreate.0 00000080:00200000:1.0:1787827083.193306:0:89698:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x2000182d1:0x966:0x0](ffff98561014ba88) 00000080:00200000:1.0:1787827083.193341:0:89698:0:(namei.c:2137:ll_create_nd()) VFS Op:name=fsa-oleg314-client.virtnet, unhashed 1 00000080:00200000:0.0:1787827083.199820:0:89701:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:0.0:1787827083.199843:0:89701:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827083.199896:0:89701:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06a00/0xb3f83dad1b7ee55f lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 89701 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827083.199907:0:89701:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06a00/0xb3f83dad1b7ee55f lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 89701 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827083.199917:0:89701:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06a00/0xb3f83dad1b7ee55f lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 89701 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787827083.199937:0:89701:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6580 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:89701:1874669228192384:192.168.203.114@tcp:101:rm.0 00000100:00100000:0.0:1787827083.200006:0:89701:0:(client.c:2715:ptlrpc_set_wait()) set ffff9856057da240 going to sleep for 11 seconds 00000100:00100000:0.0:1787827083.201402:0:89701:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6580 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:89701:1874669228192384:192.168.203.114@tcp:101:rm.0 00010000:00010000:0.0:1787827083.201428:0:89701:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06a00/0xb3f83dad1b7ee55f lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e479a expref: -99 pid: 89701 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827083.201436:0:89701:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06a00/0xb3f83dad1b7ee55f lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e479a expref: -99 pid: 89701 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827083.201446:0:89701:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06a00/0xb3f83dad1b7ee55f lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e479a expref: -99 pid: 89701 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827083.201456:0:89701:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06a00/0xb3f83dad1b7ee55f lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e479a expref: -99 pid: 89701 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827083.201466:0:89701:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06a00/0xb3f83dad1b7ee55f lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e479a expref: -99 pid: 89701 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827083.201474:0:89701:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06a00/0xb3f83dad1b7ee55f lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e479a expref: -99 pid: 89701 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827083.201481:0:89701:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06a00/0xb3f83dad1b7ee55f lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e479a expref: -99 pid: 89701 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787827083.201493:0:89701:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff985607df6580 x1874669228192384/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 576/688 e 0 to 0 dl 1787827099 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787827083.201505:0:89701:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06a00/0xb3f83dad1b7ee55f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e479a expref: -99 pid: 89701 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827083.201513:0:89701:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:0.0:1787827083.201518:0:89701:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:0.0:1787827083.201538:0:89701:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 69632 00000080:00010000:0.0:1787827083.201543:0:89701:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee55f 00000080:00010000:0.0:1787827083.201549:0:89701:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:0.0:1787827083.201552:0:89701:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee55f from it ffffab3ac1c77b58 00010000:00010000:0.0:1787827083.201555:0:89701:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06a00/0xb3f83dad1b7ee55f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e479a expref: -99 pid: 89701 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827083.201563:0:89701:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06a00/0xb3f83dad1b7ee55f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e479a expref: -99 pid: 89701 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787827083.201582:0:89701:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:0.0:1787827083.201604:0:89701:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=fsa-oleg314-client.virtnet, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), flags=8 00000080:00200000:0.0:1787827083.201607:0:89701:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=fsa-oleg314-client.virtnet, dir=[0x200000007:0x1:0x0](ffff98561158e3c8), intent=getattr 00000002:00010000:0.0:1787827083.201621:0:89701:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: fsa-oleg314-client.virtnet,[0x0:0x0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787827083.201651:0:89701:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7ee566 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 89701 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827083.201659:0:89701:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7ee566 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 89701 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827083.201667:0:89701:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7ee566 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 89701 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787827083.201681:0:89701:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6580 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:89701:1874669228192512:192.168.203.114@tcp:101:rm.0 00000100:00100000:0.0:1787827083.201750:0:89701:0:(client.c:2715:ptlrpc_set_wait()) set ffff9856057da240 going to sleep for 11 seconds 00000100:00100000:0.0:1787827083.203010:0:89701:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6580 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:89701:1874669228192512:192.168.203.114@tcp:101:rm.0 00010000:00010000:0.0:1787827083.203023:0:89701:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7ee566 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e47b6 expref: -99 pid: 89701 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827083.203039:0:89701:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7ee566 lrc: 4/1,0 mode: --/PR res: [0x2000182d1:0x966:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e47b6 expref: -99 pid: 89701 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827083.203052:0:89701:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7ee566 lrc: 4/1,0 mode: --/PR res: [0x2000182d1:0x966:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e47b6 expref: -99 pid: 89701 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827083.203060:0:89701:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7ee566 lrc: 4/1,0 mode: PR/PR res: [0x2000182d1:0x966:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e47b6 expref: -99 pid: 89701 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827083.203069:0:89701:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7ee566 lrc: 4/1,0 mode: PR/PR res: [0x2000182d1:0x966:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e47b6 expref: -99 pid: 89701 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827083.203079:0:89701:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7ee566 lrc: 5/2,0 mode: PR/PR res: [0x2000182d1:0x966:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e47b6 expref: -99 pid: 89701 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827083.203086:0:89701:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7ee566 lrc: 5/2,0 mode: PR/PR res: [0x2000182d1:0x966:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e47b6 expref: -99 pid: 89701 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827083.203093:0:89701:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7ee566 lrc: 4/1,0 mode: PR/PR res: [0x2000182d1:0x966:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e47b6 expref: -99 pid: 89701 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787827083.203102:0:89701:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff985607df6580 x1874669228192512/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 600/584 e 0 to 0 dl 1787827099 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787827083.203113:0:89701:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7ee566 lrc: 3/1,0 mode: PR/PR res: [0x2000182d1:0x966:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e47b6 expref: -99 pid: 89701 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827083.203121:0:89701:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8590033617/2406 (0 0) 00000080:00200000:0.0:1787827083.203126:0:89701:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:0.0:1787827083.203147:0:89701:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x2000182d1:0x966:0x0], updating i_size 0 00000080:00200000:0.0:1787827083.203151:0:89701:0:(namei.c:147:ll_iget()) got inode: [0x2000182d1:0x966:0x0](ffff98561014ba88): rc = 0 00000080:00200000:0.0:1787827083.203167:0:89701:0:(file.c:6854:ll_layout_conf()) [0x2000182d1:0x966:0x0]: layout version change: 4294967294 -> 4294967295 00000080:00010000:0.0:1787827083.203173:0:89701:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x2000182d1:0x966:0x0](ffff98561014ba88) for lock 0xb3f83dad1b7ee566 00000080:00010000:0.0:1787827083.203181:0:89701:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x2000182d1:0x966:0x0](ffff98561014ba88) 00000080:00010000:0.0:1787827083.203184:0:89701:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee566 from it ffffab3ac1c77b20 00010000:00010000:0.0:1787827083.203188:0:89701:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7ee566 lrc: 3/1,0 mode: PR/PR res: [0x2000182d1:0x966:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91e47b6 expref: -99 pid: 89701 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827083.203203:0:89701:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7ee566 lrc: 2/0,0 mode: PR/PR res: [0x2000182d1:0x966:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91e47b6 expref: -99 pid: 89701 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:0.0:1787827083.203226:0:89701:0:(file.c:6313:ll_getattr_dentry()) START file fsa-oleg314-client.virtnet:[0x2000182d1:0x966:0x0](ffff98561014ba88), request_mask 2047, flags 0, foreign 0 00000080:00200000:0.0:1787827083.203232:0:89701:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x2000182d1:0x966:0x0](ffff98561014ba88),name=fsa-oleg314-client.virtnet 00000002:00010000:0.0:1787827083.203245:0:89701:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x2000182d1:0x966:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827083.203254:0:89701:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7ee566 lrc: 2/1,0 mode: PR/PR res: [0x2000182d1:0x966:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91e47b6 expref: -99 pid: 89701 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787827083.203271:0:89701:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7ee566 lrc: 2/1,0 mode: PR/PR res: [0x2000182d1:0x966:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91e47b6 expref: -99 pid: 89701 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787827083.203279:0:89701:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827083.203283:0:89701:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x2000182d1:0x966:0x0](ffff98561014ba88) 00000080:00010000:0.0:1787827083.203286:0:89701:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x2000182d1:0x966:0x0](ffff98561014ba88) for lock 0xb3f83dad1b7ee566 00000080:00010000:0.0:1787827083.203290:0:89701:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee566 from it ffffab3ac1c77cb8 00010000:00010000:0.0:1787827083.203293:0:89701:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7ee566 lrc: 3/1,0 mode: PR/PR res: [0x2000182d1:0x966:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x5aa6d504d91e47b6 expref: -99 pid: 89701 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787827083.203301:0:89701:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7ee566 lrc: 2/0,0 mode: PR/PR res: [0x2000182d1:0x966:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x5aa6d504d91e47b6 expref: -99 pid: 89701 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1787827083.203342:0:89701:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:0.0:1787827083.203349:0:89701:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827083.203356:0:89701:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06a00/0xb3f83dad1b7ee55f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e479a expref: -99 pid: 89701 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787827083.203364:0:89701:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06a00/0xb3f83dad1b7ee55f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e479a expref: -99 pid: 89701 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787827083.203371:0:89701:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827083.203375:0:89701:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:0.0:1787827083.203378:0:89701:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee55f 00000080:00010000:0.0:1787827083.203382:0:89701:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee55f from it ffffab3ac1c77c10 00010000:00010000:0.0:1787827083.203385:0:89701:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06a00/0xb3f83dad1b7ee55f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e479a expref: -99 pid: 89701 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787827083.203393:0:89701:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06a00/0xb3f83dad1b7ee55f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e479a expref: -99 pid: 89701 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1787827083.203421:0:89701:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 1 00000080:00200000:0.0:1787827083.203428:0:89701:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=fsa-oleg314-client.virtnet, flags=0 00000080:00200000:0.0:1787827083.203431:0:89701:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:0.0:1787827083.203436:0:89701:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827083.203441:0:89701:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06a00/0xb3f83dad1b7ee55f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e479a expref: -99 pid: 89701 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1787827083.203446:0:89701:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06a00/0xb3f83dad1b7ee55f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e479a expref: -99 pid: 89701 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1787827083.203451:0:89701:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827083.203453:0:89701:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:0.0:1787827083.203455:0:89701:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee55f 00000080:00010000:0.0:1787827083.203457:0:89701:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee55f from it ffffab3ac1c77d68 00010000:00010000:0.0:1787827083.203459:0:89701:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06a00/0xb3f83dad1b7ee55f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e479a expref: -99 pid: 89701 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1787827083.203463:0:89701:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06a00/0xb3f83dad1b7ee55f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e479a expref: -99 pid: 89701 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:0.0:1787827083.203467:0:89701:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 3 00000080:00200000:0.0:1787827083.203471:0:89701:0:(namei.c:2418:ll_unlink()) VFS Op:name=fsa-oleg314-client.virtnet, dir=[0x200000007:0x1:0x0](ffff98561158e3c8) 00010000:00010000:0.0:1787827083.203484:0:89701:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06a00/0xb3f83dad1b7ee55f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x5aa6d504d91e479a expref: -99 pid: 89701 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827083.203503:0:89701:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7ee566 lrc: 2/0,0 mode: PR/PR res: [0x2000182d1:0x966:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x5aa6d504d91e47b6 expref: -99 pid: 89701 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1787827083.203512:0:89701:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x966:0x0]: losing layout lock 00010000:00010000:0.0:1787827083.203532:0:89701:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7ee566 lrc: 1/0,0 mode: --/PR res: [0x2000182d1:0x966:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x5aa6d504d91e47b6 expref: -99 pid: 89701 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1787827083.203540:0:89701:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06a00/0xb3f83dad1b7ee55f lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91e479a expref: -99 pid: 89701 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827083.203545:0:89701:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 2 locks packed 00010000:00010000:0.0:1787827083.203548:0:89701:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06200/0xb3f83dad1b7ee566 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x966:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x5aa6d504d91e47b6 expref: -99 pid: 89701 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1787827083.203561:0:89701:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06a00/0xb3f83dad1b7ee55f lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x5aa6d504d91e479a expref: -99 pid: 89701 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000020:00100000:0.0:1787827083.203576:0:89701:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff985602bc9800: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:0.0:1787827083.203584:0:89701:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff985607df6580 x1874669228192768/t0(0) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827083.203595:0:89701:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6580 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:89701:1874669228192768:192.168.203.114@tcp:36:rm.0 00000100:00100000:0.0:1787827083.203629:0:89701:0:(client.c:2715:ptlrpc_set_wait()) set ffff9856057da240 going to sleep for 11 seconds 00000100:00100000:0.0:1787827083.205173:0:89701:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6580 pname:cluuid:pid:xid:nid:opc:job rm:7b4e8535-9db7-485d-abf6-bda966b63320:89701:1874669228192768:192.168.203.114@tcp:36:rm.0 00000080:00200000:0.0:1787827083.205212:0:89701:0:(vvp_io.c:1885:vvp_io_init()) [0x2000182d1:0x966:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:0.0:1787827083.205221:0:89701:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x966:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787827083.205228:0:89701:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x966:0x0](ffff98561014ba88) 00000100:00100000:0.0:1787827083.508916:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:3c946890-15ca-497a-b838-fe1cba61f4dd:2384:1874669228192896:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827083.508935:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669228193024:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827083.509011:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669228193280:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787827083.509013:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669228193152:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787827083.509056:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669228193408:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787827083.510185:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:3c946890-15ca-497a-b838-fe1cba61f4dd:2384:1874669228192896:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827083.510219:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669228193024:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827083.510240:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669228193280:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787827083.510247:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669228193152:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787827083.510264:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669228193408:192.168.203.114@tcp:400:kworker.0 00010000:00010000:0.0:1787827084.084874:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827085.108932:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:1.0:1787827085.425074:0:89766:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: local REPLAY BARRIER on lustre-MDT0000 00010000:00010000:1.0:1787827086.133466:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827087.156972:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827088.180855:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827088.628885:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2383:1874669228193536:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787827088.628889:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669228193664:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787827088.628968:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669228193920:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827088.628978:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669228193792:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787827088.629004:0:2384:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1787827088.629011:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669228194048:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787827088.630608:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669228194048:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827088.630621:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669228193792:192.168.203.114@tcp:400:kworker.0 00000100:00080000:0.0:1787827088.630640:0:2384:0:(recover.c:205:ptlrpc_request_handle_notconn()) import lustre-MDT0000-mdc-ffff985602bc9800 of lustre-MDT0000_UUID@192.168.203.114@tcp abruptly disconnected: reconnecting 00000100:00080000:0.0:1787827088.630647:0:2384:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from FULL to DISCONN 00000100:02000400:0.0:1787827088.630653:0:2384:0:(import.c:182:ptlrpc_set_import_discon()) lustre-MDT0000-mdc-ffff985602bc9800: Connection to lustre-MDT0000 (at 192.168.203.114@tcp) was lost; in progress operations using this service will wait for recovery to complete 00000100:00080000:0.0:1787827088.630661:0:2384:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827088.630667:0:2384:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3137 00000100:00080000:0.0:1787827088.630682:0:2384:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787827088.630729:0:2384:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827088.630760:0:2384:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5e00 x1874669228194176/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827088.630789:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669228193664:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787827088.630808:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669228193920:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787827088.630833:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228194176:192.168.203.114@tcp:38:ptlrpcd_00_01.0 00000100:00080000:3.0:1787827088.631829:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827088.631836:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-19) 00000100:00100000:3.0:1787827088.631841:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228194176:192.168.203.114@tcp:38:ptlrpcd_00_01.0 00010000:00010000:0.0:1787827089.204806:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827090.228984:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827091.252840:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827092.276894:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827093.300844:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827093.748908:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827093.748917:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827093.748922:0:38:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827093.748936:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3239 00000100:00080000:2.0:1787827093.748950:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 5s 00000100:00080000:2.0:1787827093.748960:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787827093.748973:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827093.748982:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:3c946890-15ca-497a-b838-fe1cba61f4dd:2386:1874669228194304:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787827093.748995:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140da1c0 x1874669228194432/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827093.749081:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228194432:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787827093.749117:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228194688:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787827093.749183:0:2386:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1787827093.749192:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228194560:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787827093.749230:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228194816:192.168.203.114@tcp:400:kworker.0 00000100:00080000:0.0:1787827093.750892:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827093.750903:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-19) 00000100:00100000:0.0:1787827093.750909:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228194432:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787827093.750921:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228194560:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787827093.750943:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228194816:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787827093.750956:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228194688:192.168.203.114@tcp:400:kworker.0 00010000:00010000:0.0:1787827094.324856:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827095.350644:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827096.372922:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827097.396825:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827098.420865:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827098.868855:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827098.868859:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827098.868862:0:38:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827098.868869:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3244 00000100:00080000:2.0:1787827098.868877:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 10s 00000100:00080000:2.0:1787827098.868882:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787827098.868891:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827098.868903:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8f00 x1874669228195072/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827098.868947:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:3c946890-15ca-497a-b838-fe1cba61f4dd:2386:1874669228194944:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787827098.868979:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669228195200:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787827098.869003:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228195072:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827098.869056:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669228195456:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787827098.869065:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228195328:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787827098.870252:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669228195200:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787827098.870279:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228195328:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787827098.870282:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669228195456:192.168.203.114@tcp:400:kworker.0 00000100:00080000:0.0:1787827098.870288:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827098.870297:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-19) 00000100:00100000:0.0:1787827098.870321:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228195072:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787827099.444821:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827100.468865:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827101.492813:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827102.516838:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827103.540856:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827103.988868:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827103.988874:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827103.988879:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827103.988890:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3249 00000100:00080000:2.0:1787827103.988901:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 15s 00000100:00080000:2.0:1787827103.988909:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787827103.988919:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827103.988919:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:3c946890-15ca-497a-b838-fe1cba61f4dd:2386:1874669228195584:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787827103.988941:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140dbc00 x1874669228195712/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827103.988980:0:37:0:(import.c:65:import_set_state_nolock()) ffff9856110b0000 lustre-OST0000_UUID: changing import state from FULL to CONNECTING 00000100:00080000:2.0:1787827103.988984:0:37:0:(import.c:2101:ptlrpc_disconnect_and_idle_import()) lustre-OST0000-osc-ffff985602bc9800: disconnect after 20s idle 00000100:00080000:2.0:1787827103.988994:0:37:0:(import.c:65:import_set_state_nolock()) ffff9856110b1000 lustre-OST0001_UUID: changing import state from FULL to CONNECTING 00000100:00080000:2.0:1787827103.988997:0:37:0:(import.c:2101:ptlrpc_disconnect_and_idle_import()) lustre-OST0001-osc-ffff985602bc9800: disconnect after 20s idle 00000100:00100000:0.0:1787827103.989010:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228195712:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827103.989032:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669228195840:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787827103.989064:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228195968:192.168.203.114@tcp:9:kworker.0 00000100:00100000:0.0:1787827103.989107:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228196096:192.168.203.114@tcp:9:kworker.0 00010000:00010000:2.0:1787827103.990475:0:2385:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 0/0 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1787827103.990492:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669228195840:192.168.203.114@tcp:400:kworker.0 00010000:00010000:0.0:1787827103.990522:0:15163:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 0 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787827103.990542:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827103.990544:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827103.990546:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827103.990663:0:2386:0:(recover.c:205:ptlrpc_request_handle_notconn()) import MGC192.168.203.114@tcp of MGS@192.168.203.114@tcp abruptly disconnected: reconnecting 00000100:00080000:3.0:1787827103.990670:0:2386:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from FULL to DISCONN 00000100:02020000:3.0:1787827103.990675:0:2386:0:(import.c:186:ptlrpc_set_import_discon()) MGC192.168.203.114@tcp: Connection to MGS (at 192.168.203.114@tcp) was lost; in progress operations using this service will fail 00000100:00080000:3.0:1787827103.990679:0:2386:0:(import.c:192:ptlrpc_set_import_discon()) import MGS@192.168.203.114@tcp for MGC192.168.203.114@tcp not replayable, auto-deactivating 00000100:00080000:3.0:1787827103.990919:0:2386:0:(import.c:138:ptlrpc_deactivate_import_nolock()) setting import MGS INVALID 00000100:00100000:3.0:1787827103.990929:0:2386:0:(client.c:3589:ptlrpc_abort_inflight()) @@@ inflight req@ffff985607df7480 x1874669228193536/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827104 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827103.991092:0:2386:0:(client.c:3589:ptlrpc_abort_inflight()) @@@ inflight req@ffff9856140db0c0 x1874669228194304/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827109 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827103.991107:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2383:1874669228193536:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787827103.991108:0:2386:0:(client.c:3589:ptlrpc_abort_inflight()) @@@ inflight req@ffff9856140da940 x1874669228194944/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827114 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827103.991131:0:2386:0:(client.c:3589:ptlrpc_abort_inflight()) @@@ inflight req@ffff9856140d8b40 x1874669228195584/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827119 ref 1 fl Rpc:RNQU/200/0 rc 0/-107 job:'kworker.0' uid:0 gid:0 projid:4294967295 10000000:01000000:3.0:1787827103.991141:0:2386:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808001 00000100:00080000:3.0:1787827103.991145:0:2386:0:(pinger.c:193:ptlrpc_pinger_ir_down()) IR down 10000000:01000000:3.0:1787827103.991147:0:2386:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808002 00000100:00080000:3.0:1787827103.991151:0:2386:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827103.991157:0:2386:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3135 00000100:00080000:3.0:1787827103.991169:0:2386:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00080000:0.0:1787827103.991229:0:2382:0:(import.c:2011:ptlrpc_disconnect_idle_interpret()) @@@ inflight=1, refcount=3: rc = 0 req@ffff9856140d9a40 x1874669228195968/t0(0) o9->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827113 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827103.991241:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9856110b0000 lustre-OST0000_UUID: changing import state from CONNECTING to IDLE 00000100:00100000:0.0:1787827103.991246:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228195968:192.168.203.114@tcp:9:kworker.0 00000100:00080000:0.0:1787827103.991276:0:2382:0:(import.c:2011:ptlrpc_disconnect_idle_interpret()) @@@ inflight=1, refcount=3: rc = 0 req@ffff9856140d9680 x1874669228196096/t0(0) o9->lustre-OST0001-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827113 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827103.991278:0:2386:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82d00 x1874669228196224/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827103.991285:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9856110b1000 lustre-OST0001_UUID: changing import state from CONNECTING to IDLE 00000100:00100000:0.0:1787827103.991288:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228196096:192.168.203.114@tcp:9:kworker.0 00000100:00100000:3.0:1787827103.991295:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:3c946890-15ca-497a-b838-fe1cba61f4dd:2386:1874669228195584:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787827103.991306:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:3c946890-15ca-497a-b838-fe1cba61f4dd:2386:1874669228194944:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787827103.991307:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228196224:192.168.203.114@tcp:250:ptlrpcd_01_01.0 00000100:00100000:3.0:1787827103.991312:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:3c946890-15ca-497a-b838-fe1cba61f4dd:2386:1874669228194304:192.168.203.114@tcp:400:kworker.0 00000100:00080000:0.0:1787827103.995913:0:2382:0:(import.c:1132:ptlrpc_connect_interpret()) MGC192.168.203.114@tcp: connect to target with instance 0 10000000:01000000:0.0:1787827103.995928:0:2382:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808005 00000100:02000400:0.0:1787827103.995937:0:2382:0:(import.c:1284:ptlrpc_connect_interpret()) Evicted from MGS (at 192.168.203.114@tcp) after server handle changed from 0x5aa6d504d91c6e64 to 0x5aa6d504d91e49d8 00000100:00080000:0.0:1787827103.995942:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to EVICTED 00000100:00080000:0.0:1787827103.995948:0:2382:0:(import.c:1674:ptlrpc_import_recovery_state_machine()) evicted from MGS@192.168.203.114@tcp; invalidating 00000100:00100000:3.0:1787827103.996227:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228196224:192.168.203.114@tcp:250:ptlrpcd_01_01.0 00000100:00080000:1.0:1787827103.996278:0:90066:0:(import.c:1598:ptlrpc_invalidate_import_thread()) thread invalidate import MGC192.168.203.114@tcp to MGS@192.168.203.114@tcp 00000100:00100000:1.0:1787827103.996282:0:90066:0:(import.c:323:ptlrpc_invalidate_import()) Sleeping 20s for inflight to error out 10000000:01000000:1.0:1787827103.996285:0:90066:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808003 00010000:00010000:1.0:1787827103.996297:0:90066:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.114@tcp lock: ffff985607f56200/0xb3f83dad1b7e8d53 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x110e400000000 nid: local remote: 0x5aa6d504d91c6f28 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787827103.996309:0:90066:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.114@tcp lock: ffff985607f56200/0xb3f83dad1b7e8d53 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x110f400000000 nid: local remote: 0x5aa6d504d91c6f28 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787827103.996315:0:90066:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:1.0:1787827103.996329:0:90066:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787827103.996338:0:90066:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.114@tcp lock: ffff985607f56200/0xb3f83dad1b7e8d53 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x590f400000000 nid: local remote: 0x5aa6d504d91c6f28 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827103.996351:0:90066:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.114@tcp lock: ffff985607f57e00/0xb3f83dad1b7e8d61 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x110e400000000 nid: local remote: 0x5aa6d504d91c6f36 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787827103.996359:0:90066:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.114@tcp lock: ffff985607f57e00/0xb3f83dad1b7e8d61 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x110f400000000 nid: local remote: 0x5aa6d504d91c6f36 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787827103.996364:0:90066:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:1.0:1787827103.996372:0:90066:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787827103.996376:0:90066:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.114@tcp lock: ffff985607f57e00/0xb3f83dad1b7e8d61 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x590f400000000 nid: local remote: 0x5aa6d504d91c6f36 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827103.996411:0:90066:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.114@tcp lock: ffff985610e07c00/0xb3f83dad1b7eda5a lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x110e400000000 nid: local remote: 0x5aa6d504d91df545 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787827103.996419:0:90066:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.114@tcp lock: ffff985610e07c00/0xb3f83dad1b7eda5a lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x110f400000000 nid: local remote: 0x5aa6d504d91df545 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787827103.996425:0:90066:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x2:0x0].0x0 (lustre) 00010000:00010000:1.0:1787827103.996438:0:90066:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787827103.996443:0:90066:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.114@tcp lock: ffff985610e07c00/0xb3f83dad1b7eda5a lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x590f400000000 nid: local remote: 0x5aa6d504d91df545 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827103.996460:0:90066:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.114@tcp lock: ffff985607f56600/0xb3f83dad1b7e8d5a lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x110e400000000 nid: local remote: 0x5aa6d504d91c6f2f expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787827103.996467:0:90066:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.114@tcp lock: ffff985607f56600/0xb3f83dad1b7e8d5a lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x110f400000000 nid: local remote: 0x5aa6d504d91c6f2f expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787827103.996473:0:90066:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x736d61726170:0x3:0x0].0x0 (params) 00010000:00010000:1.0:1787827103.996493:0:90066:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787827103.996498:0:90066:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.114@tcp lock: ffff985607f56600/0xb3f83dad1b7e8d5a lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x590f400000000 nid: local remote: 0x5aa6d504d91c6f2f expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1787827103.996517:0:90066:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from EVICTED to RECOVER 00000100:00080000:1.0:1787827103.996521:0:90066:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from RECOVER to FULL 10000000:01000000:1.0:1787827103.996523:0:90066:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808004 00000100:00080000:1.0:1787827103.996531:0:90066:0:(pinger.c:186:ptlrpc_pinger_ir_up()) IR up 00000100:02000000:1.0:1787827103.996536:0:90066:0:(import.c:1743:ptlrpc_import_recovery_state_machine()) MGC192.168.203.114@tcp: Connection restored to 192.168.203.114@tcp (at 192.168.203.114@tcp) 00010000:00080000:3.0:1787827103.998307:0:2382:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=46162) req@ffff9856140dbc00 x1874669228195712/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/416 e 0 to 0 dl 1787827128 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827103.998326:0:2382:0:(import.c:1132:ptlrpc_connect_interpret()) lustre-MDT0000-mdc-ffff985602bc9800: connect to target with instance 52 00000100:00080000:3.0:1787827103.998341:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to REPLAY 00000100:00080000:3.0:1787827103.998345:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:3.0:1787827103.998347:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 219043334908 gen 9 00000100:00100000:3.0:1787827103.998352:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610f821c0 x1874669228192256/t219043334909(219043334909) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 528/448 e 0 to 0 dl 1787827099 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827103.998358:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 219043334908 last 0 00000100:00080000:3.0:1787827103.998363:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985610f821c0 x1874669228192256/t219043334909(219043334909) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 528/448 e 0 to 0 dl 1787827099 ref 1 fl New:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827103.998371:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228195712:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787827103.998391:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f821c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228192256:192.168.203.114@tcp:36:mcreate.0 00000080:00200008:3.0:1787827104.409371:0:90092:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:3.0:1787827104.409383:0:90092:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:3.0:1787827104.409405:0:90092:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787827104.409471:0:90092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 90092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827104.409480:0:90092:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 90092 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827104.409489:0:90092:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 90092 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787827104.409514:0:90092:0:(client.c:1845:ptlrpc_send_new_req()) @@@ req waiting for recovery: (FULL != REPLAY) req@ffff985610f82d00 x1874669228196352/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 576/1152 e 0 to 0 dl 0 ref 2 fl Rpc:WQU/600/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:0 00000100:00100000:3.0:1787827104.409524:0:90092:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607cd1480 going to sleep for 0 seconds 00010000:00010000:1.0:1787827104.564866:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827105.460854:0:90092:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607cd1480 going to sleep for 0 seconds 00010000:00010000:1.0:1787827105.588869:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:0.0:1787827105.652764:0:15166:0:(mgc_request.c:611:do_requeue()) updating log lustre-cliir 10000000:01000000:0.0:1787827105.652771:0:15166:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-cliir-ffff985602bc9800 from 209 10000000:01000000:0.0:1787827105.652775:0:15166:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-cliir (res 0x65727473756c) 00010000:00010000:0.0:1787827105.652820:0:15166:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.114@tcp lock: ffff985610e06a00/0xb3f83dad1b7ee574 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827105.652840:0:15166:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.114@tcp lock: ffff985610e06a00/0xb3f83dad1b7ee574 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827105.652849:0:15166:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.114@tcp lock: ffff985610e06a00/0xb3f83dad1b7ee574 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787827105.652869:0:15166:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669228196480:192.168.203.114@tcp:101:ll_cfg_requeue.0 00000100:00100000:0.0:1787827105.652944:0:15166:0:(client.c:2715:ptlrpc_set_wait()) set ffff9856057da180 going to sleep for 11 seconds 00000100:00100000:0.0:1787827105.654214:0:15166:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669228196480:192.168.203.114@tcp:101:ll_cfg_requeue.0 00010000:00010000:0.0:1787827105.654230:0:15166:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.114@tcp lock: ffff985610e06a00/0xb3f83dad1b7ee574 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x5aa6d504d91e49df expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827105.654240:0:15166:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.114@tcp lock: ffff985610e06a00/0xb3f83dad1b7ee574 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x5aa6d504d91e49df expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787827105.654308:0:15166:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669228196608:192.168.203.114@tcp:256:ll_cfg_requeue.0 00000100:00100000:0.0:1787827105.654352:0:15166:0:(client.c:2715:ptlrpc_set_wait()) set ffff9856057da180 going to sleep for 11 seconds 00000100:00100000:0.0:1787827105.655799:0:15166:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669228196608:192.168.203.114@tcp:256:ll_cfg_requeue.0 10000000:01000000:0.0:1787827105.655868:0:15166:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.114@tcp: configuration from log 'lustre-cliir' succeeded (0). 00010000:00010000:0.0:1787827105.655877:0:15166:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.114@tcp lock: ffff985610e06a00/0xb3f83dad1b7ee574 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x5aa6d504d91e49df expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827105.655886:0:15166:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.114@tcp lock: ffff985610e06a00/0xb3f83dad1b7ee574 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x5aa6d504d91e49df expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787827105.655894:0:15166:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 10000000:01000000:0.0:1787827105.655896:0:15166:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff985602bc9800 from 52 10000000:01000000:0.0:1787827105.655899:0:15166:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00010000:00010000:0.0:1787827105.655920:0:15166:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.114@tcp lock: ffff985610e07600/0xb3f83dad1b7ee57b lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827105.655928:0:15166:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.114@tcp lock: ffff985610e07600/0xb3f83dad1b7ee57b lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827105.655935:0:15166:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.114@tcp lock: ffff985610e07600/0xb3f83dad1b7ee57b lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787827105.655949:0:15166:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669228196736:192.168.203.114@tcp:101:ll_cfg_requeue.0 00000100:00100000:0.0:1787827105.655992:0:15166:0:(client.c:2715:ptlrpc_set_wait()) set ffff9856057da180 going to sleep for 11 seconds 00000100:00100000:0.0:1787827105.656999:0:15166:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669228196736:192.168.203.114@tcp:101:ll_cfg_requeue.0 00010000:00010000:0.0:1787827105.657011:0:15166:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.114@tcp lock: ffff985610e07600/0xb3f83dad1b7ee57b lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x5aa6d504d91e49e6 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827105.657018:0:15166:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.114@tcp lock: ffff985610e07600/0xb3f83dad1b7ee57b lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x5aa6d504d91e49e6 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787827105.657060:0:15166:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669228196864:192.168.203.114@tcp:501:ll_cfg_requeue.0 00000100:00100000:0.0:1787827105.657106:0:15166:0:(client.c:2715:ptlrpc_set_wait()) set ffff9856057da180 going to sleep for 11 seconds 00000100:00100000:0.0:1787827105.658102:0:15166:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669228196864:192.168.203.114@tcp:501:ll_cfg_requeue.0 00000100:00100000:0.0:1787827105.658135:0:15166:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669228196992:192.168.203.114@tcp:503:ll_cfg_requeue.0 00000100:00100000:0.0:1787827105.658192:0:15166:0:(client.c:2715:ptlrpc_set_wait()) set ffff9856057da180 going to sleep for 11 seconds 00000100:00100000:0.0:1787827105.659198:0:15166:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669228196992:192.168.203.114@tcp:503:ll_cfg_requeue.0 00000040:00080000:0.0:1787827105.659967:0:90108:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x3:0x0] index 64768 count 51 00000020:01000000:3.0:1787827105.660044:0:15166:0:(obd_config.c:2145:class_config_parse_llog()) Processed log lustre-client gen 52-51 (rc=0) 10000000:01000000:3.0:1787827105.660062:0:15166:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.114@tcp: configuration from log 'lustre-client' succeeded (0). 00010000:00010000:3.0:1787827105.660072:0:15166:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.114@tcp lock: ffff985610e07600/0xb3f83dad1b7ee57b lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x5aa6d504d91e49e6 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827105.660087:0:15166:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.114@tcp lock: ffff985610e07600/0xb3f83dad1b7ee57b lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x5aa6d504d91e49e6 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1787827105.660105:0:15166:0:(mgc_request.c:611:do_requeue()) updating log params 10000000:01000000:3.0:1787827105.660108:0:15166:0:(mgc_request.c:1802:mgc_process_log()) Process log params-ffff985602bc9800 from 7 10000000:01000000:3.0:1787827105.660110:0:15166:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for params (res 0x736d61726170) 00010000:00010000:3.0:1787827105.660138:0:15166:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.114@tcp lock: ffff985610d87800/0xb3f83dad1b7ee582 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827105.660146:0:15166:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.114@tcp lock: ffff985610d87800/0xb3f83dad1b7ee582 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827105.660154:0:15166:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.114@tcp lock: ffff985610d87800/0xb3f83dad1b7ee582 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787827105.660174:0:15166:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669228197120:192.168.203.114@tcp:101:ll_cfg_requeue.0 00000100:00100000:3.0:1787827105.660226:0:15166:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607cd1a80 going to sleep for 11 seconds 00000100:00100000:3.0:1787827105.661740:0:15166:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669228197120:192.168.203.114@tcp:101:ll_cfg_requeue.0 00010000:00010000:3.0:1787827105.661759:0:15166:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.114@tcp lock: ffff985610d87800/0xb3f83dad1b7ee582 lrc: 4/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x5aa6d504d91e49ed expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827105.661770:0:15166:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.114@tcp lock: ffff985610d87800/0xb3f83dad1b7ee582 lrc: 4/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x5aa6d504d91e49ed expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787827105.661816:0:15166:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669228197248:192.168.203.114@tcp:501:ll_cfg_requeue.0 00000100:00100000:3.0:1787827105.661868:0:15166:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607cd1a80 going to sleep for 11 seconds 00000100:00100000:3.0:1787827105.662861:0:15166:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669228197248:192.168.203.114@tcp:501:ll_cfg_requeue.0 00000100:00100000:3.0:1787827105.662890:0:15166:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669228197376:192.168.203.114@tcp:503:ll_cfg_requeue.0 00000100:00100000:3.0:1787827105.662943:0:15166:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607cd1a80 going to sleep for 11 seconds 00000100:00100000:3.0:1787827105.664035:0:15166:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669228197376:192.168.203.114@tcp:503:ll_cfg_requeue.0 00000040:00080000:3.0:1787827105.664804:0:90109:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x2:0x0] index 64768 count 7 00000020:01000000:0.0:1787827105.665314:0:15166:0:(obd_config.c:2145:class_config_parse_llog()) Processed log params gen 7-6 (rc=0) 10000000:01000000:0.0:1787827105.665331:0:15166:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.114@tcp: configuration from log 'params' succeeded (0). 00010000:00010000:0.0:1787827105.665341:0:15166:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.114@tcp lock: ffff985610d87800/0xb3f83dad1b7ee582 lrc: 3/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x5aa6d504d91e49ed expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827105.665357:0:15166:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.114@tcp lock: ffff985610d87800/0xb3f83dad1b7ee582 lrc: 2/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x5aa6d504d91e49ed expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787827105.665367:0:15166:0:(mgc_request.c:611:do_requeue()) updating log lustre-sptlrpc 10000000:01000000:0.0:1787827105.665370:0:15166:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-sptlrpc-ffff985606a30000 from 1 10000000:01000000:0.0:1787827105.665373:0:15166:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-sptlrpc (res 0x65727473756c) 00010000:00010000:0.0:1787827105.665399:0:15166:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.114@tcp lock: ffff985610e06400/0xb3f83dad1b7ee589 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827105.665407:0:15166:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.114@tcp lock: ffff985610e06400/0xb3f83dad1b7ee589 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827105.665416:0:15166:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.114@tcp lock: ffff985610e06400/0xb3f83dad1b7ee589 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787827105.665435:0:15166:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669228197504:192.168.203.114@tcp:101:ll_cfg_requeue.0 00000100:00100000:0.0:1787827105.665490:0:15166:0:(client.c:2715:ptlrpc_set_wait()) set ffff9856057da600 going to sleep for 11 seconds 00000100:00100000:0.0:1787827105.666889:0:15166:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669228197504:192.168.203.114@tcp:101:ll_cfg_requeue.0 00010000:00010000:0.0:1787827105.666906:0:15166:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.114@tcp lock: ffff985610e06400/0xb3f83dad1b7ee589 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0x5aa6d504d91e49f4 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827105.666917:0:15166:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.114@tcp lock: ffff985610e06400/0xb3f83dad1b7ee589 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1000000000000 nid: local remote: 0x5aa6d504d91e49f4 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787827105.666973:0:15166:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669228197632:192.168.203.114@tcp:501:ll_cfg_requeue.0 00000100:00100000:0.0:1787827105.667019:0:15166:0:(client.c:2715:ptlrpc_set_wait()) set ffff9856057da600 going to sleep for 11 seconds 00000100:00100000:0.0:1787827105.667856:0:15166:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:3c946890-15ca-497a-b838-fe1cba61f4dd:15166:1874669228197632:192.168.203.114@tcp:501:ll_cfg_requeue.0 10000000:01000000:0.0:1787827105.667877:0:15166:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.114@tcp: configuration from log 'lustre-sptlrpc' failed (-2). 00010000:00010000:0.0:1787827105.667883:0:15166:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.114@tcp lock: ffff985610e06400/0xb3f83dad1b7ee589 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0x5aa6d504d91e49f4 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827105.667890:0:15166:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.114@tcp lock: ffff985610e06400/0xb3f83dad1b7ee589 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0x5aa6d504d91e49f4 expref: -99 pid: 15166 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787827106.442995:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985610f821c0 x1874669228192256/t219043334909(219043334909) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 528/448 e 0 to 0 dl 1787827119 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827106.443008:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:3.0:1787827106.443010:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 219043334908 gen 9 00000100:00100000:3.0:1787827106.443014:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610f821c0 x1874669228192256/t219043334909(219043334909) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 528/448 e 0 to 0 dl 1787827119 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827106.443019:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 219043334908 last 219043334909 00000100:00080000:3.0:1787827106.443024:0:2382:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff985607df6580 x1874669228192768/t219043334910(219043334910) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827099 ref 1 fl New:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827106.443035:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f821c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228192256:192.168.203.114@tcp:36:mcreate.0 00000100:00100000:3.0:1787827106.443054:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228192768:192.168.203.114@tcp:36:rm.0 00000100:00080000:3.0:1787827106.445131:0:2382:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff985607df6580 x1874669228192768/t219043334910(219043334910) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 512/456 e 0 to 0 dl 1787827122 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827106.445142:0:2382:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:3.0:1787827106.445144:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 219043334908 gen 9 00000100:00100000:3.0:1787827106.445149:0:2382:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff985610f821c0 x1874669228192256/t219043334909(219043334909) o36->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 528/448 e 0 to 0 dl 1787827119 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827106.445155:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff985611182000 from lustre-MDT0000_UUID committed 219043334908 last 219043334910 00000100:00080000:3.0:1787827106.445160:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from REPLAY to REPLAY_LOCKS 00000100:00100000:0.0:1787827106.445482:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228192768:192.168.203.114@tcp:36:rm.0 00010000:00080000:3.0:1787827106.445729:0:90110:0:(ldlm_request.c:2972:ldlm_lock_replay_thread()) lock replay thread lustre-MDT0000-mdc-ffff985602bc9800 to lustre-MDT0000_UUID@192.168.203.114@tcp 00010000:00010000:3.0:1787827106.445733:0:90110:0:(ldlm_request.c:2888:ldlm_cancel_unused_locks_for_replay()) Dropping as many unused locks as possible before replay for namespace lustre-MDT0000-mdc-ffff985602bc9800 (3) 00000080:00200000:3.0:1787827106.445760:0:90110:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x643:0x0]: losing layout lock 00010000:00010000:3.0:1787827106.445794:0:90110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610e06000/0xb3f83dad1b7ea38e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91d0e1b expref: -99 pid: 87460 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787827106.445809:0:90110:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x643:0x0]: losing layout lock 00010000:00010000:3.0:1787827106.445814:0:90110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607f57800/0xb3f83dad1b7ea364 lrc: 0/0,0 mode: --/CR res: [0x2000182d1:0x643:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x5aa6d504d91d0da4 expref: -99 pid: 87410 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:3.0:1787827106.445828:0:90110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985607fe6e00/0xb3f83dad1b7ea379 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000000 nid: local remote: 0x5aa6d504d91d0de3 expref: -99 pid: 87423 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787827106.445838:0:90110:0:(ldlm_request.c:2901:ldlm_cancel_unused_locks_for_replay()) Canceled 0 unused locks from namespace lustre-MDT0000-mdc-ffff985602bc9800 00000100:00080000:3.0:1787827106.446243:0:90110:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from REPLAY_LOCKS to REPLAY_WAIT 00000100:00100000:0.0:1787827106.446382:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228197760:192.168.203.114@tcp:400:ldlm_lock_repla.0 00000100:00100000:0.0:1787827106.450006:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff985602bc9800: committing for last_committed 219043334910 gen 9 00000100:00080000:0.0:1787827106.450036:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from REPLAY_WAIT to RECOVER 00000100:00080000:0.0:1787827106.450049:0:2382:0:(client.c:3247:ptlrpc_resend_req()) @@@ going to resend req@ffff985610f80f00 x1874669228197760/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827122 ref 1 fl Interpret:RQU/2c0/0 rc 0/0 job:'ldlm_lock_repla.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827106.450056:0:2382:0:(client.c:3256:ptlrpc_resend_req()) @@@ it has reply, so skip it req@ffff985610f80f00 x1874669228197760/t0(0) o400->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787827122 ref 1 fl Interpret:RQU/2c0/0 rc 0/0 job:'ldlm_lock_repla.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827106.450061:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from RECOVER to FULL 00000100:02000000:0.0:1787827106.450423:0:2382:0:(import.c:1743:ptlrpc_import_recovery_state_machine()) lustre-MDT0000-mdc-ffff985602bc9800: Connection restored to 192.168.203.114@tcp (at 192.168.203.114@tcp) 00000100:00080000:0.0:1787827106.450432:0:2382:0:(recover.c:191:ptlrpc_wake_delayed()) @@@ waking (set 000000007543e60a): req@ffff985610f82d00 x1874669228196352/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 576/1152 e 0 to 0 dl 0 ref 2 fl Rpc:WQU/600/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:0 00000100:00100000:0.0:1787827106.450452:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228197760:192.168.203.114@tcp:400:ldlm_lock_repla.0 00000100:00100000:3.0:1787827106.450570:0:90092:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607cd1480 going to sleep for 11 seconds 00000100:00100000:3.0:1787827106.451930:0:90092:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:90092:1874669228196352:192.168.203.114@tcp:101:lfs.0 00010000:00010000:3.0:1787827106.451958:0:90092:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827106.451971:0:90092:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827106.451986:0:90092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827106.452003:0:90092:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827106.452016:0:90092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827106.452023:0:90092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827106.452028:0:90092:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787827106.452036:0:90092:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff985610f82d00 x1874669228196352/t0(0) o101->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 576/688 e 0 to 0 dl 1787827122 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787827106.452047:0:90092:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827106.452060:0:90092:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9856066e8000 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:3.0:1787827106.452067:0:90092:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:3.0:1787827106.452099:0:90092:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 69632 00000080:00010000:3.0:1787827106.452106:0:90092:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee56d 00000080:00010000:3.0:1787827106.452112:0:90092:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:3.0:1787827106.452115:0:90092:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee56d from it ffffab3ac0d4bcb8 00010000:00010000:3.0:1787827106.452119:0:90092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827106.452133:0:90092:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:3.0:1787827106.452945:0:90092:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 0, flags 0, foreign 0 00000080:00200000:3.0:1787827106.452953:0:90092:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:3.0:1787827106.452975:0:90092:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787827106.452987:0:90092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787827106.452999:0:90092:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787827106.453007:0:90092:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787827106.453012:0:90092:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:3.0:1787827106.453016:0:90092:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee56d 00000080:00010000:3.0:1787827106.453020:0:90092:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee56d from it ffffab3ac0d4bcb8 00010000:00010000:3.0:1787827106.453023:0:90092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787827106.453031:0:90092:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:3.0:1787827106.453150:0:90092:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:3.0:1787827106.453159:0:90092:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787827106.453167:0:90092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787827106.453181:0:90092:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:3.0:1787827106.453188:0:90092:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787827106.453192:0:90092:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:3.0:1787827106.453194:0:90092:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee56d 00000080:00010000:3.0:1787827106.453198:0:90092:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee56d from it ffffab3ac0d4bbb0 00010000:00010000:3.0:1787827106.453201:0:90092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787827106.453209:0:90092:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:3.0:1787827106.453217:0:90092:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 44 00000080:00200008:3.0:1787827106.453225:0:90092:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 100000 00000080:00200080:3.0:1787827106.453460:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00800000:00000080:3.0:1787827106.453485:0:90092:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00800000:00000004:3.0:1787827106.453494:0:90092:0:(obd_class.h:1091:obd_statfs()) lustre-MDT0000-mdc-ffff985602bc9800: age 3233, max_age 3256 00000100:00100000:3.0:1787827106.453534:0:90092:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:90092:1874669228197888:192.168.203.114@tcp:41:lfs.0 00000100:00100000:3.0:1787827106.453627:0:90092:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607cd1a80 going to sleep for 11 seconds 00000100:00100000:3.0:1787827106.454874:0:90092:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:90092:1874669228197888:192.168.203.114@tcp:41:lfs.0 00800000:00000004:3.0:1787827106.454900:0:90092:0:(obd_class.h:1117:obd_statfs()) lustre-MDT0000-mdc-ffff985602bc9800: update ffff985606a337f0 cache blocks 317333/353529 objects 1023687/1024000 00000080:00200080:3.0:1787827106.454959:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00800000:00000080:3.0:1787827106.454969:0:90092:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00800000:00000004:3.0:1787827106.454976:0:90092:0:(obd_class.h:1091:obd_statfs()) lustre-MDT0001-mdc-ffff985602bc9800: age 3233, max_age 3256 00000100:00100000:3.0:1787827106.455003:0:90092:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:90092:1874669228198016:192.168.203.114@tcp:41:lfs.0 00000100:00100000:3.0:1787827106.455043:0:90092:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607cd1a80 going to sleep for 11 seconds 00000100:00100000:3.0:1787827106.455965:0:90092:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:90092:1874669228198016:192.168.203.114@tcp:41:lfs.0 00800000:00000004:3.0:1787827106.455976:0:90092:0:(obd_class.h:1117:obd_statfs()) lustre-MDT0001-mdc-ffff985602bc9800: update ffff98560952b7f0 cache blocks 317401/353529 objects 1023737/1024000 00000080:00200080:3.0:1787827106.456006:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00800000:00000080:3.0:1787827106.456015:0:90092:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.456021:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00800000:00000080:3.0:1787827106.456025:0:90092:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.456030:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00800000:00000080:3.0:1787827106.456034:0:90092:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.456038:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00800000:00000080:3.0:1787827106.456042:0:90092:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.456046:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00800000:00000080:3.0:1787827106.456050:0:90092:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.456054:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00800000:00000080:3.0:1787827106.456057:0:90092:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.456061:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00800000:00000080:3.0:1787827106.456065:0:90092:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.456069:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00800000:00000080:3.0:1787827106.456081:0:90092:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.456086:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00800000:00000080:3.0:1787827106.456091:0:90092:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.456095:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00800000:00000080:3.0:1787827106.456098:0:90092:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.456105:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00800000:00000080:3.0:1787827106.456109:0:90092:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.456113:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00800000:00000080:3.0:1787827106.456117:0:90092:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.456121:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00800000:00000080:3.0:1787827106.456124:0:90092:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.456128:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00800000:00000080:3.0:1787827106.456132:0:90092:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.456136:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00800000:00000080:3.0:1787827106.456139:0:90092:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.456144:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00800000:00000080:3.0:1787827106.456148:0:90092:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.456153:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00800000:00000080:3.0:1787827106.456157:0:90092:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.456161:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00800000:00000080:3.0:1787827106.456165:0:90092:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.456169:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00800000:00000080:3.0:1787827106.456173:0:90092:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.456177:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00800000:00000080:3.0:1787827106.456181:0:90092:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.456186:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00800000:00000080:3.0:1787827106.456189:0:90092:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.456192:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00800000:00000080:3.0:1787827106.456195:0:90092:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.456199:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00800000:00000080:3.0:1787827106.456203:0:90092:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.456208:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00800000:00000080:3.0:1787827106.456212:0:90092:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.456216:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00800000:00000080:3.0:1787827106.456220:0:90092:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.456225:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00800000:00000080:3.0:1787827106.456229:0:90092:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.456233:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00800000:00000080:3.0:1787827106.456237:0:90092:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.456242:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00800000:00000080:3.0:1787827106.456258:0:90092:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.456270:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00800000:00000080:3.0:1787827106.456278:0:90092:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.456283:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00800000:00000080:3.0:1787827106.456288:0:90092:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.456294:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00800000:00000080:3.0:1787827106.456297:0:90092:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.456302:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00800000:00000080:3.0:1787827106.456305:0:90092:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.456309:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00800000:00000080:3.0:1787827106.456313:0:90092:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.456318:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00800000:00000080:3.0:1787827106.456322:0:90092:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.456326:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00800000:00000080:3.0:1787827106.456329:0:90092:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.456334:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00800000:00000080:3.0:1787827106.456337:0:90092:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.456341:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00800000:00000080:3.0:1787827106.456344:0:90092:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.456348:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00800000:00000080:3.0:1787827106.456351:0:90092:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.456355:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00800000:00000080:3.0:1787827106.456358:0:90092:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.456363:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00800000:00000080:3.0:1787827106.456366:0:90092:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.456370:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00800000:00000080:3.0:1787827106.456373:0:90092:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.456385:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00800000:00000080:3.0:1787827106.456389:0:90092:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.456393:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00800000:00000080:3.0:1787827106.456396:0:90092:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.456400:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00800000:00000080:3.0:1787827106.456403:0:90092:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.456407:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00800000:00000080:3.0:1787827106.456410:0:90092:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.456414:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00800000:00000080:3.0:1787827106.456417:0:90092:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.456420:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00800000:00000080:3.0:1787827106.456423:0:90092:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.456427:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00800000:00000080:3.0:1787827106.456430:0:90092:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.456434:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00800000:00000080:3.0:1787827106.456437:0:90092:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.456441:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00800000:00000080:3.0:1787827106.456444:0:90092:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.456447:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00800000:00000080:3.0:1787827106.456450:0:90092:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.456454:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00800000:00000080:3.0:1787827106.456456:0:90092:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.456460:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00800000:00000080:3.0:1787827106.456462:0:90092:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.456466:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00800000:00000080:3.0:1787827106.456469:0:90092:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.456473:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00800000:00000080:3.0:1787827106.456476:0:90092:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.456480:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00800000:00000080:3.0:1787827106.456483:0:90092:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.456486:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00800000:00000080:3.0:1787827106.456490:0:90092:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.456493:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00800000:00000080:3.0:1787827106.456496:0:90092:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.456499:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00800000:00000080:3.0:1787827106.456503:0:90092:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.456506:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00800000:00000080:3.0:1787827106.456509:0:90092:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.456513:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00800000:00000080:3.0:1787827106.456517:0:90092:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.456520:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00800000:00000080:3.0:1787827106.456523:0:90092:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.456527:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00800000:00000080:3.0:1787827106.456530:0:90092:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.456534:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00020000:00000080:3.0:1787827106.456540:0:90092:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00020000:00000004:3.0:1787827106.456546:0:90092:0:(obd_class.h:1091:obd_statfs()) lustre-OST0000-osc-ffff985602bc9800: age 3233, max_age 3256 00000100:00080000:3.0:1787827106.456555:0:90092:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3233 00000100:00080000:3.0:1787827106.456564:0:90092:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff985602bc9800: import ffff9856110b0000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827106.456573:0:90092:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683216 ocd_grant: 8388608, lost: 0. 00000100:00100000:3.0:1787827106.456588:0:90092:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f812c0 x1874669228198144/t0(0) o8->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827106.456619:0:90092:0:(client.c:1845:ptlrpc_send_new_req()) @@@ req waiting for recovery: (FULL != CONNECTING) req@ffff985610f82d00 x1874669228198272/t0(0) o13->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:7/4 lens 224/368 e 0 to 0 dl 0 ref 2 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827106.456627:0:90092:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607cd1a80 going to sleep for 0 seconds 00000100:00100000:0.0:1787827106.456644:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228198144:192.168.203.114@tcp:8:lfs.0 00000100:00100000:0.0:1787827106.457772:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff985602bc9800: committing for last_committed 8589935703 gen 7 00000100:00080000:0.0:1787827106.457779:0:2382:0:(import.c:1132:ptlrpc_connect_interpret()) lustre-OST0000-osc-ffff985602bc9800: connect to target with instance 3 00000100:00080000:0.0:1787827106.457784:0:2382:0:(import.c:956:ptlrpc_connect_set_flags()) lustre-OST0000-osc-ffff985602bc9800: Resetting ns_connect_flags to server flags: 0xa0427af2e3440038 00000100:00080000:0.0:1787827106.457792:0:2382:0:(import.c:1213:ptlrpc_connect_interpret()) connected to replayable target: lustre-OST0000_UUID 00000100:00080000:0.0:1787827106.457794:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9856110b0000 lustre-OST0000_UUID: changing import state from CONNECTING to FULL 00000080:00000004:0.0:1787827106.457800:0:2382:0:(lcommon_misc.c:94:cl_ocd_update()) Changing connect_flags: 0xa0427af2e3440038 -> 0xa0427af2e3440038 00000080:00080000:0.0:1787827106.457803:0:2382:0:(lcommon_misc.c:56:cl_init_ea_size()) updating def/max_easize: 72/65536 00000100:00080000:0.0:1787827106.457809:0:2382:0:(recover.c:191:ptlrpc_wake_delayed()) @@@ waking (set 000000007dfe487c): req@ffff985610f82d00 x1874669228198272/t0(0) o13->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:7/4 lens 224/368 e 0 to 0 dl 0 ref 2 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827106.457824:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228198144:192.168.203.114@tcp:8:lfs.0 00000100:00100000:3.0:1787827106.457881:0:90092:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607cd1a80 going to sleep for 11 seconds 00000100:00100000:3.0:1787827106.459470:0:90092:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:90092:1874669228198272:192.168.203.114@tcp:13:lfs.0 00020000:00000004:3.0:1787827106.459480:0:90092:0:(obd_class.h:1117:obd_statfs()) lustre-OST0000-osc-ffff985602bc9800: update ffff985609528450 cache blocks 901356/958279 objects 261712/262144 00000080:00200080:3.0:1787827106.459513:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00020000:00000080:3.0:1787827106.459520:0:90092:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00020000:00000004:3.0:1787827106.459524:0:90092:0:(obd_class.h:1091:obd_statfs()) lustre-OST0001-osc-ffff985602bc9800: age 3233, max_age 3256 00000100:00080000:3.0:1787827106.459530:0:90092:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3233 00000100:00080000:3.0:1787827106.459535:0:90092:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff985602bc9800: import ffff9856110b1000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827106.459540:0:90092:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683216 ocd_grant: 8437760, lost: 0. 00000100:00100000:3.0:1787827106.459551:0:90092:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80b40 x1874669228198400/t0(0) o8->lustre-OST0001-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827106.459582:0:90092:0:(client.c:1845:ptlrpc_send_new_req()) @@@ req waiting for recovery: (FULL != CONNECTING) req@ffff985610f82d00 x1874669228198528/t0(0) o13->lustre-OST0001-osc-ffff985602bc9800@192.168.203.114@tcp:7/4 lens 224/368 e 0 to 0 dl 0 ref 2 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827106.459591:0:90092:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607cd1a80 going to sleep for 0 seconds 00000100:00100000:0.0:1787827106.459602:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228198400:192.168.203.114@tcp:8:lfs.0 00000100:00100000:0.0:1787827106.460631:0:2382:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0001-osc-ffff985602bc9800: committing for last_committed 4294967636 gen 7 00000100:00080000:0.0:1787827106.460638:0:2382:0:(import.c:1132:ptlrpc_connect_interpret()) lustre-OST0001-osc-ffff985602bc9800: connect to target with instance 1 00000100:00080000:0.0:1787827106.460644:0:2382:0:(import.c:956:ptlrpc_connect_set_flags()) lustre-OST0001-osc-ffff985602bc9800: Resetting ns_connect_flags to server flags: 0xa0427af2e3440038 00000100:00080000:0.0:1787827106.460657:0:2382:0:(import.c:1213:ptlrpc_connect_interpret()) connected to replayable target: lustre-OST0001_UUID 00000100:00080000:0.0:1787827106.460659:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9856110b1000 lustre-OST0001_UUID: changing import state from CONNECTING to FULL 00000080:00000004:0.0:1787827106.460664:0:2382:0:(lcommon_misc.c:94:cl_ocd_update()) Changing connect_flags: 0xa0427af2e3440038 -> 0xa0427af2e3440038 00000080:00080000:0.0:1787827106.460668:0:2382:0:(lcommon_misc.c:56:cl_init_ea_size()) updating def/max_easize: 72/65536 00000100:00080000:0.0:1787827106.460676:0:2382:0:(recover.c:191:ptlrpc_wake_delayed()) @@@ waking (set 000000007dfe487c): req@ffff985610f82d00 x1874669228198528/t0(0) o13->lustre-OST0001-osc-ffff985602bc9800@192.168.203.114@tcp:7/4 lens 224/368 e 0 to 0 dl 0 ref 2 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827106.460721:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228198400:192.168.203.114@tcp:8:lfs.0 00000100:00100000:3.0:1787827106.460791:0:90092:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607cd1a80 going to sleep for 11 seconds 00000100:00100000:3.0:1787827106.461541:0:90092:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:90092:1874669228198528:192.168.203.114@tcp:13:lfs.0 00020000:00000004:3.0:1787827106.461552:0:90092:0:(obd_class.h:1117:obd_statfs()) lustre-OST0001-osc-ffff985602bc9800: update ffff98560952d1c0 cache blocks 901363/958279 objects 261751/262144 00000080:00200080:3.0:1787827106.461587:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00020000:00000080:3.0:1787827106.461596:0:90092:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.461603:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00020000:00000080:3.0:1787827106.461608:0:90092:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.461613:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00020000:00000080:3.0:1787827106.461616:0:90092:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.461620:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00020000:00000080:3.0:1787827106.461625:0:90092:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.461631:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00020000:00000080:3.0:1787827106.461635:0:90092:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.461640:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00020000:00000080:3.0:1787827106.461645:0:90092:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.461650:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00020000:00000080:3.0:1787827106.461653:0:90092:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.461658:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00020000:00000080:3.0:1787827106.461662:0:90092:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.461668:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00020000:00000080:3.0:1787827106.461673:0:90092:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.461678:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00020000:00000080:3.0:1787827106.461683:0:90092:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.461688:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00020000:00000080:3.0:1787827106.461726:0:90092:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.461733:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00020000:00000080:3.0:1787827106.461737:0:90092:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.461742:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00020000:00000080:3.0:1787827106.461747:0:90092:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.461752:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00020000:00000080:3.0:1787827106.461756:0:90092:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.461761:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00020000:00000080:3.0:1787827106.461765:0:90092:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.461770:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00020000:00000080:3.0:1787827106.461774:0:90092:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.461779:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00020000:00000080:3.0:1787827106.461783:0:90092:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.461788:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00020000:00000080:3.0:1787827106.461791:0:90092:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.461796:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00020000:00000080:3.0:1787827106.461799:0:90092:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.461804:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00020000:00000080:3.0:1787827106.461807:0:90092:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.461811:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00020000:00000080:3.0:1787827106.461814:0:90092:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.461818:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00020000:00000080:3.0:1787827106.461821:0:90092:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.461824:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00020000:00000080:3.0:1787827106.461827:0:90092:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.461830:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00020000:00000080:3.0:1787827106.461833:0:90092:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.461837:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00020000:00000080:3.0:1787827106.461840:0:90092:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.461844:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00020000:00000080:3.0:1787827106.461847:0:90092:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.461851:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00020000:00000080:3.0:1787827106.461854:0:90092:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.461858:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00020000:00000080:3.0:1787827106.461861:0:90092:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.461864:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00020000:00000080:3.0:1787827106.461867:0:90092:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.461871:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00020000:00000080:3.0:1787827106.461874:0:90092:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.461877:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00020000:00000080:3.0:1787827106.461880:0:90092:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.461884:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00020000:00000080:3.0:1787827106.461887:0:90092:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.461890:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00020000:00000080:3.0:1787827106.461894:0:90092:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.461898:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00020000:00000080:3.0:1787827106.461901:0:90092:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.461905:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00020000:00000080:3.0:1787827106.461908:0:90092:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.461911:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00020000:00000080:3.0:1787827106.461914:0:90092:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.461918:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00020000:00000080:3.0:1787827106.461921:0:90092:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.461924:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00020000:00000080:3.0:1787827106.461928:0:90092:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.461931:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00020000:00000080:3.0:1787827106.461934:0:90092:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.461938:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00020000:00000080:3.0:1787827106.461941:0:90092:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.461945:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00020000:00000080:3.0:1787827106.461948:0:90092:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.461952:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00020000:00000080:3.0:1787827106.461955:0:90092:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.461959:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00020000:00000080:3.0:1787827106.461962:0:90092:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.461965:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00020000:00000080:3.0:1787827106.461968:0:90092:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.461972:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00020000:00000080:3.0:1787827106.461975:0:90092:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.461978:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00020000:00000080:3.0:1787827106.461981:0:90092:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.461985:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00020000:00000080:3.0:1787827106.461988:0:90092:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.461991:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00020000:00000080:3.0:1787827106.461994:0:90092:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.461998:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00020000:00000080:3.0:1787827106.462000:0:90092:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.462004:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00020000:00000080:3.0:1787827106.462007:0:90092:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.462011:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00020000:00000080:3.0:1787827106.462014:0:90092:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.462018:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00020000:00000080:3.0:1787827106.462021:0:90092:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.462025:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00020000:00000080:3.0:1787827106.462027:0:90092:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.462031:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00020000:00000080:3.0:1787827106.462035:0:90092:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.462038:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00020000:00000080:3.0:1787827106.462041:0:90092:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.462045:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00020000:00000080:3.0:1787827106.462048:0:90092:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.462052:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00020000:00000080:3.0:1787827106.462055:0:90092:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.462059:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00020000:00000080:3.0:1787827106.462062:0:90092:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.462066:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00020000:00000080:3.0:1787827106.462068:0:90092:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.462072:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00020000:00000080:3.0:1787827106.462075:0:90092:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.462078:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00020000:00000080:3.0:1787827106.462081:0:90092:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.462085:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00020000:00000080:3.0:1787827106.462088:0:90092:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200080:3.0:1787827106.462092:0:90092:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc9c8ec3d0 00020000:00000080:3.0:1787827106.462095:0:90092:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492800 uarg=0000000000000000 00000080:00200008:3.0:1787827106.462106:0:90092:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 100000 00010000:00010000:1.0:1787827106.612781:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827107.636922:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:3.0:1787827108.436621:0:90377:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: oleg314-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 00010000:00010000:2.0:1787827108.660897:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:2.0:1787827109.064017:0:90525:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec 00000100:00100000:0.0:1787827109.109840:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a1e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:3c946890-15ca-497a-b838-fe1cba61f4dd:2384:1874669228198656:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827109.109870:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669228198784:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827109.109935:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669228199040:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827109.109980:0:2383:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1787827109.109989:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669228198912:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827109.110019:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669228199168:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827109.112672:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669228198912:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827109.112688:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669228199168:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827109.112743:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669228198784:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827109.112754:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669228199040:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827109.112800:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a1e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:3c946890-15ca-497a-b838-fe1cba61f4dd:2384:1874669228198656:192.168.203.114@tcp:400:kworker.0 00010000:00010000:1.0:1787827109.684916:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200008:2.0:1787827110.503630:0:90563:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:2.0:1787827110.503639:0:90563:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:2.0:1787827110.503657:0:90563:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827110.503674:0:90563:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:2.0:1787827110.503688:0:90563:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:2.0:1787827110.503736:0:90563:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827110.503744:0:90563:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:2.0:1787827110.503748:0:90563:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee56d 00000080:00010000:2.0:1787827110.503756:0:90563:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee56d from it ffffab3ac0c8fcb8 00010000:00010000:2.0:1787827110.503759:0:90563:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:2.0:1787827110.503769:0:90563:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00000080:00200008:2.0:1787827110.504190:0:90563:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 0, flags 0, foreign 0 00000080:00200000:2.0:1787827110.504196:0:90563:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:2.0:1787827110.504204:0:90563:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827110.504212:0:90563:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00010000:00010000:2.0:1787827110.504232:0:90563:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00000080:00200000:2.0:1787827110.504239:0:90563:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827110.504244:0:90563:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:2.0:1787827110.504246:0:90563:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee56d 00000080:00010000:2.0:1787827110.504250:0:90563:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee56d from it ffffab3ac0c8fcb8 00010000:00010000:2.0:1787827110.504253:0:90563:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00010000:00010000:2.0:1787827110.504260:0:90563:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00000080:00200000:2.0:1787827110.504333:0:90563:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:2.0:1787827110.504342:0:90563:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827110.504349:0:90563:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00010000:00010000:2.0:1787827110.504358:0:90563:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00000080:00200000:2.0:1787827110.504365:0:90563:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827110.504369:0:90563:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:2.0:1787827110.504371:0:90563:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee56d 00000080:00010000:2.0:1787827110.504375:0:90563:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee56d from it ffffab3ac0c8fbb0 00010000:00010000:2.0:1787827110.504378:0:90563:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00010000:00010000:2.0:1787827110.504386:0:90563:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00000080:00200000:2.0:1787827110.504394:0:90563:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 44 00000080:00200008:2.0:1787827110.504401:0:90563:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 100000 00000080:00200080:2.0:1787827110.504471:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00800000:00000080:2.0:1787827110.504483:0:90563:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00800000:00000004:2.0:1787827110.504488:0:90563:0:(obd_class.h:1091:obd_statfs()) lustre-MDT0000-mdc-ffff985602bc9800: age 3257, max_age 3260 00000100:00100000:2.0:1787827110.504519:0:90563:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:90563:1874669228199296:192.168.203.114@tcp:41:lfs.0 00000100:00100000:2.0:1787827110.504579:0:90563:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bf00 going to sleep for 11 seconds 00000100:00100000:2.0:1787827110.505851:0:90563:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:90563:1874669228199296:192.168.203.114@tcp:41:lfs.0 00800000:00000004:2.0:1787827110.505869:0:90563:0:(obd_class.h:1117:obd_statfs()) lustre-MDT0000-mdc-ffff985602bc9800: update ffff985606a337f0 cache blocks 317333/353529 objects 1023688/1024000 00000080:00200080:2.0:1787827110.505909:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00800000:00000080:2.0:1787827110.505918:0:90563:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00800000:00000004:2.0:1787827110.505924:0:90563:0:(obd_class.h:1091:obd_statfs()) lustre-MDT0001-mdc-ffff985602bc9800: age 3257, max_age 3260 00000100:00100000:2.0:1787827110.505942:0:90563:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:90563:1874669228199424:192.168.203.114@tcp:41:lfs.0 00000100:00100000:2.0:1787827110.506038:0:90563:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bf00 going to sleep for 11 seconds 00000100:00100000:2.0:1787827110.507124:0:90563:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:90563:1874669228199424:192.168.203.114@tcp:41:lfs.0 00800000:00000004:2.0:1787827110.507138:0:90563:0:(obd_class.h:1117:obd_statfs()) lustre-MDT0001-mdc-ffff985602bc9800: update ffff98560952b7f0 cache blocks 317401/353529 objects 1023737/1024000 00000080:00200080:2.0:1787827110.507170:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00800000:00000080:2.0:1787827110.507178:0:90563:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.507185:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00800000:00000080:2.0:1787827110.507190:0:90563:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.507196:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00800000:00000080:2.0:1787827110.507201:0:90563:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.507206:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00800000:00000080:2.0:1787827110.507209:0:90563:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.507214:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00800000:00000080:2.0:1787827110.507229:0:90563:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.507236:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00800000:00000080:2.0:1787827110.507241:0:90563:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.507246:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00800000:00000080:2.0:1787827110.507251:0:90563:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.507256:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00800000:00000080:2.0:1787827110.507260:0:90563:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.507264:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00800000:00000080:2.0:1787827110.507268:0:90563:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.507273:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00800000:00000080:2.0:1787827110.507277:0:90563:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.507282:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00800000:00000080:2.0:1787827110.507287:0:90563:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.507293:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00800000:00000080:2.0:1787827110.507297:0:90563:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.507302:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00800000:00000080:2.0:1787827110.507308:0:90563:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.507313:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00800000:00000080:2.0:1787827110.507318:0:90563:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.507323:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00800000:00000080:2.0:1787827110.507328:0:90563:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.507333:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00800000:00000080:2.0:1787827110.507337:0:90563:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.507342:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00800000:00000080:2.0:1787827110.507346:0:90563:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.507351:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00800000:00000080:2.0:1787827110.507355:0:90563:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.507359:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00800000:00000080:2.0:1787827110.507363:0:90563:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.507368:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00800000:00000080:2.0:1787827110.507372:0:90563:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.507376:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00800000:00000080:2.0:1787827110.507380:0:90563:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.507385:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00800000:00000080:2.0:1787827110.507388:0:90563:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.507392:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00800000:00000080:2.0:1787827110.507396:0:90563:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.507400:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00800000:00000080:2.0:1787827110.507404:0:90563:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.507409:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00800000:00000080:2.0:1787827110.507413:0:90563:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.507417:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00800000:00000080:2.0:1787827110.507420:0:90563:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.507424:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00800000:00000080:2.0:1787827110.507428:0:90563:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.507433:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00800000:00000080:2.0:1787827110.507436:0:90563:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.507441:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00800000:00000080:2.0:1787827110.507444:0:90563:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.507448:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00800000:00000080:2.0:1787827110.507452:0:90563:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.507456:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00800000:00000080:2.0:1787827110.507460:0:90563:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.507464:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00800000:00000080:2.0:1787827110.507467:0:90563:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.507471:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00800000:00000080:2.0:1787827110.507475:0:90563:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.507479:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00800000:00000080:2.0:1787827110.507483:0:90563:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.507488:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00800000:00000080:2.0:1787827110.507491:0:90563:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.507495:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00800000:00000080:2.0:1787827110.507499:0:90563:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.507503:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00800000:00000080:2.0:1787827110.507506:0:90563:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.507510:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00800000:00000080:2.0:1787827110.507514:0:90563:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.507518:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00800000:00000080:2.0:1787827110.507522:0:90563:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.507526:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00800000:00000080:2.0:1787827110.507529:0:90563:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.507534:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00800000:00000080:2.0:1787827110.507537:0:90563:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.507541:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00800000:00000080:2.0:1787827110.507545:0:90563:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.507549:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00800000:00000080:2.0:1787827110.507553:0:90563:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.507557:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00800000:00000080:2.0:1787827110.507561:0:90563:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.507565:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00800000:00000080:2.0:1787827110.507569:0:90563:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.507573:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00800000:00000080:2.0:1787827110.507577:0:90563:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.507580:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00800000:00000080:2.0:1787827110.507584:0:90563:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.507588:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00800000:00000080:2.0:1787827110.507591:0:90563:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.507596:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00800000:00000080:2.0:1787827110.507600:0:90563:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.507603:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00800000:00000080:2.0:1787827110.507607:0:90563:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.507610:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00800000:00000080:2.0:1787827110.507614:0:90563:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.507618:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00800000:00000080:2.0:1787827110.507622:0:90563:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.507627:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00800000:00000080:2.0:1787827110.507630:0:90563:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.507634:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00800000:00000080:2.0:1787827110.507638:0:90563:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.507642:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00800000:00000080:2.0:1787827110.507646:0:90563:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.507651:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00800000:00000080:2.0:1787827110.507654:0:90563:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.507658:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00800000:00000080:2.0:1787827110.507662:0:90563:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.507680:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00800000:00000080:2.0:1787827110.507685:0:90563:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.507711:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00800000:00000080:2.0:1787827110.507717:0:90563:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.507723:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00800000:00000080:2.0:1787827110.507727:0:90563:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.507732:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00800000:00000080:2.0:1787827110.507736:0:90563:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.507741:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00800000:00000080:2.0:1787827110.507744:0:90563:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.507749:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00800000:00000080:2.0:1787827110.507753:0:90563:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.507758:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00020000:00000080:2.0:1787827110.507765:0:90563:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00020000:00000004:2.0:1787827110.507772:0:90563:0:(obd_class.h:1091:obd_statfs()) lustre-OST0000-osc-ffff985602bc9800: age 3257, max_age 3260 00000100:00100000:2.0:1787827110.507791:0:90563:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:90563:1874669228199552:192.168.203.114@tcp:13:lfs.0 00000100:00100000:2.0:1787827110.507832:0:90563:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bf00 going to sleep for 11 seconds 00000100:00100000:2.0:1787827110.509023:0:90563:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:90563:1874669228199552:192.168.203.114@tcp:13:lfs.0 00020000:00000004:2.0:1787827110.509037:0:90563:0:(obd_class.h:1117:obd_statfs()) lustre-OST0000-osc-ffff985602bc9800: update ffff985609528450 cache blocks 901356/958279 objects 261727/262144 00000080:00200080:2.0:1787827110.509070:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00020000:00000080:2.0:1787827110.509088:0:90563:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00020000:00000004:2.0:1787827110.509093:0:90563:0:(obd_class.h:1091:obd_statfs()) lustre-OST0001-osc-ffff985602bc9800: age 3257, max_age 3260 00000100:00100000:2.0:1787827110.509108:0:90563:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:90563:1874669228199680:192.168.203.114@tcp:13:lfs.0 00000100:00100000:2.0:1787827110.509279:0:90563:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bf00 going to sleep for 11 seconds 00000100:00100000:2.0:1787827110.510110:0:90563:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:90563:1874669228199680:192.168.203.114@tcp:13:lfs.0 00020000:00000004:2.0:1787827110.510204:0:90563:0:(obd_class.h:1117:obd_statfs()) lustre-OST0001-osc-ffff985602bc9800: update ffff98560952d1c0 cache blocks 901363/958279 objects 261751/262144 00000080:00200080:2.0:1787827110.510244:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00020000:00000080:2.0:1787827110.510252:0:90563:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.510259:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00020000:00000080:2.0:1787827110.510264:0:90563:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.510269:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00020000:00000080:2.0:1787827110.510272:0:90563:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.510276:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00020000:00000080:2.0:1787827110.510280:0:90563:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.510285:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00020000:00000080:2.0:1787827110.510289:0:90563:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.510293:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00020000:00000080:2.0:1787827110.510297:0:90563:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.510301:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00020000:00000080:2.0:1787827110.510305:0:90563:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.510309:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00020000:00000080:2.0:1787827110.510312:0:90563:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.510316:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00020000:00000080:2.0:1787827110.510320:0:90563:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.510325:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00020000:00000080:2.0:1787827110.510329:0:90563:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.510334:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00020000:00000080:2.0:1787827110.510338:0:90563:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.510343:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00020000:00000080:2.0:1787827110.510346:0:90563:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.510349:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00020000:00000080:2.0:1787827110.510352:0:90563:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.510356:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00020000:00000080:2.0:1787827110.510359:0:90563:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.510362:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00020000:00000080:2.0:1787827110.510365:0:90563:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.510369:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00020000:00000080:2.0:1787827110.510372:0:90563:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.510375:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00020000:00000080:2.0:1787827110.510378:0:90563:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.510382:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00020000:00000080:2.0:1787827110.510385:0:90563:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.510388:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00020000:00000080:2.0:1787827110.510391:0:90563:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.510395:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00020000:00000080:2.0:1787827110.510398:0:90563:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.510402:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00020000:00000080:2.0:1787827110.510405:0:90563:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.510408:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00020000:00000080:2.0:1787827110.510411:0:90563:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.510416:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00020000:00000080:2.0:1787827110.510419:0:90563:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.510423:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00020000:00000080:2.0:1787827110.510428:0:90563:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.510432:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00020000:00000080:2.0:1787827110.510445:0:90563:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.510451:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00020000:00000080:2.0:1787827110.510454:0:90563:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.510459:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00020000:00000080:2.0:1787827110.510463:0:90563:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.510467:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00020000:00000080:2.0:1787827110.510470:0:90563:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.510474:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00020000:00000080:2.0:1787827110.510478:0:90563:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.510483:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00020000:00000080:2.0:1787827110.510486:0:90563:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.510490:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00020000:00000080:2.0:1787827110.510493:0:90563:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.510496:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00020000:00000080:2.0:1787827110.510500:0:90563:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.510504:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00020000:00000080:2.0:1787827110.510507:0:90563:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.510510:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00020000:00000080:2.0:1787827110.510513:0:90563:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.510517:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00020000:00000080:2.0:1787827110.510520:0:90563:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.510523:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00020000:00000080:2.0:1787827110.510526:0:90563:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.510530:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00020000:00000080:2.0:1787827110.510533:0:90563:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.510536:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00020000:00000080:2.0:1787827110.510539:0:90563:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.510543:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00020000:00000080:2.0:1787827110.510546:0:90563:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.510549:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00020000:00000080:2.0:1787827110.510552:0:90563:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.510555:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00020000:00000080:2.0:1787827110.510558:0:90563:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.510562:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00020000:00000080:2.0:1787827110.510565:0:90563:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.510568:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00020000:00000080:2.0:1787827110.510571:0:90563:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.510575:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00020000:00000080:2.0:1787827110.510578:0:90563:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.510582:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00020000:00000080:2.0:1787827110.510585:0:90563:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.510588:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00020000:00000080:2.0:1787827110.510591:0:90563:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.510594:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00020000:00000080:2.0:1787827110.510597:0:90563:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.510601:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00020000:00000080:2.0:1787827110.510604:0:90563:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.510607:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00020000:00000080:2.0:1787827110.510610:0:90563:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.510614:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00020000:00000080:2.0:1787827110.510616:0:90563:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.510620:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00020000:00000080:2.0:1787827110.510623:0:90563:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.510626:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00020000:00000080:2.0:1787827110.510629:0:90563:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.510633:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00020000:00000080:2.0:1787827110.510636:0:90563:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.510639:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00020000:00000080:2.0:1787827110.510642:0:90563:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.510646:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00020000:00000080:2.0:1787827110.510649:0:90563:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.510652:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00020000:00000080:2.0:1787827110.510655:0:90563:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.510658:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00020000:00000080:2.0:1787827110.510662:0:90563:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.510665:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00020000:00000080:2.0:1787827110.510668:0:90563:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.510672:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00020000:00000080:2.0:1787827110.510675:0:90563:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.510678:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00020000:00000080:2.0:1787827110.510681:0:90563:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.510684:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00020000:00000080:2.0:1787827110.510687:0:90563:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.510717:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00020000:00000080:2.0:1787827110.510726:0:90563:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200080:2.0:1787827110.510731:0:90563:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff02c54820 00020000:00000080:2.0:1787827110.510734:0:90563:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff9856110f3c00 uarg=0000000000000000 00000080:00200008:2.0:1787827110.510746:0:90563:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 100000 00010000:00010000:2.0:1787827110.709331:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827111.734047:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827112.756920:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827113.781246:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827114.228922:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2383:1874669228199808:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827114.228922:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669228199936:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787827114.229023:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669228200064:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827114.229024:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669228200192:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787827114.229064:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669228200320:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827114.230950:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669228199936:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787827114.230958:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2383:1874669228199808:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827114.230975:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669228200192:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787827114.231117:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669228200064:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787827114.231131:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669228200320:192.168.203.114@tcp:400:kworker.0 00010000:00010000:1.0:1787827114.804887:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827115.828897:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827116.852917:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827117.877006:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827118.900953:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827119.349145:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:3c946890-15ca-497a-b838-fe1cba61f4dd:2386:1874669228200448:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787827119.349295:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228200704:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787827119.349329:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228200960:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787827119.349387:0:2386:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:2.0:1787827119.349396:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228200576:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787827119.349425:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228200832:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787827119.353507:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228200576:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787827119.353532:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228200704:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787827119.356633:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228200832:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787827119.356656:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:3c946890-15ca-497a-b838-fe1cba61f4dd:2386:1874669228200448:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787827119.356669:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228200960:192.168.203.114@tcp:400:kworker.0 00010000:00010000:1.0:1787827119.924964:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827120.949001:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827121.973772:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827122.996890:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:3.0:1787827123.264607:0:91154:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == replay-single test 44a: race in target handle connect ========================================================== 06:38:42 (1787827122) 00010000:00010000:3.0:1787827124.021959:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827124.468971:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:3c946890-15ca-497a-b838-fe1cba61f4dd:2384:1874669228201088:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787827124.469000:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669228201216:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827124.469053:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669228201344:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787827124.469057:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669228201472:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827124.469093:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669228201600:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827124.471119:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669228201344:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827124.471178:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:3c946890-15ca-497a-b838-fe1cba61f4dd:2384:1874669228201088:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827124.471203:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669228201600:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787827124.471239:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669228201216:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787827124.471252:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669228201472:192.168.203.114@tcp:400:kworker.0 00010000:00010000:1.0:1787827125.044864:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000080:3.0:1787827125.168730:0:91292:0:(class_obd.c:336:class_handle_ioctl()) obdclass: cmd=40086685 len=0 uarg=00007fff1e558480 00000002:00000080:3.0:1787827125.168747:0:91292:0:(mdc_request.c:2317:mdc_iocontrol()) lustre-MDT0000-mdc-ffff985602bc9800: cmd=40086685 len=576 karg=ffff985608492800 uarg=0000000000000000 00000100:00080000:3.0:1787827125.168755:0:91292:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from FULL to DISCONN 00000100:02000400:3.0:1787827125.168764:0:91292:0:(import.c:182:ptlrpc_set_import_discon()) lustre-MDT0000-mdc-ffff985602bc9800: Connection to lustre-MDT0000 (at 192.168.203.114@tcp) was lost; in progress operations using this service will wait for recovery to complete 00000100:00080000:3.0:1787827125.168770:0:91292:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827125.168775:0:91292:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3254 00000100:00080000:3.0:1787827125.168788:0:91292:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827125.168798:0:91292:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827125.168822:0:91292:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81e00 x1874669228201728/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827125.168847:0:91292:0:(recover.c:337:ptlrpc_recover_import()) lustre-MDT0000_UUID: recovery started, waiting 20 seconds 00000100:00100000:0.0:1787827125.168875:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228201728:192.168.203.114@tcp:38:lctl.0 00010000:00010000:2.0:1787827126.068931:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827127.092858:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827128.117381:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827129.140923:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827129.588855:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827129.588862:0:38:0:(pinger.c:247:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1787827129.588891:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2385:1874669228201856:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787827129.588905:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228201984:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787827129.588983:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228202240:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787827129.588990:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669228202112:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787827129.590732:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2385:1874669228201856:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787827129.591113:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228201984:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787827129.591125:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228202240:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787827129.591169:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669228202112:192.168.203.114@tcp:400:kworker.0 00010000:00010000:1.0:1787827130.164857:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827130.503091:0:2382:0:(import.c:1132:ptlrpc_connect_interpret()) lustre-MDT0000-mdc-ffff985602bc9800: connect to target with instance 52 00000100:00080000:0.0:1787827130.503118:0:2382:0:(import.c:1297:ptlrpc_connect_interpret()) reconnected to lustre-MDT0000_UUID@192.168.203.114@tcp after partition 00000100:00080000:0.0:1787827130.503122:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to RECOVER 00000100:00080000:0.0:1787827130.503128:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from RECOVER to FULL 00000100:02000000:3.0:1787827130.503383:0:2382:0:(import.c:1743:ptlrpc_import_recovery_state_machine()) lustre-MDT0000-mdc-ffff985602bc9800: Connection restored to 192.168.203.114@tcp (at 192.168.203.114@tcp) 00000100:00100000:3.0:1787827130.503403:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228201728:192.168.203.114@tcp:38:lctl.0 00000100:00080000:1.0:1787827130.503447:0:91292:0:(recover.c:348:ptlrpc_recover_import()) lustre-MDT0000_UUID: recovery finished FULL, rc = 0 00000080:00200008:3.0:1787827130.508404:0:91295:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:3.0:1787827130.508419:0:91295:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:3.0:1787827130.508446:0:91295:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787827130.508465:0:91295:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00010000:00010000:3.0:1787827130.508481:0:91295:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00000080:00200000:3.0:1787827130.508490:0:91295:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787827130.508500:0:91295:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:3.0:1787827130.508504:0:91295:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee56d 00000080:00010000:3.0:1787827130.508513:0:91295:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee56d from it ffffab3ac1cc7cb8 00010000:00010000:3.0:1787827130.508518:0:91295:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00010000:00010000:3.0:1787827130.508528:0:91295:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 1 00000080:00200008:3.0:1787827130.509120:0:91295:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 0, flags 0, foreign 0 00000080:00200000:3.0:1787827130.509123:0:91295:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:3.0:1787827130.509130:0:91295:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787827130.509137:0:91295:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 1 00010000:00010000:3.0:1787827130.509146:0:91295:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 1 00000080:00200000:3.0:1787827130.509152:0:91295:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787827130.509156:0:91295:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:3.0:1787827130.509159:0:91295:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee56d 00000080:00010000:3.0:1787827130.509163:0:91295:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee56d from it ffffab3ac1cc7cb8 00010000:00010000:3.0:1787827130.509166:0:91295:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 1 00010000:00010000:3.0:1787827130.509174:0:91295:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 1 00000080:00200000:3.0:1787827130.509264:0:91295:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:3.0:1787827130.509273:0:91295:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787827130.509280:0:91295:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 1 00010000:00010000:3.0:1787827130.509289:0:91295:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 1 00000080:00200000:3.0:1787827130.509296:0:91295:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787827130.509300:0:91295:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:3.0:1787827130.509302:0:91295:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee56d 00000080:00010000:3.0:1787827130.509306:0:91295:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee56d from it ffffab3ac1cc7bb0 00010000:00010000:3.0:1787827130.509309:0:91295:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 1 00010000:00010000:3.0:1787827130.509316:0:91295:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 9 lru_type: 1 00000080:00200000:3.0:1787827130.509324:0:91295:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 44 00000080:00200008:3.0:1787827130.509331:0:91295:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 100000 00000080:00200080:3.0:1787827130.509413:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00800000:00000080:3.0:1787827130.509427:0:91295:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00800000:00000004:3.0:1787827130.509434:0:91295:0:(obd_class.h:1091:obd_statfs()) lustre-MDT0000-mdc-ffff985602bc9800: age 3261, max_age 3280 00000100:00100000:3.0:1787827130.509472:0:91295:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:91295:1874669228202368:192.168.203.114@tcp:41:lfs.0 00000100:00100000:3.0:1787827130.509540:0:91295:0:(client.c:2715:ptlrpc_set_wait()) set ffff985606a6e240 going to sleep for 11 seconds 00000100:00100000:3.0:1787827130.510853:0:91295:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:91295:1874669228202368:192.168.203.114@tcp:41:lfs.0 00800000:00000004:3.0:1787827130.510867:0:91295:0:(obd_class.h:1117:obd_statfs()) lustre-MDT0000-mdc-ffff985602bc9800: update ffff985606a337f0 cache blocks 317333/353529 objects 1023688/1024000 00000080:00200080:3.0:1787827130.510901:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00800000:00000080:3.0:1787827130.510908:0:91295:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00800000:00000004:3.0:1787827130.510912:0:91295:0:(obd_class.h:1091:obd_statfs()) lustre-MDT0001-mdc-ffff985602bc9800: age 3261, max_age 3280 00000100:00100000:3.0:1787827130.510925:0:91295:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:91295:1874669228202496:192.168.203.114@tcp:41:lfs.0 00000100:00100000:3.0:1787827130.510959:0:91295:0:(client.c:2715:ptlrpc_set_wait()) set ffff985606a6e240 going to sleep for 11 seconds 00000100:00100000:3.0:1787827130.511782:0:91295:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:91295:1874669228202496:192.168.203.114@tcp:41:lfs.0 00800000:00000004:3.0:1787827130.511792:0:91295:0:(obd_class.h:1117:obd_statfs()) lustre-MDT0001-mdc-ffff985602bc9800: update ffff98560952b7f0 cache blocks 317401/353529 objects 1023737/1024000 00000080:00200080:3.0:1787827130.511823:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00800000:00000080:3.0:1787827130.511832:0:91295:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.511839:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00800000:00000080:3.0:1787827130.511843:0:91295:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.511848:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00800000:00000080:3.0:1787827130.511852:0:91295:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.511857:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00800000:00000080:3.0:1787827130.511861:0:91295:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.511865:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00800000:00000080:3.0:1787827130.511869:0:91295:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.511872:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00800000:00000080:3.0:1787827130.511875:0:91295:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.511879:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00800000:00000080:3.0:1787827130.511882:0:91295:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.511886:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00800000:00000080:3.0:1787827130.511889:0:91295:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.511893:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00800000:00000080:3.0:1787827130.511896:0:91295:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.511899:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00800000:00000080:3.0:1787827130.511902:0:91295:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.511907:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00800000:00000080:3.0:1787827130.511910:0:91295:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.511914:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00800000:00000080:3.0:1787827130.511917:0:91295:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.511921:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00800000:00000080:3.0:1787827130.511924:0:91295:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.511927:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00800000:00000080:3.0:1787827130.511930:0:91295:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.511934:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00800000:00000080:3.0:1787827130.511936:0:91295:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.511940:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00800000:00000080:3.0:1787827130.511943:0:91295:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.511946:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00800000:00000080:3.0:1787827130.511948:0:91295:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.511952:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00800000:00000080:3.0:1787827130.511955:0:91295:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.511959:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00800000:00000080:3.0:1787827130.511962:0:91295:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.511965:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00800000:00000080:3.0:1787827130.511968:0:91295:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.511972:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00800000:00000080:3.0:1787827130.511976:0:91295:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.511979:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00800000:00000080:3.0:1787827130.511982:0:91295:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.511986:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00800000:00000080:3.0:1787827130.511990:0:91295:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.511994:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00800000:00000080:3.0:1787827130.511997:0:91295:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.512001:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00800000:00000080:3.0:1787827130.512004:0:91295:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.512008:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00800000:00000080:3.0:1787827130.512011:0:91295:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.512013:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00800000:00000080:3.0:1787827130.512015:0:91295:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.512017:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00800000:00000080:3.0:1787827130.512019:0:91295:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.512021:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00800000:00000080:3.0:1787827130.512022:0:91295:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.512025:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00800000:00000080:3.0:1787827130.512028:0:91295:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.512031:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00800000:00000080:3.0:1787827130.512035:0:91295:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.512038:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00800000:00000080:3.0:1787827130.512042:0:91295:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.512045:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00800000:00000080:3.0:1787827130.512048:0:91295:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.512053:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00800000:00000080:3.0:1787827130.512056:0:91295:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.512060:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00800000:00000080:3.0:1787827130.512063:0:91295:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.512067:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00800000:00000080:3.0:1787827130.512070:0:91295:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.512074:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00800000:00000080:3.0:1787827130.512077:0:91295:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.512100:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00800000:00000080:3.0:1787827130.512108:0:91295:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.512113:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00800000:00000080:3.0:1787827130.512116:0:91295:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.512119:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00800000:00000080:3.0:1787827130.512123:0:91295:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.512127:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00800000:00000080:3.0:1787827130.512129:0:91295:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.512133:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00800000:00000080:3.0:1787827130.512137:0:91295:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.512140:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00800000:00000080:3.0:1787827130.512143:0:91295:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.512147:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00800000:00000080:3.0:1787827130.512151:0:91295:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.512155:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00800000:00000080:3.0:1787827130.512159:0:91295:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.512163:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00800000:00000080:3.0:1787827130.512167:0:91295:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.512171:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00800000:00000080:3.0:1787827130.512174:0:91295:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.512178:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00800000:00000080:3.0:1787827130.512181:0:91295:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.512185:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00800000:00000080:3.0:1787827130.512188:0:91295:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.512192:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00800000:00000080:3.0:1787827130.512195:0:91295:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.512199:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00800000:00000080:3.0:1787827130.512202:0:91295:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.512206:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00800000:00000080:3.0:1787827130.512209:0:91295:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.512212:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00800000:00000080:3.0:1787827130.512214:0:91295:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.512216:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00800000:00000080:3.0:1787827130.512218:0:91295:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.512220:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00800000:00000080:3.0:1787827130.512222:0:91295:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.512224:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00800000:00000080:3.0:1787827130.512225:0:91295:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.512227:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00800000:00000080:3.0:1787827130.512229:0:91295:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.512231:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00800000:00000080:3.0:1787827130.512233:0:91295:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.512235:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00800000:00000080:3.0:1787827130.512237:0:91295:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.512239:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00800000:00000080:3.0:1787827130.512241:0:91295:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.512243:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00800000:00000080:3.0:1787827130.512244:0:91295:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.512246:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00800000:00000080:3.0:1787827130.512248:0:91295:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.512250:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00800000:00000080:3.0:1787827130.512252:0:91295:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.512254:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00020000:00000080:3.0:1787827130.512258:0:91295:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00020000:00000004:3.0:1787827130.512263:0:91295:0:(obd_class.h:1091:obd_statfs()) lustre-OST0000-osc-ffff985602bc9800: age 3261, max_age 3280 00000100:00100000:3.0:1787827130.512279:0:91295:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:91295:1874669228202624:192.168.203.114@tcp:13:lfs.0 00000100:00100000:3.0:1787827130.512311:0:91295:0:(client.c:2715:ptlrpc_set_wait()) set ffff985606a6e240 going to sleep for 11 seconds 00000100:00100000:3.0:1787827130.513236:0:91295:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:91295:1874669228202624:192.168.203.114@tcp:13:lfs.0 00020000:00000004:3.0:1787827130.513250:0:91295:0:(obd_class.h:1117:obd_statfs()) lustre-OST0000-osc-ffff985602bc9800: update ffff985609528450 cache blocks 901356/958279 objects 261727/262144 00000080:00200080:3.0:1787827130.513290:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00020000:00000080:3.0:1787827130.513300:0:91295:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00020000:00000004:3.0:1787827130.513306:0:91295:0:(obd_class.h:1091:obd_statfs()) lustre-OST0001-osc-ffff985602bc9800: age 3261, max_age 3280 00000100:00100000:3.0:1787827130.513325:0:91295:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:91295:1874669228202752:192.168.203.114@tcp:13:lfs.0 00000100:00100000:3.0:1787827130.513365:0:91295:0:(client.c:2715:ptlrpc_set_wait()) set ffff985606a6e240 going to sleep for 11 seconds 00000100:00100000:3.0:1787827130.514305:0:91295:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:91295:1874669228202752:192.168.203.114@tcp:13:lfs.0 00020000:00000004:3.0:1787827130.514319:0:91295:0:(obd_class.h:1117:obd_statfs()) lustre-OST0001-osc-ffff985602bc9800: update ffff98560952d1c0 cache blocks 901363/958279 objects 261735/262144 00000080:00200080:3.0:1787827130.514362:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00020000:00000080:3.0:1787827130.514372:0:91295:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.514379:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00020000:00000080:3.0:1787827130.514383:0:91295:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.514388:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00020000:00000080:3.0:1787827130.514392:0:91295:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.514396:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00020000:00000080:3.0:1787827130.514400:0:91295:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.514404:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00020000:00000080:3.0:1787827130.514409:0:91295:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.514413:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00020000:00000080:3.0:1787827130.514418:0:91295:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.514423:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00020000:00000080:3.0:1787827130.514427:0:91295:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.514431:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00020000:00000080:3.0:1787827130.514434:0:91295:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.514438:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00020000:00000080:3.0:1787827130.514441:0:91295:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.514445:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00020000:00000080:3.0:1787827130.514449:0:91295:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.514453:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00020000:00000080:3.0:1787827130.514456:0:91295:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.514460:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00020000:00000080:3.0:1787827130.514463:0:91295:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.514467:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00020000:00000080:3.0:1787827130.514471:0:91295:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.514475:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00020000:00000080:3.0:1787827130.514479:0:91295:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.514483:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00020000:00000080:3.0:1787827130.514487:0:91295:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.514490:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00020000:00000080:3.0:1787827130.514494:0:91295:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.514498:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00020000:00000080:3.0:1787827130.514501:0:91295:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.514505:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00020000:00000080:3.0:1787827130.514509:0:91295:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.514513:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00020000:00000080:3.0:1787827130.514516:0:91295:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.514520:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00020000:00000080:3.0:1787827130.514524:0:91295:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.514528:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00020000:00000080:3.0:1787827130.514531:0:91295:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.514535:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00020000:00000080:3.0:1787827130.514539:0:91295:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.514542:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00020000:00000080:3.0:1787827130.514546:0:91295:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.514560:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00020000:00000080:3.0:1787827130.514564:0:91295:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.514568:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00020000:00000080:3.0:1787827130.514571:0:91295:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.514576:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00020000:00000080:3.0:1787827130.514579:0:91295:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.514583:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00020000:00000080:3.0:1787827130.514587:0:91295:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.514590:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00020000:00000080:3.0:1787827130.514594:0:91295:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.514598:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00020000:00000080:3.0:1787827130.514601:0:91295:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.514605:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00020000:00000080:3.0:1787827130.514608:0:91295:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.514612:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00020000:00000080:3.0:1787827130.514616:0:91295:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.514619:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00020000:00000080:3.0:1787827130.514623:0:91295:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.514627:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00020000:00000080:3.0:1787827130.514631:0:91295:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.514635:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00020000:00000080:3.0:1787827130.514639:0:91295:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.514643:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00020000:00000080:3.0:1787827130.514646:0:91295:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.514650:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00020000:00000080:3.0:1787827130.514654:0:91295:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.514658:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00020000:00000080:3.0:1787827130.514662:0:91295:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.514665:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00020000:00000080:3.0:1787827130.514669:0:91295:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.514673:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00020000:00000080:3.0:1787827130.514676:0:91295:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.514680:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00020000:00000080:3.0:1787827130.514684:0:91295:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.514688:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00020000:00000080:3.0:1787827130.514721:0:91295:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.514727:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00020000:00000080:3.0:1787827130.514731:0:91295:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.514735:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00020000:00000080:3.0:1787827130.514738:0:91295:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.514742:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00020000:00000080:3.0:1787827130.514745:0:91295:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.514748:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00020000:00000080:3.0:1787827130.514751:0:91295:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.514754:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00020000:00000080:3.0:1787827130.514757:0:91295:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.514761:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00020000:00000080:3.0:1787827130.514763:0:91295:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.514767:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00020000:00000080:3.0:1787827130.514770:0:91295:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.514773:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00020000:00000080:3.0:1787827130.514776:0:91295:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.514780:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00020000:00000080:3.0:1787827130.514782:0:91295:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.514786:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00020000:00000080:3.0:1787827130.514789:0:91295:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.514793:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00020000:00000080:3.0:1787827130.514795:0:91295:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.514799:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00020000:00000080:3.0:1787827130.514802:0:91295:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.514805:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00020000:00000080:3.0:1787827130.514808:0:91295:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.514812:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00020000:00000080:3.0:1787827130.514815:0:91295:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.514818:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00020000:00000080:3.0:1787827130.514821:0:91295:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.514825:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00020000:00000080:3.0:1787827130.514828:0:91295:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.514832:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00020000:00000080:3.0:1787827130.514835:0:91295:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.514840:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00020000:00000080:3.0:1787827130.514843:0:91295:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.514847:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00020000:00000080:3.0:1787827130.514851:0:91295:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.514854:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00020000:00000080:3.0:1787827130.514857:0:91295:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.514861:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00020000:00000080:3.0:1787827130.514864:0:91295:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200080:3.0:1787827130.514868:0:91295:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff7ba54e10 00020000:00000080:3.0:1787827130.514872:0:91295:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608492c00 uarg=0000000000000000 00000080:00200008:3.0:1787827130.514885:0:91295:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 100000 00000020:00000080:2.0:1787827131.149981:0:91313:0:(class_obd.c:336:class_handle_ioctl()) obdclass: cmd=40086685 len=0 uarg=00007ffe45aa86f0 00000002:00000080:2.0:1787827131.149999:0:91313:0:(mdc_request.c:2317:mdc_iocontrol()) lustre-MDT0000-mdc-ffff985602bc9800: cmd=40086685 len=576 karg=ffff9856110f1400 uarg=0000000000000000 00000100:00080000:2.0:1787827131.150007:0:91313:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from FULL to DISCONN 00000100:02000400:2.0:1787827131.150015:0:91313:0:(import.c:182:ptlrpc_set_import_discon()) lustre-MDT0000-mdc-ffff985602bc9800: Connection to lustre-MDT0000 (at 192.168.203.114@tcp) was lost; in progress operations using this service will wait for recovery to complete 00000100:00080000:2.0:1787827131.150021:0:91313:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827131.150025:0:91313:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3275 00000100:00080000:2.0:1787827131.150040:0:91313:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787827131.150051:0:91313:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827131.150091:0:91313:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d83c0 x1874669228202880/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827131.150119:0:91313:0:(recover.c:337:ptlrpc_recover_import()) lustre-MDT0000_UUID: recovery started, waiting 20 seconds 00000100:00100000:3.0:1787827131.150151:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228202880:192.168.203.114@tcp:38:lctl.0 00010000:00010000:2.0:1787827131.188892:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827132.212900:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827133.238044:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827134.260887:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827134.708881:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827134.708888:0:66417:0:(pinger.c:247:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:0.0:1787827134.708928:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2383:1874669228203008:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827134.708935:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a1e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669228203136:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787827134.709028:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669228203264:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827134.709030:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669228203392:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787827134.710675:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2383:1874669228203008:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787827134.710723:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669228203264:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827134.710773:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a1e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669228203136:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827134.710785:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669228203392:192.168.203.114@tcp:400:kworker.0 00010000:00010000:1.0:1787827135.285579:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827135.732854:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827135.732858:0:66417:0:(pinger.c:247:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:1.0:1787827135.732883:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669228203648:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787827135.732902:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2383:1874669228203520:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787827135.732965:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669228203776:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827135.732977:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669228203904:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787827135.734480:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2383:1874669228203520:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827135.734499:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669228203904:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787827135.734500:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669228203776:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827135.734527:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669228203648:192.168.203.114@tcp:400:kworker.0 00010000:00010000:2.0:1787827136.308888:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827136.646516:0:2382:0:(import.c:1132:ptlrpc_connect_interpret()) lustre-MDT0000-mdc-ffff985602bc9800: connect to target with instance 52 00000100:00080000:3.0:1787827136.646542:0:2382:0:(import.c:1297:ptlrpc_connect_interpret()) reconnected to lustre-MDT0000_UUID@192.168.203.114@tcp after partition 00000100:00080000:3.0:1787827136.646555:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to RECOVER 00000100:00080000:3.0:1787827136.646562:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from RECOVER to FULL 00000100:02000000:0.0:1787827136.646966:0:2382:0:(import.c:1743:ptlrpc_import_recovery_state_machine()) lustre-MDT0000-mdc-ffff985602bc9800: Connection restored to 192.168.203.114@tcp (at 192.168.203.114@tcp) 00000100:00100000:0.0:1787827136.646981:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228202880:192.168.203.114@tcp:38:lctl.0 00000100:00080000:2.0:1787827136.647082:0:91313:0:(recover.c:348:ptlrpc_recover_import()) lustre-MDT0000_UUID: recovery finished FULL, rc = 0 00000080:00200008:3.0:1787827136.652644:0:91315:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:3.0:1787827136.652658:0:91315:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:3.0:1787827136.652678:0:91315:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787827136.652744:0:91315:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 9 lru_type: 1 00010000:00010000:3.0:1787827136.652761:0:91315:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 9 lru_type: 1 00000080:00200000:3.0:1787827136.652770:0:91315:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787827136.652779:0:91315:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:3.0:1787827136.652783:0:91315:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee56d 00000080:00010000:3.0:1787827136.652791:0:91315:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee56d from it ffffab3ac1d77cb8 00010000:00010000:3.0:1787827136.652795:0:91315:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 9 lru_type: 1 00010000:00010000:3.0:1787827136.652804:0:91315:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 10 lru_type: 1 00000080:00200008:3.0:1787827136.653373:0:91315:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 0, flags 0, foreign 0 00000080:00200000:3.0:1787827136.653378:0:91315:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:3.0:1787827136.653387:0:91315:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787827136.653395:0:91315:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 10 lru_type: 1 00010000:00010000:3.0:1787827136.653403:0:91315:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 10 lru_type: 1 00000080:00200000:3.0:1787827136.653409:0:91315:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787827136.653413:0:91315:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:3.0:1787827136.653415:0:91315:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee56d 00000080:00010000:3.0:1787827136.653419:0:91315:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee56d from it ffffab3ac1d77cb8 00010000:00010000:3.0:1787827136.653421:0:91315:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 10 lru_type: 1 00010000:00010000:3.0:1787827136.653428:0:91315:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 11 lru_type: 1 00000080:00200000:3.0:1787827136.653498:0:91315:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:3.0:1787827136.653506:0:91315:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787827136.653513:0:91315:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 11 lru_type: 1 00010000:00010000:3.0:1787827136.653521:0:91315:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 11 lru_type: 1 00000080:00200000:3.0:1787827136.653532:0:91315:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787827136.653536:0:91315:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:3.0:1787827136.653539:0:91315:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee56d 00000080:00010000:3.0:1787827136.653542:0:91315:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee56d from it ffffab3ac1d77bb0 00010000:00010000:3.0:1787827136.653546:0:91315:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 11 lru_type: 1 00010000:00010000:3.0:1787827136.653553:0:91315:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 12 lru_type: 1 00000080:00200000:3.0:1787827136.653562:0:91315:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 44 00000080:00200008:3.0:1787827136.653567:0:91315:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 100000 00000080:00200080:3.0:1787827136.653630:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00800000:00000080:3.0:1787827136.653643:0:91315:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00800000:00000004:3.0:1787827136.653650:0:91315:0:(obd_class.h:1091:obd_statfs()) lustre-MDT0000-mdc-ffff985602bc9800: age 3281, max_age 3286 00000100:00100000:3.0:1787827136.653752:0:91315:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:91315:1874669228204032:192.168.203.114@tcp:41:lfs.0 00000100:00100000:3.0:1787827136.653899:0:91315:0:(client.c:2715:ptlrpc_set_wait()) set ffff9856088406c0 going to sleep for 12 seconds 00000100:00100000:3.0:1787827136.655322:0:91315:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:91315:1874669228204032:192.168.203.114@tcp:41:lfs.0 00800000:00000004:3.0:1787827136.655343:0:91315:0:(obd_class.h:1117:obd_statfs()) lustre-MDT0000-mdc-ffff985602bc9800: update ffff985606a337f0 cache blocks 317333/353529 objects 1023688/1024000 00000080:00200080:3.0:1787827136.655403:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00800000:00000080:3.0:1787827136.655414:0:91315:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00800000:00000004:3.0:1787827136.655420:0:91315:0:(obd_class.h:1091:obd_statfs()) lustre-MDT0001-mdc-ffff985602bc9800: age 3281, max_age 3286 00000100:00100000:3.0:1787827136.655442:0:91315:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:91315:1874669228204160:192.168.203.114@tcp:41:lfs.0 00000100:00100000:3.0:1787827136.655494:0:91315:0:(client.c:2715:ptlrpc_set_wait()) set ffff9856088406c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787827136.656824:0:91315:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:91315:1874669228204160:192.168.203.114@tcp:41:lfs.0 00800000:00000004:3.0:1787827136.656860:0:91315:0:(obd_class.h:1117:obd_statfs()) lustre-MDT0001-mdc-ffff985602bc9800: update ffff98560952b7f0 cache blocks 317401/353529 objects 1023737/1024000 00000080:00200080:3.0:1787827136.656936:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00800000:00000080:3.0:1787827136.656955:0:91315:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.656964:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00800000:00000080:3.0:1787827136.656969:0:91315:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.656974:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00800000:00000080:3.0:1787827136.656979:0:91315:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.656985:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00800000:00000080:3.0:1787827136.656989:0:91315:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.656995:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00800000:00000080:3.0:1787827136.656999:0:91315:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.657005:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00800000:00000080:3.0:1787827136.657009:0:91315:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.657016:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00800000:00000080:3.0:1787827136.657019:0:91315:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.657022:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00800000:00000080:3.0:1787827136.657024:0:91315:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.657029:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00800000:00000080:3.0:1787827136.657031:0:91315:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.657034:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00800000:00000080:3.0:1787827136.657049:0:91315:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.657053:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00800000:00000080:3.0:1787827136.657055:0:91315:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.657057:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00800000:00000080:3.0:1787827136.657060:0:91315:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.657063:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00800000:00000080:3.0:1787827136.657065:0:91315:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.657067:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00800000:00000080:3.0:1787827136.657069:0:91315:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.657071:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00800000:00000080:3.0:1787827136.657073:0:91315:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.657075:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00800000:00000080:3.0:1787827136.657077:0:91315:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.657079:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00800000:00000080:3.0:1787827136.657080:0:91315:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.657082:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00800000:00000080:3.0:1787827136.657084:0:91315:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.657086:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00800000:00000080:3.0:1787827136.657088:0:91315:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.657090:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00800000:00000080:3.0:1787827136.657091:0:91315:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.657093:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00800000:00000080:3.0:1787827136.657095:0:91315:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.657097:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00800000:00000080:3.0:1787827136.657099:0:91315:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.657101:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00800000:00000080:3.0:1787827136.657102:0:91315:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.657105:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00800000:00000080:3.0:1787827136.657106:0:91315:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.657109:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00800000:00000080:3.0:1787827136.657111:0:91315:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.657114:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00800000:00000080:3.0:1787827136.657116:0:91315:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.657118:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00800000:00000080:3.0:1787827136.657119:0:91315:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.657121:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00800000:00000080:3.0:1787827136.657123:0:91315:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.657125:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00800000:00000080:3.0:1787827136.657127:0:91315:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.657129:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00800000:00000080:3.0:1787827136.657130:0:91315:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.657132:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00800000:00000080:3.0:1787827136.657135:0:91315:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.657140:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00800000:00000080:3.0:1787827136.657143:0:91315:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.657147:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00800000:00000080:3.0:1787827136.657150:0:91315:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.657153:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00800000:00000080:3.0:1787827136.657155:0:91315:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.657159:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00800000:00000080:3.0:1787827136.657162:0:91315:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.657165:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00800000:00000080:3.0:1787827136.657169:0:91315:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.657173:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00800000:00000080:3.0:1787827136.657177:0:91315:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.657180:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00800000:00000080:3.0:1787827136.657184:0:91315:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.657188:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00800000:00000080:3.0:1787827136.657192:0:91315:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.657196:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00800000:00000080:3.0:1787827136.657200:0:91315:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.657204:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00800000:00000080:3.0:1787827136.657207:0:91315:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.657211:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00800000:00000080:3.0:1787827136.657215:0:91315:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.657219:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00800000:00000080:3.0:1787827136.657223:0:91315:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.657227:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00800000:00000080:3.0:1787827136.657237:0:91315:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.657243:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00800000:00000080:3.0:1787827136.657247:0:91315:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.657251:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00800000:00000080:3.0:1787827136.657254:0:91315:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.657258:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00800000:00000080:3.0:1787827136.657262:0:91315:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.657265:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00800000:00000080:3.0:1787827136.657268:0:91315:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.657271:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00800000:00000080:3.0:1787827136.657274:0:91315:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.657279:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00800000:00000080:3.0:1787827136.657282:0:91315:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.657286:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00800000:00000080:3.0:1787827136.657289:0:91315:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.657295:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00800000:00000080:3.0:1787827136.657297:0:91315:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.657301:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00800000:00000080:3.0:1787827136.657304:0:91315:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.657307:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00800000:00000080:3.0:1787827136.657311:0:91315:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.657314:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00800000:00000080:3.0:1787827136.657317:0:91315:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.657321:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00800000:00000080:3.0:1787827136.657324:0:91315:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.657328:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00800000:00000080:3.0:1787827136.657332:0:91315:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.657336:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00800000:00000080:3.0:1787827136.657339:0:91315:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.657344:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00800000:00000080:3.0:1787827136.657356:0:91315:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.657360:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00800000:00000080:3.0:1787827136.657364:0:91315:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.657368:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00800000:00000080:3.0:1787827136.657372:0:91315:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.657376:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00800000:00000080:3.0:1787827136.657380:0:91315:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.657384:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00800000:00000080:3.0:1787827136.657387:0:91315:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.657391:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00020000:00000080:3.0:1787827136.657501:0:91315:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00020000:00000004:3.0:1787827136.657508:0:91315:0:(obd_class.h:1091:obd_statfs()) lustre-OST0000-osc-ffff985602bc9800: age 3281, max_age 3286 00000100:00100000:3.0:1787827136.657544:0:91315:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:91315:1874669228204288:192.168.203.114@tcp:13:lfs.0 00000100:00100000:3.0:1787827136.657621:0:91315:0:(client.c:2715:ptlrpc_set_wait()) set ffff9856088406c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787827136.658741:0:91315:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:91315:1874669228204288:192.168.203.114@tcp:13:lfs.0 00020000:00000004:3.0:1787827136.658765:0:91315:0:(obd_class.h:1117:obd_statfs()) lustre-OST0000-osc-ffff985602bc9800: update ffff985609528450 cache blocks 901356/958279 objects 261727/262144 00000080:00200080:3.0:1787827136.658827:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00020000:00000080:3.0:1787827136.658842:0:91315:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00020000:00000004:3.0:1787827136.658850:0:91315:0:(obd_class.h:1091:obd_statfs()) lustre-OST0001-osc-ffff985602bc9800: age 3281, max_age 3286 00000100:00100000:3.0:1787827136.658890:0:91315:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:91315:1874669228204416:192.168.203.114@tcp:13:lfs.0 00000100:00100000:3.0:1787827136.658940:0:91315:0:(client.c:2715:ptlrpc_set_wait()) set ffff9856088406c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787827136.659937:0:91315:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:91315:1874669228204416:192.168.203.114@tcp:13:lfs.0 00020000:00000004:3.0:1787827136.659953:0:91315:0:(obd_class.h:1117:obd_statfs()) lustre-OST0001-osc-ffff985602bc9800: update ffff98560952d1c0 cache blocks 901363/958279 objects 261735/262144 00000080:00200080:3.0:1787827136.659995:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00020000:00000080:3.0:1787827136.660005:0:91315:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.660012:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00020000:00000080:3.0:1787827136.660017:0:91315:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.660021:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00020000:00000080:3.0:1787827136.660025:0:91315:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.660029:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00020000:00000080:3.0:1787827136.660034:0:91315:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.660052:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00020000:00000080:3.0:1787827136.660058:0:91315:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.660062:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00020000:00000080:3.0:1787827136.660066:0:91315:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.660071:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00020000:00000080:3.0:1787827136.660075:0:91315:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.660078:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00020000:00000080:3.0:1787827136.660082:0:91315:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.660086:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00020000:00000080:3.0:1787827136.660089:0:91315:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.660093:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00020000:00000080:3.0:1787827136.660096:0:91315:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.660100:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00020000:00000080:3.0:1787827136.660104:0:91315:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.660108:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00020000:00000080:3.0:1787827136.660112:0:91315:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.660116:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00020000:00000080:3.0:1787827136.660120:0:91315:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.660124:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00020000:00000080:3.0:1787827136.660128:0:91315:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.660131:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00020000:00000080:3.0:1787827136.660135:0:91315:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.660139:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00020000:00000080:3.0:1787827136.660143:0:91315:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.660148:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00020000:00000080:3.0:1787827136.660151:0:91315:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.660155:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00020000:00000080:3.0:1787827136.660158:0:91315:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.660162:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00020000:00000080:3.0:1787827136.660165:0:91315:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.660169:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00020000:00000080:3.0:1787827136.660172:0:91315:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.660177:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00020000:00000080:3.0:1787827136.660180:0:91315:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.660184:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00020000:00000080:3.0:1787827136.660188:0:91315:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.660192:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00020000:00000080:3.0:1787827136.660195:0:91315:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.660199:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00020000:00000080:3.0:1787827136.660202:0:91315:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.660207:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00020000:00000080:3.0:1787827136.660210:0:91315:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.660214:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00020000:00000080:3.0:1787827136.660218:0:91315:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.660222:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00020000:00000080:3.0:1787827136.660225:0:91315:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.660229:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00020000:00000080:3.0:1787827136.660233:0:91315:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.660237:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00020000:00000080:3.0:1787827136.660241:0:91315:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.660244:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00020000:00000080:3.0:1787827136.660248:0:91315:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.660252:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00020000:00000080:3.0:1787827136.660256:0:91315:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.660269:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00020000:00000080:3.0:1787827136.660274:0:91315:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.660282:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00020000:00000080:3.0:1787827136.660287:0:91315:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.660291:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00020000:00000080:3.0:1787827136.660294:0:91315:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.660298:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00020000:00000080:3.0:1787827136.660302:0:91315:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.660306:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00020000:00000080:3.0:1787827136.660310:0:91315:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.660314:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00020000:00000080:3.0:1787827136.660317:0:91315:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.660321:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00020000:00000080:3.0:1787827136.660325:0:91315:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.660329:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00020000:00000080:3.0:1787827136.660333:0:91315:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.660336:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00020000:00000080:3.0:1787827136.660340:0:91315:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.660353:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00020000:00000080:3.0:1787827136.660358:0:91315:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.660367:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00020000:00000080:3.0:1787827136.660371:0:91315:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.660375:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00020000:00000080:3.0:1787827136.660379:0:91315:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.660383:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00020000:00000080:3.0:1787827136.660386:0:91315:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.660390:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00020000:00000080:3.0:1787827136.660393:0:91315:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.660397:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00020000:00000080:3.0:1787827136.660400:0:91315:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.660404:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00020000:00000080:3.0:1787827136.660408:0:91315:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.660412:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00020000:00000080:3.0:1787827136.660415:0:91315:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.660419:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00020000:00000080:3.0:1787827136.660422:0:91315:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.660426:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00020000:00000080:3.0:1787827136.660429:0:91315:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.660433:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00020000:00000080:3.0:1787827136.660436:0:91315:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.660440:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00020000:00000080:3.0:1787827136.660455:0:91315:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.660462:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00020000:00000080:3.0:1787827136.660466:0:91315:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.660471:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00020000:00000080:3.0:1787827136.660475:0:91315:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.660480:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00020000:00000080:3.0:1787827136.660483:0:91315:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.660487:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00020000:00000080:3.0:1787827136.660490:0:91315:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.660494:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00020000:00000080:3.0:1787827136.660498:0:91315:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.660502:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00020000:00000080:3.0:1787827136.660505:0:91315:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.660509:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00020000:00000080:3.0:1787827136.660512:0:91315:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.660516:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00020000:00000080:3.0:1787827136.660520:0:91315:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.660524:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00020000:00000080:3.0:1787827136.660527:0:91315:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.660530:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00020000:00000080:3.0:1787827136.660534:0:91315:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200080:3.0:1787827136.660538:0:91315:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0ceb8990 00020000:00000080:3.0:1787827136.660541:0:91315:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608491c00 uarg=0000000000000000 00000080:00200008:3.0:1787827136.660564:0:91315:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 100000 00000020:00000080:1.0:1787827137.314900:0:91334:0:(class_obd.c:336:class_handle_ioctl()) obdclass: cmd=40086685 len=0 uarg=00007ffffccc2970 00000002:00000080:1.0:1787827137.314917:0:91334:0:(mdc_request.c:2317:mdc_iocontrol()) lustre-MDT0000-mdc-ffff985602bc9800: cmd=40086685 len=576 karg=ffff98560865f000 uarg=0000000000000000 00000100:00080000:1.0:1787827137.314925:0:91334:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from FULL to DISCONN 00000100:02000400:1.0:1787827137.314932:0:91334:0:(import.c:182:ptlrpc_set_import_discon()) lustre-MDT0000-mdc-ffff985602bc9800: Connection to lustre-MDT0000 (at 192.168.203.114@tcp) was lost; in progress operations using this service will wait for recovery to complete 00000100:00080000:1.0:1787827137.314940:0:91334:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827137.314945:0:91334:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3281 00000100:00080000:1.0:1787827137.314959:0:91334:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787827137.314969:0:91334:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827137.314996:0:91334:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a1680 x1874669228204544/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827137.315019:0:91334:0:(recover.c:337:ptlrpc_recover_import()) lustre-MDT0000_UUID: recovery started, waiting 20 seconds 00000100:00100000:0.0:1787827137.315076:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228204544:192.168.203.114@tcp:38:lctl.0 00010000:00010000:0.0:1787827137.332782:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827138.356939:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827139.380846:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827140.404853:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827140.852955:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2383:1874669228204672:192.168.203.114@tcp:400:kworker.0 00000100:00080000:0.0:1787827140.853279:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827140.853285:0:54101:0:(pinger.c:247:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:1.0:1787827140.853398:0:2383:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1787827140.853407:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669228204800:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827140.853877:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669228205056:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827140.853917:0:2384:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1787827140.853923:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669228204928:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827140.854419:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2383:1874669228204672:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827140.855974:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669228204928:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827140.855988:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669228205056:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787827140.856130:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669228204800:192.168.203.114@tcp:400:kworker.0 00010000:00010000:2.0:1787827141.428960:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827141.877072:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827141.877080:0:54101:0:(pinger.c:247:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1787827141.877129:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2385:1874669228205184:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787827141.877157:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228205312:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787827141.877217:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669228205440:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787827141.877227:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228205568:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787827141.878651:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228205312:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787827141.878670:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228205568:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787827141.878670:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2385:1874669228205184:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787827141.878734:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669228205440:192.168.203.114@tcp:400:kworker.0 00010000:00010000:0.0:1787827142.452848:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827142.792128:0:2382:0:(import.c:1132:ptlrpc_connect_interpret()) lustre-MDT0000-mdc-ffff985602bc9800: connect to target with instance 52 00000100:00080000:0.0:1787827142.792157:0:2382:0:(import.c:1297:ptlrpc_connect_interpret()) reconnected to lustre-MDT0000_UUID@192.168.203.114@tcp after partition 00000100:00080000:0.0:1787827142.792160:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to RECOVER 00000100:00080000:0.0:1787827142.792165:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from RECOVER to FULL 00000100:02000000:0.0:1787827142.792506:0:2382:0:(import.c:1743:ptlrpc_import_recovery_state_machine()) lustre-MDT0000-mdc-ffff985602bc9800: Connection restored to 192.168.203.114@tcp (at 192.168.203.114@tcp) 00000100:00100000:0.0:1787827142.792522:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228204544:192.168.203.114@tcp:38:lctl.0 00000100:00080000:1.0:1787827142.792551:0:91334:0:(recover.c:348:ptlrpc_recover_import()) lustre-MDT0000_UUID: recovery finished FULL, rc = 0 00000080:00200008:1.0:1787827142.798126:0:91336:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:1.0:1787827142.798139:0:91336:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787827142.798160:0:91336:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827142.798180:0:91336:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 12 lru_type: 1 00010000:00010000:1.0:1787827142.798195:0:91336:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 12 lru_type: 1 00000080:00200000:1.0:1787827142.798203:0:91336:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827142.798209:0:91336:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787827142.798212:0:91336:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee56d 00000080:00010000:1.0:1787827142.798220:0:91336:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee56d from it ffffab3ac1db7cb8 00010000:00010000:1.0:1787827142.798223:0:91336:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 12 lru_type: 1 00010000:00010000:1.0:1787827142.798231:0:91336:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 13 lru_type: 1 00000080:00200008:1.0:1787827142.798758:0:91336:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 0, flags 0, foreign 0 00000080:00200000:1.0:1787827142.798763:0:91336:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787827142.798772:0:91336:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827142.798781:0:91336:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 13 lru_type: 1 00010000:00010000:1.0:1787827142.798790:0:91336:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 13 lru_type: 1 00000080:00200000:1.0:1787827142.798797:0:91336:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827142.798801:0:91336:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787827142.798803:0:91336:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee56d 00000080:00010000:1.0:1787827142.798808:0:91336:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee56d from it ffffab3ac1db7cb8 00010000:00010000:1.0:1787827142.798811:0:91336:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 13 lru_type: 1 00010000:00010000:1.0:1787827142.798819:0:91336:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 14 lru_type: 1 00000080:00200000:1.0:1787827142.798900:0:91336:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787827142.798910:0:91336:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827142.798918:0:91336:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 14 lru_type: 1 00010000:00010000:1.0:1787827142.798928:0:91336:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 14 lru_type: 1 00000080:00200000:1.0:1787827142.798935:0:91336:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827142.798939:0:91336:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787827142.798942:0:91336:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee56d 00000080:00010000:1.0:1787827142.798946:0:91336:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee56d from it ffffab3ac1db7bb0 00010000:00010000:1.0:1787827142.798950:0:91336:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 14 lru_type: 1 00010000:00010000:1.0:1787827142.798959:0:91336:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 15 lru_type: 1 00000080:00200000:1.0:1787827142.798967:0:91336:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 44 00000080:00200008:1.0:1787827142.798974:0:91336:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 100000 00000080:00200080:1.0:1787827142.799076:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00800000:00000080:1.0:1787827142.799091:0:91336:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00800000:00000004:1.0:1787827142.799098:0:91336:0:(obd_class.h:1091:obd_statfs()) lustre-MDT0000-mdc-ffff985602bc9800: age 3287, max_age 3292 00000100:00100000:1.0:1787827142.799137:0:91336:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a03c0 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:91336:1874669228205696:192.168.203.114@tcp:41:lfs.0 00000100:00100000:1.0:1787827142.799218:0:91336:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5be40 going to sleep for 12 seconds 00000100:00100000:1.0:1787827142.800800:0:91336:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a03c0 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:91336:1874669228205696:192.168.203.114@tcp:41:lfs.0 00800000:00000004:1.0:1787827142.800822:0:91336:0:(obd_class.h:1117:obd_statfs()) lustre-MDT0000-mdc-ffff985602bc9800: update ffff985606a337f0 cache blocks 317333/353529 objects 1023688/1024000 00000080:00200080:1.0:1787827142.800875:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00800000:00000080:1.0:1787827142.800887:0:91336:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00800000:00000004:1.0:1787827142.800894:0:91336:0:(obd_class.h:1091:obd_statfs()) lustre-MDT0001-mdc-ffff985602bc9800: age 3287, max_age 3292 00000100:00100000:1.0:1787827142.800916:0:91336:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a03c0 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:91336:1874669228205824:192.168.203.114@tcp:41:lfs.0 00000100:00100000:1.0:1787827142.800957:0:91336:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5be40 going to sleep for 12 seconds 00000100:00100000:1.0:1787827142.802208:0:91336:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a03c0 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:91336:1874669228205824:192.168.203.114@tcp:41:lfs.0 00800000:00000004:1.0:1787827142.802224:0:91336:0:(obd_class.h:1117:obd_statfs()) lustre-MDT0001-mdc-ffff985602bc9800: update ffff98560952b7f0 cache blocks 317401/353529 objects 1023737/1024000 00000080:00200080:1.0:1787827142.802264:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00800000:00000080:1.0:1787827142.802275:0:91336:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.802282:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00800000:00000080:1.0:1787827142.802286:0:91336:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.802290:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00800000:00000080:1.0:1787827142.802294:0:91336:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.802299:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00800000:00000080:1.0:1787827142.802303:0:91336:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.802308:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00800000:00000080:1.0:1787827142.802312:0:91336:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.802317:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00800000:00000080:1.0:1787827142.802321:0:91336:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.802326:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00800000:00000080:1.0:1787827142.802330:0:91336:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.802334:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00800000:00000080:1.0:1787827142.802348:0:91336:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.802354:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00800000:00000080:1.0:1787827142.802359:0:91336:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.802363:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00800000:00000080:1.0:1787827142.802366:0:91336:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.802370:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00800000:00000080:1.0:1787827142.802374:0:91336:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.802377:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00800000:00000080:1.0:1787827142.802381:0:91336:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.802384:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00800000:00000080:1.0:1787827142.802388:0:91336:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.802402:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00800000:00000080:1.0:1787827142.802406:0:91336:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.802410:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00800000:00000080:1.0:1787827142.802413:0:91336:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.802417:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00800000:00000080:1.0:1787827142.802420:0:91336:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.802423:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00800000:00000080:1.0:1787827142.802427:0:91336:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.802431:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00800000:00000080:1.0:1787827142.802433:0:91336:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.802438:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00800000:00000080:1.0:1787827142.802450:0:91336:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.802455:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00800000:00000080:1.0:1787827142.802458:0:91336:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.802462:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00800000:00000080:1.0:1787827142.802465:0:91336:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.802469:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00800000:00000080:1.0:1787827142.802472:0:91336:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.802476:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00800000:00000080:1.0:1787827142.802479:0:91336:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.802483:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00800000:00000080:1.0:1787827142.802487:0:91336:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.802491:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00800000:00000080:1.0:1787827142.802494:0:91336:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.802498:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00800000:00000080:1.0:1787827142.802502:0:91336:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.802505:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00800000:00000080:1.0:1787827142.802509:0:91336:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.802512:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00800000:00000080:1.0:1787827142.802515:0:91336:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.802519:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00800000:00000080:1.0:1787827142.802522:0:91336:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.802525:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00800000:00000080:1.0:1787827142.802529:0:91336:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.802532:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00800000:00000080:1.0:1787827142.802536:0:91336:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.802540:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00800000:00000080:1.0:1787827142.802543:0:91336:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.802547:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00800000:00000080:1.0:1787827142.802551:0:91336:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.802554:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00800000:00000080:1.0:1787827142.802558:0:91336:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.802561:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00800000:00000080:1.0:1787827142.802564:0:91336:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.802568:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00800000:00000080:1.0:1787827142.802571:0:91336:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.802574:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00800000:00000080:1.0:1787827142.802577:0:91336:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.802581:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00800000:00000080:1.0:1787827142.802584:0:91336:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.802588:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00800000:00000080:1.0:1787827142.802591:0:91336:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.802595:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00800000:00000080:1.0:1787827142.802598:0:91336:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.802602:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00800000:00000080:1.0:1787827142.802605:0:91336:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.802608:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00800000:00000080:1.0:1787827142.802611:0:91336:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.802615:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00800000:00000080:1.0:1787827142.802618:0:91336:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.802622:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00800000:00000080:1.0:1787827142.802626:0:91336:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.802630:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00800000:00000080:1.0:1787827142.802633:0:91336:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.802637:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00800000:00000080:1.0:1787827142.802640:0:91336:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.802644:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00800000:00000080:1.0:1787827142.802648:0:91336:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.802652:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00800000:00000080:1.0:1787827142.802655:0:91336:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.802659:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00800000:00000080:1.0:1787827142.802663:0:91336:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.802666:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00800000:00000080:1.0:1787827142.802670:0:91336:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.802674:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00800000:00000080:1.0:1787827142.802677:0:91336:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.802681:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00800000:00000080:1.0:1787827142.802684:0:91336:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.802688:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00800000:00000080:1.0:1787827142.802712:0:91336:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.802719:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00800000:00000080:1.0:1787827142.802723:0:91336:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.802727:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00800000:00000080:1.0:1787827142.802732:0:91336:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.802736:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00800000:00000080:1.0:1787827142.802740:0:91336:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.802744:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00800000:00000080:1.0:1787827142.802747:0:91336:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.802751:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00800000:00000080:1.0:1787827142.802754:0:91336:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.802758:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00800000:00000080:1.0:1787827142.802761:0:91336:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.802765:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00800000:00000080:1.0:1787827142.802769:0:91336:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.802773:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00800000:00000080:1.0:1787827142.802776:0:91336:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.802780:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00800000:00000080:1.0:1787827142.802783:0:91336:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.802787:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00800000:00000080:1.0:1787827142.802790:0:91336:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.802794:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00020000:00000080:1.0:1787827142.802800:0:91336:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00020000:00000004:1.0:1787827142.802807:0:91336:0:(obd_class.h:1091:obd_statfs()) lustre-OST0000-osc-ffff985602bc9800: age 3287, max_age 3292 00000100:00100000:1.0:1787827142.802831:0:91336:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a03c0 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:91336:1874669228205952:192.168.203.114@tcp:13:lfs.0 00000100:00100000:1.0:1787827142.802892:0:91336:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5be40 going to sleep for 11 seconds 00000100:00100000:1.0:1787827142.804223:0:91336:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a03c0 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:91336:1874669228205952:192.168.203.114@tcp:13:lfs.0 00020000:00000004:1.0:1787827142.804251:0:91336:0:(obd_class.h:1117:obd_statfs()) lustre-OST0000-osc-ffff985602bc9800: update ffff985609528450 cache blocks 901356/958279 objects 261727/262144 00000080:00200080:1.0:1787827142.804327:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00020000:00000080:1.0:1787827142.804344:0:91336:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00020000:00000004:1.0:1787827142.804355:0:91336:0:(obd_class.h:1091:obd_statfs()) lustre-OST0001-osc-ffff985602bc9800: age 3287, max_age 3292 00000100:00100000:1.0:1787827142.804405:0:91336:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a03c0 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:91336:1874669228206080:192.168.203.114@tcp:13:lfs.0 00000100:00100000:1.0:1787827142.804482:0:91336:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5be40 going to sleep for 11 seconds 00000100:00100000:1.0:1787827142.807436:0:91336:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a03c0 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:91336:1874669228206080:192.168.203.114@tcp:13:lfs.0 00020000:00000004:1.0:1787827142.807474:0:91336:0:(obd_class.h:1117:obd_statfs()) lustre-OST0001-osc-ffff985602bc9800: update ffff98560952d1c0 cache blocks 901363/958279 objects 261735/262144 00000080:00200080:1.0:1787827142.807580:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00020000:00000080:1.0:1787827142.807604:0:91336:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.807612:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00020000:00000080:1.0:1787827142.807617:0:91336:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.807622:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00020000:00000080:1.0:1787827142.807625:0:91336:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.807629:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00020000:00000080:1.0:1787827142.807632:0:91336:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.807636:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00020000:00000080:1.0:1787827142.807639:0:91336:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.807644:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00020000:00000080:1.0:1787827142.807648:0:91336:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.807652:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00020000:00000080:1.0:1787827142.807655:0:91336:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.807659:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00020000:00000080:1.0:1787827142.807663:0:91336:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.807666:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00020000:00000080:1.0:1787827142.807676:0:91336:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.807681:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00020000:00000080:1.0:1787827142.807685:0:91336:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.807913:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00020000:00000080:1.0:1787827142.807942:0:91336:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.807961:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00020000:00000080:1.0:1787827142.807967:0:91336:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.807971:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00020000:00000080:1.0:1787827142.807975:0:91336:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.807979:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00020000:00000080:1.0:1787827142.807983:0:91336:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.807988:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00020000:00000080:1.0:1787827142.807991:0:91336:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.808009:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00020000:00000080:1.0:1787827142.808014:0:91336:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.808018:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00020000:00000080:1.0:1787827142.808022:0:91336:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.808025:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00020000:00000080:1.0:1787827142.808029:0:91336:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.808033:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00020000:00000080:1.0:1787827142.808037:0:91336:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.808041:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00020000:00000080:1.0:1787827142.808045:0:91336:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.808048:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00020000:00000080:1.0:1787827142.808052:0:91336:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.808056:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00020000:00000080:1.0:1787827142.808059:0:91336:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.808064:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00020000:00000080:1.0:1787827142.808068:0:91336:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.808072:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00020000:00000080:1.0:1787827142.808076:0:91336:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.808080:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00020000:00000080:1.0:1787827142.808083:0:91336:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.808087:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00020000:00000080:1.0:1787827142.808091:0:91336:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.808094:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00020000:00000080:1.0:1787827142.808098:0:91336:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.808102:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00020000:00000080:1.0:1787827142.808106:0:91336:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.808110:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00020000:00000080:1.0:1787827142.808114:0:91336:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.808117:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00020000:00000080:1.0:1787827142.808121:0:91336:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.808134:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00020000:00000080:1.0:1787827142.808142:0:91336:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.808147:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00020000:00000080:1.0:1787827142.808151:0:91336:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.808155:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00020000:00000080:1.0:1787827142.808159:0:91336:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.808163:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00020000:00000080:1.0:1787827142.808166:0:91336:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.808170:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00020000:00000080:1.0:1787827142.808174:0:91336:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.808177:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00020000:00000080:1.0:1787827142.808181:0:91336:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.808184:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00020000:00000080:1.0:1787827142.808188:0:91336:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.808192:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00020000:00000080:1.0:1787827142.808195:0:91336:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.808199:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00020000:00000080:1.0:1787827142.808202:0:91336:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.808207:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00020000:00000080:1.0:1787827142.808210:0:91336:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.808214:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00020000:00000080:1.0:1787827142.808217:0:91336:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.808221:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00020000:00000080:1.0:1787827142.808224:0:91336:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.808228:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00020000:00000080:1.0:1787827142.808231:0:91336:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.808235:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00020000:00000080:1.0:1787827142.808239:0:91336:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.808242:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00020000:00000080:1.0:1787827142.808246:0:91336:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.808249:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00020000:00000080:1.0:1787827142.808252:0:91336:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.808256:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00020000:00000080:1.0:1787827142.808259:0:91336:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.808264:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00020000:00000080:1.0:1787827142.808268:0:91336:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.808272:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00020000:00000080:1.0:1787827142.808276:0:91336:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.808280:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00020000:00000080:1.0:1787827142.808283:0:91336:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.808288:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00020000:00000080:1.0:1787827142.808291:0:91336:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.808295:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00020000:00000080:1.0:1787827142.808299:0:91336:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.808303:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00020000:00000080:1.0:1787827142.808312:0:91336:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.808317:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00020000:00000080:1.0:1787827142.808321:0:91336:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.808325:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00020000:00000080:1.0:1787827142.808328:0:91336:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.808332:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00020000:00000080:1.0:1787827142.808336:0:91336:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.808340:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00020000:00000080:1.0:1787827142.808343:0:91336:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.808347:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00020000:00000080:1.0:1787827142.808351:0:91336:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.808355:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00020000:00000080:1.0:1787827142.808359:0:91336:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.808363:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00020000:00000080:1.0:1787827142.808368:0:91336:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.808372:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00020000:00000080:1.0:1787827142.808376:0:91336:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.808380:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00020000:00000080:1.0:1787827142.808383:0:91336:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200080:1.0:1787827142.808388:0:91336:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffc0859f340 00020000:00000080:1.0:1787827142.808391:0:91336:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00000080:00200008:1.0:1787827142.808421:0:91336:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 100000 00010000:00010000:3.0:1787827143.476877:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000080:0.0:1787827143.497245:0:91355:0:(class_obd.c:336:class_handle_ioctl()) obdclass: cmd=40086685 len=0 uarg=00007ffeb5443bd0 00000002:00000080:0.0:1787827143.497265:0:91355:0:(mdc_request.c:2317:mdc_iocontrol()) lustre-MDT0000-mdc-ffff985602bc9800: cmd=40086685 len=576 karg=ffff985608616c00 uarg=0000000000000000 00000100:00080000:0.0:1787827143.497273:0:91355:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from FULL to DISCONN 00000100:02000400:0.0:1787827143.497281:0:91355:0:(import.c:182:ptlrpc_set_import_discon()) lustre-MDT0000-mdc-ffff985602bc9800: Connection to lustre-MDT0000 (at 192.168.203.114@tcp) was lost; in progress operations using this service will wait for recovery to complete 00000100:00080000:0.0:1787827143.497288:0:91355:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827143.497293:0:91355:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3288 00000100:00080000:0.0:1787827143.497308:0:91355:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787827143.497320:0:91355:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827143.497345:0:91355:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5e00 x1874669228206208/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827143.497376:0:91355:0:(recover.c:337:ptlrpc_recover_import()) lustre-MDT0000_UUID: recovery started, waiting 20 seconds 00000100:00100000:3.0:1787827143.497449:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228206208:192.168.203.114@tcp:38:lctl.0 00010000:00010000:3.0:1787827144.500897:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827145.524917:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827146.548935:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827146.996886:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827146.996892:0:66417:0:(pinger.c:247:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:0.0:1787827146.996924:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2383:1874669228206336:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827146.996935:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669228206464:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787827146.997021:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669228206592:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827146.997021:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669228206720:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787827146.998073:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2383:1874669228206336:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827146.998099:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669228206464:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787827146.998301:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669228206592:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827146.998325:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669228206720:192.168.203.114@tcp:400:kworker.0 00010000:00010000:0.0:1787827147.572810:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827148.020863:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827148.020870:0:66417:0:(pinger.c:247:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:0.0:1787827148.020910:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2383:1874669228206848:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827148.020914:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669228206976:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787827148.021015:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669228207104:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827148.021018:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669228207232:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827148.022404:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669228206976:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827148.022421:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669228207232:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787827148.022455:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2383:1874669228206848:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787827148.022467:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669228207104:192.168.203.114@tcp:400:kworker.0 00010000:00010000:3.0:1787827148.597071:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827148.932299:0:2382:0:(import.c:1132:ptlrpc_connect_interpret()) lustre-MDT0000-mdc-ffff985602bc9800: connect to target with instance 52 00000100:00080000:3.0:1787827148.932340:0:2382:0:(import.c:1297:ptlrpc_connect_interpret()) reconnected to lustre-MDT0000_UUID@192.168.203.114@tcp after partition 00000100:00080000:3.0:1787827148.932345:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to RECOVER 00000100:00080000:3.0:1787827148.932353:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from RECOVER to FULL 00000100:02000000:0.0:1787827148.932656:0:2382:0:(import.c:1743:ptlrpc_import_recovery_state_machine()) lustre-MDT0000-mdc-ffff985602bc9800: Connection restored to 192.168.203.114@tcp (at 192.168.203.114@tcp) 00000100:00100000:0.0:1787827148.932681:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228206208:192.168.203.114@tcp:38:lctl.0 00000100:00080000:1.0:1787827148.932739:0:91355:0:(recover.c:348:ptlrpc_recover_import()) lustre-MDT0000_UUID: recovery finished FULL, rc = 0 00000080:00200008:0.0:1787827148.938266:0:91358:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:0.0:1787827148.938276:0:91358:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:0.0:1787827148.938297:0:91358:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827148.938312:0:91358:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 15 lru_type: 1 00010000:00010000:0.0:1787827148.938323:0:91358:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 15 lru_type: 1 00000080:00200000:0.0:1787827148.938329:0:91358:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827148.938335:0:91358:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:0.0:1787827148.938337:0:91358:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee56d 00000080:00010000:0.0:1787827148.938342:0:91358:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee56d from it ffffab3ac1dffcb8 00010000:00010000:0.0:1787827148.938345:0:91358:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 15 lru_type: 1 00010000:00010000:0.0:1787827148.938351:0:91358:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 16 lru_type: 1 00000080:00200008:0.0:1787827148.938887:0:91358:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 0, flags 0, foreign 0 00000080:00200000:0.0:1787827148.938892:0:91358:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:0.0:1787827148.938900:0:91358:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827148.938908:0:91358:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 16 lru_type: 1 00010000:00010000:0.0:1787827148.938918:0:91358:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 16 lru_type: 1 00000080:00200000:0.0:1787827148.938925:0:91358:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827148.938929:0:91358:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:0.0:1787827148.938932:0:91358:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee56d 00000080:00010000:0.0:1787827148.938937:0:91358:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee56d from it ffffab3ac1dffcb8 00010000:00010000:0.0:1787827148.938940:0:91358:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 16 lru_type: 1 00010000:00010000:0.0:1787827148.938968:0:91358:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 17 lru_type: 1 00000080:00200000:0.0:1787827148.939047:0:91358:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:0.0:1787827148.939055:0:91358:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827148.939061:0:91358:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 17 lru_type: 1 00010000:00010000:0.0:1787827148.939070:0:91358:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 17 lru_type: 1 00000080:00200000:0.0:1787827148.939076:0:91358:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827148.939080:0:91358:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:0.0:1787827148.939082:0:91358:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee56d 00000080:00010000:0.0:1787827148.939085:0:91358:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee56d from it ffffab3ac1dffbb0 00010000:00010000:0.0:1787827148.939089:0:91358:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 17 lru_type: 1 00010000:00010000:0.0:1787827148.939096:0:91358:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 18 lru_type: 1 00000080:00200000:0.0:1787827148.939105:0:91358:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 44 00000080:00200008:0.0:1787827148.939112:0:91358:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 100000 00000080:00200080:0.0:1787827148.939182:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00800000:00000080:0.0:1787827148.939197:0:91358:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00800000:00000004:0.0:1787827148.939204:0:91358:0:(obd_class.h:1091:obd_statfs()) lustre-MDT0000-mdc-ffff985602bc9800: age 3293, max_age 3298 00000100:00100000:0.0:1787827148.939241:0:91358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:91358:1874669228207360:192.168.203.114@tcp:41:lfs.0 00000100:00100000:0.0:1787827148.939482:0:91358:0:(client.c:2715:ptlrpc_set_wait()) set ffff98560860b180 going to sleep for 12 seconds 00000100:00100000:0.0:1787827148.940724:0:91358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:91358:1874669228207360:192.168.203.114@tcp:41:lfs.0 00800000:00000004:0.0:1787827148.940748:0:91358:0:(obd_class.h:1117:obd_statfs()) lustre-MDT0000-mdc-ffff985602bc9800: update ffff985606a337f0 cache blocks 317333/353529 objects 1023688/1024000 00000080:00200080:0.0:1787827148.940787:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00800000:00000080:0.0:1787827148.940795:0:91358:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00800000:00000004:0.0:1787827148.940799:0:91358:0:(obd_class.h:1091:obd_statfs()) lustre-MDT0001-mdc-ffff985602bc9800: age 3293, max_age 3298 00000100:00100000:0.0:1787827148.940815:0:91358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:91358:1874669228207488:192.168.203.114@tcp:41:lfs.0 00000100:00100000:0.0:1787827148.940849:0:91358:0:(client.c:2715:ptlrpc_set_wait()) set ffff98560860b180 going to sleep for 12 seconds 00000100:00100000:0.0:1787827148.941833:0:91358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:91358:1874669228207488:192.168.203.114@tcp:41:lfs.0 00800000:00000004:0.0:1787827148.941846:0:91358:0:(obd_class.h:1117:obd_statfs()) lustre-MDT0001-mdc-ffff985602bc9800: update ffff98560952b7f0 cache blocks 317401/353529 objects 1023737/1024000 00000080:00200080:0.0:1787827148.941886:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00800000:00000080:0.0:1787827148.941896:0:91358:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.941903:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00800000:00000080:0.0:1787827148.941908:0:91358:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.941913:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00800000:00000080:0.0:1787827148.941917:0:91358:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.941922:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00800000:00000080:0.0:1787827148.941926:0:91358:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.941931:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00800000:00000080:0.0:1787827148.941949:0:91358:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.941965:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00800000:00000080:0.0:1787827148.941970:0:91358:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.941974:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00800000:00000080:0.0:1787827148.941979:0:91358:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.941983:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00800000:00000080:0.0:1787827148.941987:0:91358:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.941991:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00800000:00000080:0.0:1787827148.941994:0:91358:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.941998:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00800000:00000080:0.0:1787827148.942001:0:91358:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.942006:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00800000:00000080:0.0:1787827148.942009:0:91358:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.942014:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00800000:00000080:0.0:1787827148.942017:0:91358:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.942021:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00800000:00000080:0.0:1787827148.942025:0:91358:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.942029:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00800000:00000080:0.0:1787827148.942032:0:91358:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.942036:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00800000:00000080:0.0:1787827148.942040:0:91358:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.942044:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00800000:00000080:0.0:1787827148.942047:0:91358:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.942052:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00800000:00000080:0.0:1787827148.942055:0:91358:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.942059:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00800000:00000080:0.0:1787827148.942061:0:91358:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.942065:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00800000:00000080:0.0:1787827148.942068:0:91358:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.942071:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00800000:00000080:0.0:1787827148.942074:0:91358:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.942078:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00800000:00000080:0.0:1787827148.942080:0:91358:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.942083:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00800000:00000080:0.0:1787827148.942086:0:91358:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.942089:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00800000:00000080:0.0:1787827148.942099:0:91358:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.942103:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00800000:00000080:0.0:1787827148.942105:0:91358:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.942109:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00800000:00000080:0.0:1787827148.942113:0:91358:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.942116:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00800000:00000080:0.0:1787827148.942119:0:91358:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.942122:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00800000:00000080:0.0:1787827148.942125:0:91358:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.942128:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00800000:00000080:0.0:1787827148.942132:0:91358:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.942136:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00800000:00000080:0.0:1787827148.942139:0:91358:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.942143:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00800000:00000080:0.0:1787827148.942147:0:91358:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.942151:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00800000:00000080:0.0:1787827148.942154:0:91358:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.942158:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00800000:00000080:0.0:1787827148.942162:0:91358:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.942165:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00800000:00000080:0.0:1787827148.942169:0:91358:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.942172:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00800000:00000080:0.0:1787827148.942175:0:91358:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.942178:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00800000:00000080:0.0:1787827148.942182:0:91358:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.942186:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00800000:00000080:0.0:1787827148.942189:0:91358:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.942192:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00800000:00000080:0.0:1787827148.942196:0:91358:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.942199:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00800000:00000080:0.0:1787827148.942202:0:91358:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.942206:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00800000:00000080:0.0:1787827148.942209:0:91358:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.942212:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00800000:00000080:0.0:1787827148.942216:0:91358:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.942219:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00800000:00000080:0.0:1787827148.942222:0:91358:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.942226:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00800000:00000080:0.0:1787827148.942229:0:91358:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.942233:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00800000:00000080:0.0:1787827148.942236:0:91358:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.942239:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00800000:00000080:0.0:1787827148.942242:0:91358:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.942246:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00800000:00000080:0.0:1787827148.942249:0:91358:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.942252:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00800000:00000080:0.0:1787827148.942255:0:91358:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.942259:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00800000:00000080:0.0:1787827148.942263:0:91358:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.942266:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00800000:00000080:0.0:1787827148.942269:0:91358:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.942272:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00800000:00000080:0.0:1787827148.942274:0:91358:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.942278:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00800000:00000080:0.0:1787827148.942281:0:91358:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.942285:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00800000:00000080:0.0:1787827148.942288:0:91358:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.942292:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00800000:00000080:0.0:1787827148.942296:0:91358:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.942300:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00800000:00000080:0.0:1787827148.942303:0:91358:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.942307:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00800000:00000080:0.0:1787827148.942311:0:91358:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.942315:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00800000:00000080:0.0:1787827148.942318:0:91358:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.942323:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00800000:00000080:0.0:1787827148.942326:0:91358:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.942330:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00800000:00000080:0.0:1787827148.942334:0:91358:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.942338:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00800000:00000080:0.0:1787827148.942342:0:91358:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.942345:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00800000:00000080:0.0:1787827148.942350:0:91358:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.942354:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00800000:00000080:0.0:1787827148.942357:0:91358:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.942361:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00800000:00000080:0.0:1787827148.942365:0:91358:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.942369:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00800000:00000080:0.0:1787827148.942372:0:91358:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.942376:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00800000:00000080:0.0:1787827148.942379:0:91358:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.942383:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00020000:00000080:0.0:1787827148.942389:0:91358:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00020000:00000004:0.0:1787827148.942397:0:91358:0:(obd_class.h:1091:obd_statfs()) lustre-OST0000-osc-ffff985602bc9800: age 3293, max_age 3298 00000100:00100000:0.0:1787827148.942417:0:91358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:91358:1874669228207616:192.168.203.114@tcp:13:lfs.0 00000100:00100000:0.0:1787827148.942459:0:91358:0:(client.c:2715:ptlrpc_set_wait()) set ffff98560860b180 going to sleep for 11 seconds 00000100:00100000:0.0:1787827148.943556:0:91358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:91358:1874669228207616:192.168.203.114@tcp:13:lfs.0 00020000:00000004:0.0:1787827148.943571:0:91358:0:(obd_class.h:1117:obd_statfs()) lustre-OST0000-osc-ffff985602bc9800: update ffff985609528450 cache blocks 901356/958279 objects 261727/262144 00000080:00200080:0.0:1787827148.943618:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00020000:00000080:0.0:1787827148.943628:0:91358:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00020000:00000004:0.0:1787827148.943633:0:91358:0:(obd_class.h:1091:obd_statfs()) lustre-OST0001-osc-ffff985602bc9800: age 3293, max_age 3298 00000100:00100000:0.0:1787827148.943652:0:91358:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:91358:1874669228207744:192.168.203.114@tcp:13:lfs.0 00000100:00100000:0.0:1787827148.943714:0:91358:0:(client.c:2715:ptlrpc_set_wait()) set ffff98560860b180 going to sleep for 11 seconds 00000100:00100000:0.0:1787827148.944563:0:91358:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:91358:1874669228207744:192.168.203.114@tcp:13:lfs.0 00020000:00000004:0.0:1787827148.944575:0:91358:0:(obd_class.h:1117:obd_statfs()) lustre-OST0001-osc-ffff985602bc9800: update ffff98560952d1c0 cache blocks 901363/958279 objects 261735/262144 00000080:00200080:0.0:1787827148.944604:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00020000:00000080:0.0:1787827148.944612:0:91358:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.944618:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00020000:00000080:0.0:1787827148.944623:0:91358:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.944628:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00020000:00000080:0.0:1787827148.944632:0:91358:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.944637:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00020000:00000080:0.0:1787827148.944640:0:91358:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.944645:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00020000:00000080:0.0:1787827148.944649:0:91358:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.944654:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00020000:00000080:0.0:1787827148.944657:0:91358:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.944661:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00020000:00000080:0.0:1787827148.944664:0:91358:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.944667:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00020000:00000080:0.0:1787827148.944671:0:91358:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.944675:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00020000:00000080:0.0:1787827148.944678:0:91358:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.944682:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00020000:00000080:0.0:1787827148.944685:0:91358:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.944689:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00020000:00000080:0.0:1787827148.944716:0:91358:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.944721:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00020000:00000080:0.0:1787827148.944726:0:91358:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.944732:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00020000:00000080:0.0:1787827148.944736:0:91358:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.944752:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00020000:00000080:0.0:1787827148.944757:0:91358:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.944762:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00020000:00000080:0.0:1787827148.944765:0:91358:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.944769:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00020000:00000080:0.0:1787827148.944773:0:91358:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.944777:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00020000:00000080:0.0:1787827148.944780:0:91358:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.944784:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00020000:00000080:0.0:1787827148.944788:0:91358:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.944792:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00020000:00000080:0.0:1787827148.944795:0:91358:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.944798:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00020000:00000080:0.0:1787827148.944802:0:91358:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.944806:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00020000:00000080:0.0:1787827148.944809:0:91358:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.944813:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00020000:00000080:0.0:1787827148.944816:0:91358:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.944820:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00020000:00000080:0.0:1787827148.944823:0:91358:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.944827:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00020000:00000080:0.0:1787827148.944831:0:91358:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.944834:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00020000:00000080:0.0:1787827148.944837:0:91358:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.944842:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00020000:00000080:0.0:1787827148.944845:0:91358:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.944849:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00020000:00000080:0.0:1787827148.944852:0:91358:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.944856:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00020000:00000080:0.0:1787827148.944860:0:91358:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.944864:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00020000:00000080:0.0:1787827148.944867:0:91358:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.944871:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00020000:00000080:0.0:1787827148.944874:0:91358:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.944878:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00020000:00000080:0.0:1787827148.944881:0:91358:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.944885:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00020000:00000080:0.0:1787827148.944888:0:91358:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.944892:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00020000:00000080:0.0:1787827148.944895:0:91358:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.944899:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00020000:00000080:0.0:1787827148.944902:0:91358:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.944906:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00020000:00000080:0.0:1787827148.944909:0:91358:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.944913:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00020000:00000080:0.0:1787827148.944916:0:91358:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.944920:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00020000:00000080:0.0:1787827148.944923:0:91358:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.944927:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00020000:00000080:0.0:1787827148.944930:0:91358:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.944934:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00020000:00000080:0.0:1787827148.944936:0:91358:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.944949:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00020000:00000080:0.0:1787827148.944966:0:91358:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.944971:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00020000:00000080:0.0:1787827148.944974:0:91358:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.944978:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00020000:00000080:0.0:1787827148.944981:0:91358:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.944985:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00020000:00000080:0.0:1787827148.944988:0:91358:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.944992:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00020000:00000080:0.0:1787827148.944996:0:91358:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.945000:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00020000:00000080:0.0:1787827148.945003:0:91358:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.945007:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00020000:00000080:0.0:1787827148.945010:0:91358:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.945013:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00020000:00000080:0.0:1787827148.945016:0:91358:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.945020:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00020000:00000080:0.0:1787827148.945023:0:91358:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.945028:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00020000:00000080:0.0:1787827148.945032:0:91358:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.945036:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00020000:00000080:0.0:1787827148.945039:0:91358:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.945043:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00020000:00000080:0.0:1787827148.945046:0:91358:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.945049:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00020000:00000080:0.0:1787827148.945053:0:91358:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.945057:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00020000:00000080:0.0:1787827148.945060:0:91358:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.945064:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00020000:00000080:0.0:1787827148.945067:0:91358:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.945071:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00020000:00000080:0.0:1787827148.945075:0:91358:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.945079:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00020000:00000080:0.0:1787827148.945082:0:91358:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.945086:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00020000:00000080:0.0:1787827148.945089:0:91358:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.945093:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00020000:00000080:0.0:1787827148.945097:0:91358:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.945100:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00020000:00000080:0.0:1787827148.945104:0:91358:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.945108:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00020000:00000080:0.0:1787827148.945111:0:91358:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.945115:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00020000:00000080:0.0:1787827148.945118:0:91358:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.945123:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00020000:00000080:0.0:1787827148.945126:0:91358:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200080:0.0:1787827148.945129:0:91358:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe84ba8a10 00020000:00000080:0.0:1787827148.945132:0:91358:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608617400 uarg=0000000000000000 00000080:00200008:0.0:1787827148.945144:0:91358:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 100000 00000020:00000080:0.0:1787827149.578821:0:91376:0:(class_obd.c:336:class_handle_ioctl()) obdclass: cmd=40086685 len=0 uarg=00007ffcbf5098a0 00000002:00000080:0.0:1787827149.578841:0:91376:0:(mdc_request.c:2317:mdc_iocontrol()) lustre-MDT0000-mdc-ffff985602bc9800: cmd=40086685 len=576 karg=ffff985608617c00 uarg=0000000000000000 00000100:00080000:0.0:1787827149.578849:0:91376:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from FULL to DISCONN 00000100:02000400:0.0:1787827149.578858:0:91376:0:(import.c:182:ptlrpc_set_import_discon()) lustre-MDT0000-mdc-ffff985602bc9800: Connection to lustre-MDT0000 (at 192.168.203.114@tcp) was lost; in progress operations using this service will wait for recovery to complete 00000100:00080000:0.0:1787827149.578864:0:91376:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827149.578868:0:91376:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3294 00000100:00080000:0.0:1787827149.578882:0:91376:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787827149.578893:0:91376:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827149.578919:0:91376:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5a40 x1874669228207872/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827149.578957:0:91376:0:(recover.c:337:ptlrpc_recover_import()) lustre-MDT0000_UUID: recovery started, waiting 20 seconds 00000100:00100000:3.0:1787827149.579076:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228207872:192.168.203.114@tcp:38:lctl.0 00010000:00010000:3.0:1787827149.622053:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827150.644900:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827151.668845:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827152.692826:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827153.140909:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827153.140916:0:54101:0:(pinger.c:247:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1787827153.140956:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2385:1874669228208000:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787827153.140986:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228208128:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787827153.141042:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228208384:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787827153.141048:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669228208256:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787827153.142415:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228208384:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787827153.142505:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669228208256:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787827153.142546:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228208128:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787827153.142762:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2385:1874669228208000:192.168.203.114@tcp:400:kworker.0 00010000:00010000:3.0:1787827153.716933:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827154.164913:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827154.164930:0:54101:0:(pinger.c:247:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1787827154.164990:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2385:1874669228208512:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787827154.164998:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228208640:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787827154.165107:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228208896:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787827154.165110:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669228208768:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787827154.166603:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228208640:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787827154.166623:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2385:1874669228208512:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787827154.166629:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228208896:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787827154.166651:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669228208768:192.168.203.114@tcp:400:kworker.0 00010000:00010000:3.0:1787827154.740960:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827155.081224:0:2382:0:(import.c:1132:ptlrpc_connect_interpret()) lustre-MDT0000-mdc-ffff985602bc9800: connect to target with instance 52 00000100:00080000:3.0:1787827155.081251:0:2382:0:(import.c:1297:ptlrpc_connect_interpret()) reconnected to lustre-MDT0000_UUID@192.168.203.114@tcp after partition 00000100:00080000:3.0:1787827155.081255:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to RECOVER 00000100:00080000:3.0:1787827155.081268:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from RECOVER to FULL 00000100:02000000:0.0:1787827155.081496:0:2382:0:(import.c:1743:ptlrpc_import_recovery_state_machine()) lustre-MDT0000-mdc-ffff985602bc9800: Connection restored to 192.168.203.114@tcp (at 192.168.203.114@tcp) 00000100:00100000:0.0:1787827155.081514:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228207872:192.168.203.114@tcp:38:lctl.0 00000100:00080000:1.0:1787827155.081547:0:91376:0:(recover.c:348:ptlrpc_recover_import()) lustre-MDT0000_UUID: recovery finished FULL, rc = 0 00000080:00200008:1.0:1787827155.086211:0:91379:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:1.0:1787827155.086223:0:91379:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787827155.086241:0:91379:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827155.086258:0:91379:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 18 lru_type: 1 00010000:00010000:1.0:1787827155.086272:0:91379:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 18 lru_type: 1 00000080:00200000:1.0:1787827155.086280:0:91379:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827155.086286:0:91379:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787827155.086289:0:91379:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee56d 00000080:00010000:1.0:1787827155.086297:0:91379:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee56d from it ffffab3ac1dc7cb8 00010000:00010000:1.0:1787827155.086300:0:91379:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 18 lru_type: 1 00010000:00010000:1.0:1787827155.086307:0:91379:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 19 lru_type: 1 00000080:00200008:1.0:1787827155.086760:0:91379:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 0, flags 0, foreign 0 00000080:00200000:1.0:1787827155.086765:0:91379:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787827155.086773:0:91379:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827155.086782:0:91379:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 19 lru_type: 1 00010000:00010000:1.0:1787827155.086791:0:91379:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 19 lru_type: 1 00000080:00200000:1.0:1787827155.086797:0:91379:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827155.086802:0:91379:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787827155.086805:0:91379:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee56d 00000080:00010000:1.0:1787827155.086809:0:91379:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee56d from it ffffab3ac1dc7cb8 00010000:00010000:1.0:1787827155.086812:0:91379:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 19 lru_type: 1 00010000:00010000:1.0:1787827155.086819:0:91379:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 20 lru_type: 1 00000080:00200000:1.0:1787827155.086886:0:91379:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787827155.086892:0:91379:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827155.086897:0:91379:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 20 lru_type: 1 00010000:00010000:1.0:1787827155.086903:0:91379:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 20 lru_type: 1 00000080:00200000:1.0:1787827155.086920:0:91379:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827155.086924:0:91379:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787827155.086927:0:91379:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee56d 00000080:00010000:1.0:1787827155.086931:0:91379:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee56d from it ffffab3ac1dc7bb0 00010000:00010000:1.0:1787827155.086934:0:91379:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 20 lru_type: 1 00010000:00010000:1.0:1787827155.086939:0:91379:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 21 lru_type: 1 00000080:00200000:1.0:1787827155.086945:0:91379:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 44 00000080:00200008:1.0:1787827155.086950:0:91379:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 100000 00000080:00200080:1.0:1787827155.087011:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00800000:00000080:1.0:1787827155.087022:0:91379:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00800000:00000004:1.0:1787827155.087027:0:91379:0:(obd_class.h:1091:obd_statfs()) lustre-MDT0000-mdc-ffff985602bc9800: age 3299, max_age 3304 00000100:00100000:1.0:1787827155.087058:0:91379:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a30c0 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:91379:1874669228209024:192.168.203.114@tcp:41:lfs.0 00000100:00100000:1.0:1787827155.087123:0:91379:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bf00 going to sleep for 12 seconds 00000100:00100000:1.0:1787827155.088231:0:91379:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a30c0 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:91379:1874669228209024:192.168.203.114@tcp:41:lfs.0 00800000:00000004:1.0:1787827155.088246:0:91379:0:(obd_class.h:1117:obd_statfs()) lustre-MDT0000-mdc-ffff985602bc9800: update ffff985606a337f0 cache blocks 317333/353529 objects 1023688/1024000 00000080:00200080:1.0:1787827155.088276:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00800000:00000080:1.0:1787827155.088282:0:91379:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00800000:00000004:1.0:1787827155.088286:0:91379:0:(obd_class.h:1091:obd_statfs()) lustre-MDT0001-mdc-ffff985602bc9800: age 3299, max_age 3304 00000100:00100000:1.0:1787827155.088298:0:91379:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a30c0 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:91379:1874669228209152:192.168.203.114@tcp:41:lfs.0 00000100:00100000:1.0:1787827155.088328:0:91379:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bf00 going to sleep for 12 seconds 00000100:00100000:1.0:1787827155.089178:0:91379:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a30c0 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:91379:1874669228209152:192.168.203.114@tcp:41:lfs.0 00800000:00000004:1.0:1787827155.089190:0:91379:0:(obd_class.h:1117:obd_statfs()) lustre-MDT0001-mdc-ffff985602bc9800: update ffff98560952b7f0 cache blocks 317401/353529 objects 1023737/1024000 00000080:00200080:1.0:1787827155.089226:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00800000:00000080:1.0:1787827155.089233:0:91379:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.089238:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00800000:00000080:1.0:1787827155.089240:0:91379:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.089244:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00800000:00000080:1.0:1787827155.089247:0:91379:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.089250:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00800000:00000080:1.0:1787827155.089252:0:91379:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.089254:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00800000:00000080:1.0:1787827155.089261:0:91379:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.089265:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00800000:00000080:1.0:1787827155.089267:0:91379:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.089270:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00800000:00000080:1.0:1787827155.089272:0:91379:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.089275:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00800000:00000080:1.0:1787827155.089277:0:91379:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.089279:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00800000:00000080:1.0:1787827155.089281:0:91379:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.089283:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00800000:00000080:1.0:1787827155.089285:0:91379:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.089287:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00800000:00000080:1.0:1787827155.089288:0:91379:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.089290:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00800000:00000080:1.0:1787827155.089292:0:91379:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.089294:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00800000:00000080:1.0:1787827155.089296:0:91379:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.089298:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00800000:00000080:1.0:1787827155.089299:0:91379:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.089301:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00800000:00000080:1.0:1787827155.089304:0:91379:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.089306:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00800000:00000080:1.0:1787827155.089307:0:91379:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.089309:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00800000:00000080:1.0:1787827155.089311:0:91379:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.089313:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00800000:00000080:1.0:1787827155.089314:0:91379:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.089321:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00800000:00000080:1.0:1787827155.089323:0:91379:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.089325:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00800000:00000080:1.0:1787827155.089327:0:91379:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.089329:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00800000:00000080:1.0:1787827155.089330:0:91379:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.089332:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00800000:00000080:1.0:1787827155.089334:0:91379:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.089336:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00800000:00000080:1.0:1787827155.089338:0:91379:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.089340:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00800000:00000080:1.0:1787827155.089342:0:91379:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.089344:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00800000:00000080:1.0:1787827155.089346:0:91379:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.089348:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00800000:00000080:1.0:1787827155.089350:0:91379:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.089352:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00800000:00000080:1.0:1787827155.089353:0:91379:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.089356:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00800000:00000080:1.0:1787827155.089358:0:91379:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.089360:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00800000:00000080:1.0:1787827155.089364:0:91379:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.089368:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00800000:00000080:1.0:1787827155.089372:0:91379:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.089375:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00800000:00000080:1.0:1787827155.089378:0:91379:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.089389:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00800000:00000080:1.0:1787827155.089394:0:91379:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.089398:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00800000:00000080:1.0:1787827155.089401:0:91379:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.089406:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00800000:00000080:1.0:1787827155.089409:0:91379:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.089412:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00800000:00000080:1.0:1787827155.089415:0:91379:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.089419:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00800000:00000080:1.0:1787827155.089422:0:91379:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.089425:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00800000:00000080:1.0:1787827155.089427:0:91379:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.089430:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00800000:00000080:1.0:1787827155.089432:0:91379:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.089434:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00800000:00000080:1.0:1787827155.089435:0:91379:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.089437:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00800000:00000080:1.0:1787827155.089440:0:91379:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.089442:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00800000:00000080:1.0:1787827155.089444:0:91379:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.089446:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00800000:00000080:1.0:1787827155.089448:0:91379:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.089450:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00800000:00000080:1.0:1787827155.089451:0:91379:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.089453:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00800000:00000080:1.0:1787827155.089455:0:91379:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.089457:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00800000:00000080:1.0:1787827155.089458:0:91379:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.089460:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00800000:00000080:1.0:1787827155.089462:0:91379:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.089464:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00800000:00000080:1.0:1787827155.089465:0:91379:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.089467:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00800000:00000080:1.0:1787827155.089469:0:91379:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.089471:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00800000:00000080:1.0:1787827155.089472:0:91379:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.089474:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00800000:00000080:1.0:1787827155.089476:0:91379:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.089478:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00800000:00000080:1.0:1787827155.089479:0:91379:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.089481:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00800000:00000080:1.0:1787827155.089483:0:91379:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.089485:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00800000:00000080:1.0:1787827155.089487:0:91379:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.089490:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00800000:00000080:1.0:1787827155.089491:0:91379:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.089493:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00800000:00000080:1.0:1787827155.089495:0:91379:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.089497:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00800000:00000080:1.0:1787827155.089498:0:91379:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.089500:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00800000:00000080:1.0:1787827155.089502:0:91379:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.089504:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00800000:00000080:1.0:1787827155.089506:0:91379:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.089508:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00800000:00000080:1.0:1787827155.089509:0:91379:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.089511:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00800000:00000080:1.0:1787827155.089513:0:91379:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.089515:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00800000:00000080:1.0:1787827155.089516:0:91379:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.089518:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00800000:00000080:1.0:1787827155.089520:0:91379:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.089522:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00800000:00000080:1.0:1787827155.089524:0:91379:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.089526:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00020000:00000080:1.0:1787827155.089529:0:91379:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00020000:00000004:1.0:1787827155.089534:0:91379:0:(obd_class.h:1091:obd_statfs()) lustre-OST0000-osc-ffff985602bc9800: age 3299, max_age 3304 00000100:00100000:1.0:1787827155.089548:0:91379:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a30c0 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:91379:1874669228209280:192.168.203.114@tcp:13:lfs.0 00000100:00100000:1.0:1787827155.089578:0:91379:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bf00 going to sleep for 11 seconds 00000100:00100000:1.0:1787827155.090395:0:91379:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a30c0 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:91379:1874669228209280:192.168.203.114@tcp:13:lfs.0 00020000:00000004:1.0:1787827155.090404:0:91379:0:(obd_class.h:1117:obd_statfs()) lustre-OST0000-osc-ffff985602bc9800: update ffff985609528450 cache blocks 901356/958279 objects 261727/262144 00000080:00200080:1.0:1787827155.090438:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00020000:00000080:1.0:1787827155.090447:0:91379:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00020000:00000004:1.0:1787827155.090451:0:91379:0:(obd_class.h:1091:obd_statfs()) lustre-OST0001-osc-ffff985602bc9800: age 3299, max_age 3304 00000100:00100000:1.0:1787827155.090468:0:91379:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a30c0 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:91379:1874669228209408:192.168.203.114@tcp:13:lfs.0 00000100:00100000:1.0:1787827155.090506:0:91379:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bf00 going to sleep for 11 seconds 00000100:00100000:1.0:1787827155.091267:0:91379:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a30c0 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:91379:1874669228209408:192.168.203.114@tcp:13:lfs.0 00020000:00000004:1.0:1787827155.091275:0:91379:0:(obd_class.h:1117:obd_statfs()) lustre-OST0001-osc-ffff985602bc9800: update ffff98560952d1c0 cache blocks 901363/958279 objects 261735/262144 00000080:00200080:1.0:1787827155.091297:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00020000:00000080:1.0:1787827155.091303:0:91379:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.091308:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00020000:00000080:1.0:1787827155.091310:0:91379:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.091313:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00020000:00000080:1.0:1787827155.091316:0:91379:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.091318:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00020000:00000080:1.0:1787827155.091320:0:91379:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.091322:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00020000:00000080:1.0:1787827155.091324:0:91379:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.091327:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00020000:00000080:1.0:1787827155.091329:0:91379:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.091332:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00020000:00000080:1.0:1787827155.091335:0:91379:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.091337:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00020000:00000080:1.0:1787827155.091340:0:91379:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.091342:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00020000:00000080:1.0:1787827155.091344:0:91379:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.091346:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00020000:00000080:1.0:1787827155.091348:0:91379:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.091350:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00020000:00000080:1.0:1787827155.091351:0:91379:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.091353:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00020000:00000080:1.0:1787827155.091355:0:91379:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.091357:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00020000:00000080:1.0:1787827155.091358:0:91379:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.091360:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00020000:00000080:1.0:1787827155.091362:0:91379:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.091364:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00020000:00000080:1.0:1787827155.091366:0:91379:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.091368:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00020000:00000080:1.0:1787827155.091369:0:91379:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.091371:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00020000:00000080:1.0:1787827155.091373:0:91379:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.091375:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00020000:00000080:1.0:1787827155.091376:0:91379:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.091378:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00020000:00000080:1.0:1787827155.091380:0:91379:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.091383:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00020000:00000080:1.0:1787827155.091385:0:91379:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.091387:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00020000:00000080:1.0:1787827155.091388:0:91379:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.091390:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00020000:00000080:1.0:1787827155.091392:0:91379:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.091394:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00020000:00000080:1.0:1787827155.091395:0:91379:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.091397:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00020000:00000080:1.0:1787827155.091399:0:91379:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.091401:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00020000:00000080:1.0:1787827155.091402:0:91379:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.091404:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00020000:00000080:1.0:1787827155.091406:0:91379:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.091408:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00020000:00000080:1.0:1787827155.091409:0:91379:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.091411:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00020000:00000080:1.0:1787827155.091413:0:91379:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.091415:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00020000:00000080:1.0:1787827155.091417:0:91379:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.091419:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00020000:00000080:1.0:1787827155.091420:0:91379:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.091422:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00020000:00000080:1.0:1787827155.091424:0:91379:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.091426:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00020000:00000080:1.0:1787827155.091428:0:91379:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.091430:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00020000:00000080:1.0:1787827155.091431:0:91379:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.091433:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00020000:00000080:1.0:1787827155.091435:0:91379:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.091437:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00020000:00000080:1.0:1787827155.091440:0:91379:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.091444:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00020000:00000080:1.0:1787827155.091447:0:91379:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.091450:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00020000:00000080:1.0:1787827155.091453:0:91379:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.091457:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00020000:00000080:1.0:1787827155.091460:0:91379:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.091464:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00020000:00000080:1.0:1787827155.091467:0:91379:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.091471:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00020000:00000080:1.0:1787827155.091474:0:91379:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.091478:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00020000:00000080:1.0:1787827155.091482:0:91379:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.091486:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00020000:00000080:1.0:1787827155.091489:0:91379:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.091493:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00020000:00000080:1.0:1787827155.091495:0:91379:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.091497:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00020000:00000080:1.0:1787827155.091499:0:91379:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.091502:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00020000:00000080:1.0:1787827155.091504:0:91379:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.091506:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00020000:00000080:1.0:1787827155.091508:0:91379:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.091515:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00020000:00000080:1.0:1787827155.091517:0:91379:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.091519:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00020000:00000080:1.0:1787827155.091521:0:91379:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.091523:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00020000:00000080:1.0:1787827155.091525:0:91379:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.091527:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00020000:00000080:1.0:1787827155.091529:0:91379:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.091531:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00020000:00000080:1.0:1787827155.091533:0:91379:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.091535:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00020000:00000080:1.0:1787827155.091536:0:91379:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.091538:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00020000:00000080:1.0:1787827155.091540:0:91379:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.091542:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00020000:00000080:1.0:1787827155.091544:0:91379:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.091546:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00020000:00000080:1.0:1787827155.091547:0:91379:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.091549:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00020000:00000080:1.0:1787827155.091551:0:91379:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.091553:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00020000:00000080:1.0:1787827155.091555:0:91379:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.091557:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00020000:00000080:1.0:1787827155.091560:0:91379:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.091562:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00020000:00000080:1.0:1787827155.091563:0:91379:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.091565:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00020000:00000080:1.0:1787827155.091567:0:91379:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.091569:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00020000:00000080:1.0:1787827155.091571:0:91379:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.091573:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00020000:00000080:1.0:1787827155.091574:0:91379:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827155.091576:0:91379:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff2f39bef0 00020000:00000080:1.0:1787827155.091578:0:91379:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200008:1.0:1787827155.091589:0:91379:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 100000 00000020:00000080:1.0:1787827155.727439:0:91397:0:(class_obd.c:336:class_handle_ioctl()) obdclass: cmd=40086685 len=0 uarg=00007ffc10ff4db0 00000002:00000080:1.0:1787827155.727458:0:91397:0:(mdc_request.c:2317:mdc_iocontrol()) lustre-MDT0000-mdc-ffff985602bc9800: cmd=40086685 len=576 karg=ffff98560865f400 uarg=0000000000000000 00000100:00080000:1.0:1787827155.727466:0:91397:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from FULL to DISCONN 00000100:02000400:1.0:1787827155.727475:0:91397:0:(import.c:182:ptlrpc_set_import_discon()) lustre-MDT0000-mdc-ffff985602bc9800: Connection to lustre-MDT0000 (at 192.168.203.114@tcp) was lost; in progress operations using this service will wait for recovery to complete 00000100:00080000:1.0:1787827155.727481:0:91397:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827155.727486:0:91397:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3300 00000100:00080000:1.0:1787827155.727500:0:91397:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787827155.727511:0:91397:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827155.727540:0:91397:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3480 x1874669228209536/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827155.727577:0:91397:0:(recover.c:337:ptlrpc_recover_import()) lustre-MDT0000_UUID: recovery started, waiting 20 seconds 00000100:00100000:0.0:1787827155.727613:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228209536:192.168.203.114@tcp:38:lctl.0 00010000:00010000:1.0:1787827155.764951:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827156.788920:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827157.812898:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827158.836873:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827159.284964:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827159.284972:0:54101:0:(pinger.c:247:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1787827159.284997:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2385:1874669228209664:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787827159.285024:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228209792:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787827159.285091:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228210048:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787827159.285108:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669228209920:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787827159.286877:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228209792:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787827159.286901:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228210048:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787827159.286923:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669228209920:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787827159.286929:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2385:1874669228209664:192.168.203.114@tcp:400:kworker.0 00010000:00010000:3.0:1787827159.861505:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827160.885027:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827161.216082:0:2382:0:(import.c:1132:ptlrpc_connect_interpret()) lustre-MDT0000-mdc-ffff985602bc9800: connect to target with instance 52 00000100:00080000:0.0:1787827161.216110:0:2382:0:(import.c:1297:ptlrpc_connect_interpret()) reconnected to lustre-MDT0000_UUID@192.168.203.114@tcp after partition 00000100:00080000:0.0:1787827161.216114:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to RECOVER 00000100:00080000:0.0:1787827161.216120:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from RECOVER to FULL 00000100:02000000:0.0:1787827161.216411:0:2382:0:(import.c:1743:ptlrpc_import_recovery_state_machine()) lustre-MDT0000-mdc-ffff985602bc9800: Connection restored to 192.168.203.114@tcp (at 192.168.203.114@tcp) 00000100:00080000:1.0:1787827161.216455:0:91397:0:(recover.c:348:ptlrpc_recover_import()) lustre-MDT0000_UUID: recovery finished FULL, rc = 0 00000100:00100000:0.0:1787827161.216457:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228209536:192.168.203.114@tcp:38:lctl.0 00000080:00200008:1.0:1787827161.222768:0:91400:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:1.0:1787827161.222781:0:91400:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787827161.222801:0:91400:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827161.222818:0:91400:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 21 lru_type: 1 00010000:00010000:1.0:1787827161.222833:0:91400:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 21 lru_type: 1 00000080:00200000:1.0:1787827161.222841:0:91400:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827161.222848:0:91400:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787827161.222852:0:91400:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee56d 00000080:00010000:1.0:1787827161.222859:0:91400:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee56d from it ffffab3ac1d27cb8 00010000:00010000:1.0:1787827161.222864:0:91400:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 21 lru_type: 1 00010000:00010000:1.0:1787827161.222886:0:91400:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 22 lru_type: 1 00000080:00200008:1.0:1787827161.223387:0:91400:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 0, flags 0, foreign 0 00000080:00200000:1.0:1787827161.223393:0:91400:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787827161.223401:0:91400:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827161.223409:0:91400:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 22 lru_type: 1 00010000:00010000:1.0:1787827161.223420:0:91400:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 22 lru_type: 1 00000080:00200000:1.0:1787827161.223426:0:91400:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827161.223431:0:91400:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787827161.223434:0:91400:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee56d 00000080:00010000:1.0:1787827161.223438:0:91400:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee56d from it ffffab3ac1d27cb8 00010000:00010000:1.0:1787827161.223441:0:91400:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 22 lru_type: 1 00010000:00010000:1.0:1787827161.223449:0:91400:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 23 lru_type: 1 00000080:00200000:1.0:1787827161.223531:0:91400:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787827161.223539:0:91400:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827161.223546:0:91400:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 23 lru_type: 1 00010000:00010000:1.0:1787827161.223556:0:91400:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 23 lru_type: 1 00000080:00200000:1.0:1787827161.223563:0:91400:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827161.223567:0:91400:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787827161.223569:0:91400:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee56d 00000080:00010000:1.0:1787827161.223573:0:91400:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee56d from it ffffab3ac1d27bb0 00010000:00010000:1.0:1787827161.223576:0:91400:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 23 lru_type: 1 00010000:00010000:1.0:1787827161.223584:0:91400:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 24 lru_type: 1 00000080:00200000:1.0:1787827161.223593:0:91400:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 44 00000080:00200008:1.0:1787827161.223599:0:91400:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 100000 00000080:00200080:1.0:1787827161.223682:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00800000:00000080:1.0:1787827161.223718:0:91400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00800000:00000004:1.0:1787827161.223725:0:91400:0:(obd_class.h:1091:obd_statfs()) lustre-MDT0000-mdc-ffff985602bc9800: age 3305, max_age 3310 00000100:00100000:1.0:1787827161.223764:0:91400:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0f00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:91400:1874669228210176:192.168.203.114@tcp:41:lfs.0 00000100:00100000:1.0:1787827161.223843:0:91400:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5b840 going to sleep for 12 seconds 00000100:00100000:1.0:1787827161.225511:0:91400:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0f00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:91400:1874669228210176:192.168.203.114@tcp:41:lfs.0 00800000:00000004:1.0:1787827161.225532:0:91400:0:(obd_class.h:1117:obd_statfs()) lustre-MDT0000-mdc-ffff985602bc9800: update ffff985606a337f0 cache blocks 317333/353529 objects 1023688/1024000 00000080:00200080:1.0:1787827161.225589:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00800000:00000080:1.0:1787827161.225600:0:91400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00800000:00000004:1.0:1787827161.225606:0:91400:0:(obd_class.h:1091:obd_statfs()) lustre-MDT0001-mdc-ffff985602bc9800: age 3305, max_age 3310 00000100:00100000:1.0:1787827161.225627:0:91400:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0f00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:91400:1874669228210304:192.168.203.114@tcp:41:lfs.0 00000100:00100000:1.0:1787827161.225678:0:91400:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5b840 going to sleep for 12 seconds 00000100:00100000:1.0:1787827161.227829:0:91400:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0f00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:91400:1874669228210304:192.168.203.114@tcp:41:lfs.0 00800000:00000004:1.0:1787827161.227850:0:91400:0:(obd_class.h:1117:obd_statfs()) lustre-MDT0001-mdc-ffff985602bc9800: update ffff98560952b7f0 cache blocks 317401/353529 objects 1023737/1024000 00000080:00200080:1.0:1787827161.227911:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00800000:00000080:1.0:1787827161.227924:0:91400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.227932:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00800000:00000080:1.0:1787827161.227936:0:91400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.227942:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00800000:00000080:1.0:1787827161.227947:0:91400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.227951:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00800000:00000080:1.0:1787827161.227955:0:91400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.227960:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00800000:00000080:1.0:1787827161.227965:0:91400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.227969:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00800000:00000080:1.0:1787827161.227973:0:91400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.227976:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00800000:00000080:1.0:1787827161.227980:0:91400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.227983:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00800000:00000080:1.0:1787827161.227987:0:91400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.227990:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00800000:00000080:1.0:1787827161.227999:0:91400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.228004:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00800000:00000080:1.0:1787827161.228007:0:91400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.228011:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00800000:00000080:1.0:1787827161.228015:0:91400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.228019:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00800000:00000080:1.0:1787827161.228023:0:91400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.228027:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00800000:00000080:1.0:1787827161.228030:0:91400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.228034:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00800000:00000080:1.0:1787827161.228038:0:91400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.228042:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00800000:00000080:1.0:1787827161.228046:0:91400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.228050:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00800000:00000080:1.0:1787827161.228054:0:91400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.228057:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00800000:00000080:1.0:1787827161.228061:0:91400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.228064:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00800000:00000080:1.0:1787827161.228068:0:91400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.228072:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00800000:00000080:1.0:1787827161.228075:0:91400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.228079:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00800000:00000080:1.0:1787827161.228083:0:91400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.228086:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00800000:00000080:1.0:1787827161.228090:0:91400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.228093:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00800000:00000080:1.0:1787827161.228096:0:91400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.228100:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00800000:00000080:1.0:1787827161.228104:0:91400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.228108:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00800000:00000080:1.0:1787827161.228112:0:91400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.228115:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00800000:00000080:1.0:1787827161.228119:0:91400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.228122:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00800000:00000080:1.0:1787827161.228126:0:91400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.228130:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00800000:00000080:1.0:1787827161.228134:0:91400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.228138:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00800000:00000080:1.0:1787827161.228142:0:91400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.228145:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00800000:00000080:1.0:1787827161.228149:0:91400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.228154:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00800000:00000080:1.0:1787827161.228158:0:91400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.228162:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00800000:00000080:1.0:1787827161.228166:0:91400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.228170:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00800000:00000080:1.0:1787827161.228174:0:91400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.228178:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00800000:00000080:1.0:1787827161.228182:0:91400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.228186:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00800000:00000080:1.0:1787827161.228189:0:91400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.228194:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00800000:00000080:1.0:1787827161.228197:0:91400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.228201:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00800000:00000080:1.0:1787827161.228205:0:91400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.228209:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00800000:00000080:1.0:1787827161.228213:0:91400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.228217:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00800000:00000080:1.0:1787827161.228220:0:91400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.228224:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00800000:00000080:1.0:1787827161.228228:0:91400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.228231:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00800000:00000080:1.0:1787827161.228236:0:91400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.228239:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00800000:00000080:1.0:1787827161.228243:0:91400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.228247:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00800000:00000080:1.0:1787827161.228250:0:91400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.228255:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00800000:00000080:1.0:1787827161.228258:0:91400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.228262:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00800000:00000080:1.0:1787827161.228266:0:91400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.228269:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00800000:00000080:1.0:1787827161.228273:0:91400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.228276:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00800000:00000080:1.0:1787827161.228280:0:91400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.228284:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00800000:00000080:1.0:1787827161.228288:0:91400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.228292:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00800000:00000080:1.0:1787827161.228295:0:91400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.228299:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00800000:00000080:1.0:1787827161.228302:0:91400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.228306:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00800000:00000080:1.0:1787827161.228309:0:91400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.228313:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00800000:00000080:1.0:1787827161.228317:0:91400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.228321:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00800000:00000080:1.0:1787827161.228338:0:91400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.228343:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00800000:00000080:1.0:1787827161.228347:0:91400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.228351:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00800000:00000080:1.0:1787827161.228354:0:91400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.228358:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00800000:00000080:1.0:1787827161.228362:0:91400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.228365:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00800000:00000080:1.0:1787827161.228368:0:91400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.228372:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00800000:00000080:1.0:1787827161.228376:0:91400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.228380:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00800000:00000080:1.0:1787827161.228383:0:91400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.228391:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00800000:00000080:1.0:1787827161.228394:0:91400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.228398:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00800000:00000080:1.0:1787827161.228402:0:91400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.228405:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00800000:00000080:1.0:1787827161.228408:0:91400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.228412:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00800000:00000080:1.0:1787827161.228415:0:91400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.228419:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00800000:00000080:1.0:1787827161.228423:0:91400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.228427:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00020000:00000080:1.0:1787827161.228434:0:91400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00020000:00000004:1.0:1787827161.228442:0:91400:0:(obd_class.h:1091:obd_statfs()) lustre-OST0000-osc-ffff985602bc9800: age 3305, max_age 3310 00000100:00100000:1.0:1787827161.228468:0:91400:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0f00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:91400:1874669228210432:192.168.203.114@tcp:13:lfs.0 00000100:00100000:1.0:1787827161.228533:0:91400:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5b840 going to sleep for 11 seconds 00000100:00100000:1.0:1787827161.230212:0:91400:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0f00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:91400:1874669228210432:192.168.203.114@tcp:13:lfs.0 00020000:00000004:1.0:1787827161.230230:0:91400:0:(obd_class.h:1117:obd_statfs()) lustre-OST0000-osc-ffff985602bc9800: update ffff985609528450 cache blocks 901356/958279 objects 261727/262144 00000080:00200080:1.0:1787827161.230283:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00020000:00000080:1.0:1787827161.230294:0:91400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00020000:00000004:1.0:1787827161.230300:0:91400:0:(obd_class.h:1091:obd_statfs()) lustre-OST0001-osc-ffff985602bc9800: age 3305, max_age 3310 00000100:00100000:1.0:1787827161.230336:0:91400:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0f00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:91400:1874669228210560:192.168.203.114@tcp:13:lfs.0 00000100:00100000:1.0:1787827161.230388:0:91400:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5b840 going to sleep for 11 seconds 00000100:00100000:1.0:1787827161.231514:0:91400:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0f00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:91400:1874669228210560:192.168.203.114@tcp:13:lfs.0 00020000:00000004:1.0:1787827161.231527:0:91400:0:(obd_class.h:1117:obd_statfs()) lustre-OST0001-osc-ffff985602bc9800: update ffff98560952d1c0 cache blocks 901363/958279 objects 261735/262144 00000080:00200080:1.0:1787827161.231561:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00020000:00000080:1.0:1787827161.231569:0:91400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.231576:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00020000:00000080:1.0:1787827161.231580:0:91400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.231584:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00020000:00000080:1.0:1787827161.231588:0:91400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.231593:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00020000:00000080:1.0:1787827161.231596:0:91400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.231601:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00020000:00000080:1.0:1787827161.231605:0:91400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.231609:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00020000:00000080:1.0:1787827161.231613:0:91400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.231617:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00020000:00000080:1.0:1787827161.231622:0:91400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.231626:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00020000:00000080:1.0:1787827161.231631:0:91400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.231635:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00020000:00000080:1.0:1787827161.231638:0:91400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.231642:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00020000:00000080:1.0:1787827161.231646:0:91400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.231650:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00020000:00000080:1.0:1787827161.231653:0:91400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.231657:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00020000:00000080:1.0:1787827161.231660:0:91400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.231664:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00020000:00000080:1.0:1787827161.231668:0:91400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.231672:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00020000:00000080:1.0:1787827161.231675:0:91400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.231679:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00020000:00000080:1.0:1787827161.231683:0:91400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.231687:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00020000:00000080:1.0:1787827161.231734:0:91400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.231745:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00020000:00000080:1.0:1787827161.231750:0:91400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.231754:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00020000:00000080:1.0:1787827161.231758:0:91400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.231762:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00020000:00000080:1.0:1787827161.231766:0:91400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.231770:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00020000:00000080:1.0:1787827161.231773:0:91400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.231777:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00020000:00000080:1.0:1787827161.231781:0:91400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.231785:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00020000:00000080:1.0:1787827161.231789:0:91400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.231793:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00020000:00000080:1.0:1787827161.231796:0:91400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.231800:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00020000:00000080:1.0:1787827161.231804:0:91400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.231808:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00020000:00000080:1.0:1787827161.231812:0:91400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.231816:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00020000:00000080:1.0:1787827161.231820:0:91400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.231824:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00020000:00000080:1.0:1787827161.231827:0:91400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.231831:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00020000:00000080:1.0:1787827161.231835:0:91400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.231839:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00020000:00000080:1.0:1787827161.231842:0:91400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.231846:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00020000:00000080:1.0:1787827161.231849:0:91400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.231853:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00020000:00000080:1.0:1787827161.231856:0:91400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.231860:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00020000:00000080:1.0:1787827161.231863:0:91400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.231880:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00020000:00000080:1.0:1787827161.231886:0:91400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.231890:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00020000:00000080:1.0:1787827161.231894:0:91400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.231898:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00020000:00000080:1.0:1787827161.231901:0:91400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.231905:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00020000:00000080:1.0:1787827161.231909:0:91400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.231913:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00020000:00000080:1.0:1787827161.231916:0:91400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.231920:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00020000:00000080:1.0:1787827161.231924:0:91400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.231927:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00020000:00000080:1.0:1787827161.231930:0:91400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.231934:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00020000:00000080:1.0:1787827161.231937:0:91400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.231940:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00020000:00000080:1.0:1787827161.231944:0:91400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.231948:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00020000:00000080:1.0:1787827161.231951:0:91400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.231954:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00020000:00000080:1.0:1787827161.231957:0:91400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.231961:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00020000:00000080:1.0:1787827161.231963:0:91400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.231968:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00020000:00000080:1.0:1787827161.231971:0:91400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.231975:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00020000:00000080:1.0:1787827161.231978:0:91400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.231982:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00020000:00000080:1.0:1787827161.231985:0:91400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.231988:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00020000:00000080:1.0:1787827161.231991:0:91400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.231995:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00020000:00000080:1.0:1787827161.231997:0:91400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.232001:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00020000:00000080:1.0:1787827161.232003:0:91400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.232007:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00020000:00000080:1.0:1787827161.232010:0:91400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.232013:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00020000:00000080:1.0:1787827161.232016:0:91400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.232030:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00020000:00000080:1.0:1787827161.232035:0:91400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.232040:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00020000:00000080:1.0:1787827161.232043:0:91400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.232047:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00020000:00000080:1.0:1787827161.232050:0:91400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.232054:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00020000:00000080:1.0:1787827161.232058:0:91400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.232062:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00020000:00000080:1.0:1787827161.232066:0:91400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.232070:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00020000:00000080:1.0:1787827161.232074:0:91400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.232077:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00020000:00000080:1.0:1787827161.232081:0:91400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.232085:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00020000:00000080:1.0:1787827161.232088:0:91400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.232092:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00020000:00000080:1.0:1787827161.232095:0:91400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.232099:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00020000:00000080:1.0:1787827161.232102:0:91400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200080:1.0:1787827161.232106:0:91400:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffcd634a720 00020000:00000080:1.0:1787827161.232109:0:91400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f400 uarg=0000000000000000 00000080:00200008:1.0:1787827161.232121:0:91400:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 100000 00010000:00010000:2.0:1787827161.908845:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000080:0.0:1787827161.929211:0:91418:0:(class_obd.c:336:class_handle_ioctl()) obdclass: cmd=40086685 len=0 uarg=00007fffe6d9d610 00000002:00000080:0.0:1787827161.929228:0:91418:0:(mdc_request.c:2317:mdc_iocontrol()) lustre-MDT0000-mdc-ffff985602bc9800: cmd=40086685 len=576 karg=ffff985608615000 uarg=0000000000000000 00000100:00080000:0.0:1787827161.929235:0:91418:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from FULL to DISCONN 00000100:02000400:0.0:1787827161.929244:0:91418:0:(import.c:182:ptlrpc_set_import_discon()) lustre-MDT0000-mdc-ffff985602bc9800: Connection to lustre-MDT0000 (at 192.168.203.114@tcp) was lost; in progress operations using this service will wait for recovery to complete 00000100:00080000:0.0:1787827161.929250:0:91418:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827161.929254:0:91418:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3306 00000100:00080000:0.0:1787827161.929268:0:91418:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787827161.929279:0:91418:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827161.929303:0:91418:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5e00 x1874669228210688/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827161.929331:0:91418:0:(recover.c:337:ptlrpc_recover_import()) lustre-MDT0000_UUID: recovery started, waiting 20 seconds 00000100:00100000:3.0:1787827161.929369:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228210688:192.168.203.114@tcp:38:lctl.0 00010000:00010000:0.0:1787827162.932900:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827163.956883:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827164.404884:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827164.404890:0:38:0:(pinger.c:247:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:1.0:1787827164.404938:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2383:1874669228210816:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787827164.404946:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669228210944:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787827164.405032:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669228211200:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827164.405040:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669228211072:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787827164.407119:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669228210944:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827164.407129:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2383:1874669228210816:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787827164.407140:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669228211200:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827164.407152:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669228211072:192.168.203.114@tcp:400:kworker.0 00010000:00010000:3.0:1787827164.980898:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827165.428948:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827165.428953:0:38:0:(pinger.c:247:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1787827165.428989:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2385:1874669228211328:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787827165.429004:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228211456:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787827165.429061:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669228211584:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787827165.429065:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228211712:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787827165.430626:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228211456:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787827165.430636:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2385:1874669228211328:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787827165.430644:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228211712:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787827165.430657:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669228211584:192.168.203.114@tcp:400:kworker.0 00010000:00010000:1.0:1787827166.004841:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827167.028881:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827167.369254:0:2382:0:(import.c:1132:ptlrpc_connect_interpret()) lustre-MDT0000-mdc-ffff985602bc9800: connect to target with instance 52 00000100:00080000:3.0:1787827167.369286:0:2382:0:(import.c:1297:ptlrpc_connect_interpret()) reconnected to lustre-MDT0000_UUID@192.168.203.114@tcp after partition 00000100:00080000:3.0:1787827167.369291:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to RECOVER 00000100:00080000:3.0:1787827167.369297:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from RECOVER to FULL 00000100:02000000:0.0:1787827167.369511:0:2382:0:(import.c:1743:ptlrpc_import_recovery_state_machine()) lustre-MDT0000-mdc-ffff985602bc9800: Connection restored to 192.168.203.114@tcp (at 192.168.203.114@tcp) 00000100:00100000:0.0:1787827167.369527:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228210688:192.168.203.114@tcp:38:lctl.0 00000100:00080000:1.0:1787827167.369566:0:91418:0:(recover.c:348:ptlrpc_recover_import()) lustre-MDT0000_UUID: recovery finished FULL, rc = 0 00000080:00200008:1.0:1787827167.374530:0:91421:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:1.0:1787827167.374541:0:91421:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787827167.374558:0:91421:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827167.374576:0:91421:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 24 lru_type: 1 00010000:00010000:1.0:1787827167.374591:0:91421:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 24 lru_type: 1 00000080:00200000:1.0:1787827167.374600:0:91421:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827167.374608:0:91421:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787827167.374611:0:91421:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee56d 00000080:00010000:1.0:1787827167.374618:0:91421:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee56d from it ffffab3ac1eafcb8 00010000:00010000:1.0:1787827167.374623:0:91421:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 24 lru_type: 1 00010000:00010000:1.0:1787827167.374632:0:91421:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 25 lru_type: 1 00000080:00200008:1.0:1787827167.375122:0:91421:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 0, flags 0, foreign 0 00000080:00200000:1.0:1787827167.375125:0:91421:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787827167.375134:0:91421:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827167.375142:0:91421:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 25 lru_type: 1 00010000:00010000:1.0:1787827167.375150:0:91421:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 25 lru_type: 1 00000080:00200000:1.0:1787827167.375157:0:91421:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827167.375162:0:91421:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787827167.375164:0:91421:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee56d 00000080:00010000:1.0:1787827167.375169:0:91421:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee56d from it ffffab3ac1eafcb8 00010000:00010000:1.0:1787827167.375173:0:91421:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 25 lru_type: 1 00010000:00010000:1.0:1787827167.375180:0:91421:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 26 lru_type: 1 00000080:00200000:1.0:1787827167.375260:0:91421:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787827167.375267:0:91421:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827167.375272:0:91421:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 26 lru_type: 1 00010000:00010000:1.0:1787827167.375278:0:91421:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 26 lru_type: 1 00000080:00200000:1.0:1787827167.375282:0:91421:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827167.375286:0:91421:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787827167.375288:0:91421:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee56d 00000080:00010000:1.0:1787827167.375292:0:91421:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee56d from it ffffab3ac1eafbb0 00010000:00010000:1.0:1787827167.375294:0:91421:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 26 lru_type: 1 00010000:00010000:1.0:1787827167.375301:0:91421:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 27 lru_type: 1 00000080:00200000:1.0:1787827167.375309:0:91421:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 44 00000080:00200008:1.0:1787827167.375315:0:91421:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 100000 00000080:00200080:1.0:1787827167.375398:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00800000:00000080:1.0:1787827167.375412:0:91421:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00800000:00000004:1.0:1787827167.375420:0:91421:0:(obd_class.h:1091:obd_statfs()) lustre-MDT0000-mdc-ffff985602bc9800: age 3311, max_age 3317 00000100:00100000:1.0:1787827167.375461:0:91421:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a21c0 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:91421:1874669228211840:192.168.203.114@tcp:41:lfs.0 00000100:00100000:1.0:1787827167.375537:0:91421:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bd80 going to sleep for 12 seconds 00000100:00100000:1.0:1787827167.376660:0:91421:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a21c0 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:91421:1874669228211840:192.168.203.114@tcp:41:lfs.0 00800000:00000004:1.0:1787827167.376676:0:91421:0:(obd_class.h:1117:obd_statfs()) lustre-MDT0000-mdc-ffff985602bc9800: update ffff985606a337f0 cache blocks 317333/353529 objects 1023688/1024000 00000080:00200080:1.0:1787827167.376733:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00800000:00000080:1.0:1787827167.376742:0:91421:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00800000:00000004:1.0:1787827167.376748:0:91421:0:(obd_class.h:1091:obd_statfs()) lustre-MDT0001-mdc-ffff985602bc9800: age 3311, max_age 3317 00000100:00100000:1.0:1787827167.376766:0:91421:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a21c0 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:91421:1874669228211968:192.168.203.114@tcp:41:lfs.0 00000100:00100000:1.0:1787827167.376801:0:91421:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bd80 going to sleep for 12 seconds 00000100:00100000:1.0:1787827167.377582:0:91421:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a21c0 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:91421:1874669228211968:192.168.203.114@tcp:41:lfs.0 00800000:00000004:1.0:1787827167.377592:0:91421:0:(obd_class.h:1117:obd_statfs()) lustre-MDT0001-mdc-ffff985602bc9800: update ffff98560952b7f0 cache blocks 317401/353529 objects 1023737/1024000 00000080:00200080:1.0:1787827167.377619:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00800000:00000080:1.0:1787827167.377626:0:91421:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.377630:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00800000:00000080:1.0:1787827167.377632:0:91421:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.377635:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00800000:00000080:1.0:1787827167.377637:0:91421:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.377639:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00800000:00000080:1.0:1787827167.377642:0:91421:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.377645:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00800000:00000080:1.0:1787827167.377646:0:91421:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.377650:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00800000:00000080:1.0:1787827167.377654:0:91421:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.377660:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00800000:00000080:1.0:1787827167.377664:0:91421:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.377668:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00800000:00000080:1.0:1787827167.377671:0:91421:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.377675:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00800000:00000080:1.0:1787827167.377678:0:91421:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.377682:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00800000:00000080:1.0:1787827167.377686:0:91421:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.377713:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00800000:00000080:1.0:1787827167.377718:0:91421:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.377723:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00800000:00000080:1.0:1787827167.377726:0:91421:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.377729:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00800000:00000080:1.0:1787827167.377731:0:91421:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.377734:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00800000:00000080:1.0:1787827167.377736:0:91421:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.377739:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00800000:00000080:1.0:1787827167.377741:0:91421:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.377743:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00800000:00000080:1.0:1787827167.377745:0:91421:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.377748:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00800000:00000080:1.0:1787827167.377750:0:91421:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.377753:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00800000:00000080:1.0:1787827167.377755:0:91421:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.377757:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00800000:00000080:1.0:1787827167.377760:0:91421:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.377762:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00800000:00000080:1.0:1787827167.377765:0:91421:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.377768:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00800000:00000080:1.0:1787827167.377772:0:91421:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.377775:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00800000:00000080:1.0:1787827167.377779:0:91421:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.377782:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00800000:00000080:1.0:1787827167.377786:0:91421:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.377789:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00800000:00000080:1.0:1787827167.377793:0:91421:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.377797:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00800000:00000080:1.0:1787827167.377800:0:91421:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.377804:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00800000:00000080:1.0:1787827167.377808:0:91421:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.377812:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00800000:00000080:1.0:1787827167.377815:0:91421:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.377819:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00800000:00000080:1.0:1787827167.377833:0:91421:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.377837:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00800000:00000080:1.0:1787827167.377840:0:91421:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.377842:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00800000:00000080:1.0:1787827167.377844:0:91421:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.377846:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00800000:00000080:1.0:1787827167.377848:0:91421:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.377851:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00800000:00000080:1.0:1787827167.377852:0:91421:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.377855:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00800000:00000080:1.0:1787827167.377856:0:91421:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.377859:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00800000:00000080:1.0:1787827167.377860:0:91421:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.377862:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00800000:00000080:1.0:1787827167.377864:0:91421:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.377866:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00800000:00000080:1.0:1787827167.377868:0:91421:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.377870:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00800000:00000080:1.0:1787827167.377871:0:91421:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.377873:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00800000:00000080:1.0:1787827167.377875:0:91421:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.377877:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00800000:00000080:1.0:1787827167.377879:0:91421:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.377881:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00800000:00000080:1.0:1787827167.377883:0:91421:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.377885:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00800000:00000080:1.0:1787827167.377886:0:91421:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.377888:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00800000:00000080:1.0:1787827167.377890:0:91421:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.377892:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00800000:00000080:1.0:1787827167.377894:0:91421:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.377896:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00800000:00000080:1.0:1787827167.377898:0:91421:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.377900:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00800000:00000080:1.0:1787827167.377902:0:91421:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.377906:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00800000:00000080:1.0:1787827167.377910:0:91421:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.377914:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00800000:00000080:1.0:1787827167.377917:0:91421:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.377921:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00800000:00000080:1.0:1787827167.377924:0:91421:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.377928:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00800000:00000080:1.0:1787827167.377932:0:91421:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.377936:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00800000:00000080:1.0:1787827167.377939:0:91421:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.377943:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00800000:00000080:1.0:1787827167.377947:0:91421:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.377951:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00800000:00000080:1.0:1787827167.377954:0:91421:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.377959:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00800000:00000080:1.0:1787827167.377961:0:91421:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.377965:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00800000:00000080:1.0:1787827167.377969:0:91421:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.377973:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00800000:00000080:1.0:1787827167.377976:0:91421:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.377980:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00800000:00000080:1.0:1787827167.377983:0:91421:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.377988:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00800000:00000080:1.0:1787827167.377991:0:91421:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.377995:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00800000:00000080:1.0:1787827167.377999:0:91421:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.378003:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00800000:00000080:1.0:1787827167.378016:0:91421:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.378020:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00800000:00000080:1.0:1787827167.378022:0:91421:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.378024:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00800000:00000080:1.0:1787827167.378026:0:91421:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.378029:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00800000:00000080:1.0:1787827167.378030:0:91421:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.378032:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00800000:00000080:1.0:1787827167.378034:0:91421:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.378037:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00020000:00000080:1.0:1787827167.378043:0:91421:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00020000:00000004:1.0:1787827167.378050:0:91421:0:(obd_class.h:1091:obd_statfs()) lustre-OST0000-osc-ffff985602bc9800: age 3311, max_age 3317 00000100:00100000:1.0:1787827167.378068:0:91421:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a21c0 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:91421:1874669228212096:192.168.203.114@tcp:13:lfs.0 00000100:00100000:1.0:1787827167.378112:0:91421:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bd80 going to sleep for 11 seconds 00000100:00100000:1.0:1787827167.378991:0:91421:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a21c0 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:91421:1874669228212096:192.168.203.114@tcp:13:lfs.0 00020000:00000004:1.0:1787827167.379004:0:91421:0:(obd_class.h:1117:obd_statfs()) lustre-OST0000-osc-ffff985602bc9800: update ffff985609528450 cache blocks 901356/958279 objects 261727/262144 00000080:00200080:1.0:1787827167.379039:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00020000:00000080:1.0:1787827167.379047:0:91421:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00020000:00000004:1.0:1787827167.379051:0:91421:0:(obd_class.h:1091:obd_statfs()) lustre-OST0001-osc-ffff985602bc9800: age 3311, max_age 3317 00000100:00100000:1.0:1787827167.379069:0:91421:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a21c0 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:91421:1874669228212224:192.168.203.114@tcp:13:lfs.0 00000100:00100000:1.0:1787827167.379109:0:91421:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bd80 going to sleep for 11 seconds 00000100:00100000:1.0:1787827167.379898:0:91421:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a21c0 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:91421:1874669228212224:192.168.203.114@tcp:13:lfs.0 00020000:00000004:1.0:1787827167.379910:0:91421:0:(obd_class.h:1117:obd_statfs()) lustre-OST0001-osc-ffff985602bc9800: update ffff98560952d1c0 cache blocks 901363/958279 objects 261735/262144 00000080:00200080:1.0:1787827167.379938:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00020000:00000080:1.0:1787827167.379947:0:91421:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.379953:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00020000:00000080:1.0:1787827167.379958:0:91421:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.379964:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00020000:00000080:1.0:1787827167.379968:0:91421:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.379973:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00020000:00000080:1.0:1787827167.379977:0:91421:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.379981:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00020000:00000080:1.0:1787827167.379983:0:91421:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.379987:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00020000:00000080:1.0:1787827167.379990:0:91421:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.379993:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00020000:00000080:1.0:1787827167.379997:0:91421:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.380001:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00020000:00000080:1.0:1787827167.380004:0:91421:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.380008:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00020000:00000080:1.0:1787827167.380011:0:91421:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.380013:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00020000:00000080:1.0:1787827167.380015:0:91421:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.380019:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00020000:00000080:1.0:1787827167.380021:0:91421:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.380023:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00020000:00000080:1.0:1787827167.380025:0:91421:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.380027:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00020000:00000080:1.0:1787827167.380028:0:91421:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.380030:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00020000:00000080:1.0:1787827167.380032:0:91421:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.380034:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00020000:00000080:1.0:1787827167.380036:0:91421:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.380038:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00020000:00000080:1.0:1787827167.380039:0:91421:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.380041:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00020000:00000080:1.0:1787827167.380043:0:91421:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.380045:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00020000:00000080:1.0:1787827167.380046:0:91421:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.380048:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00020000:00000080:1.0:1787827167.380050:0:91421:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.380052:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00020000:00000080:1.0:1787827167.380054:0:91421:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.380056:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00020000:00000080:1.0:1787827167.380057:0:91421:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.380059:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00020000:00000080:1.0:1787827167.380061:0:91421:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.380063:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00020000:00000080:1.0:1787827167.380065:0:91421:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.380067:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00020000:00000080:1.0:1787827167.380069:0:91421:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.380071:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00020000:00000080:1.0:1787827167.380073:0:91421:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.380075:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00020000:00000080:1.0:1787827167.380077:0:91421:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.380078:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00020000:00000080:1.0:1787827167.380080:0:91421:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.380082:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00020000:00000080:1.0:1787827167.380084:0:91421:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.380086:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00020000:00000080:1.0:1787827167.380087:0:91421:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.380089:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00020000:00000080:1.0:1787827167.380091:0:91421:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.380093:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00020000:00000080:1.0:1787827167.380094:0:91421:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.380096:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00020000:00000080:1.0:1787827167.380098:0:91421:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.380100:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00020000:00000080:1.0:1787827167.380102:0:91421:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.380104:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00020000:00000080:1.0:1787827167.380105:0:91421:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.380107:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00020000:00000080:1.0:1787827167.380109:0:91421:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.380111:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00020000:00000080:1.0:1787827167.380113:0:91421:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.380116:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00020000:00000080:1.0:1787827167.380117:0:91421:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.380119:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00020000:00000080:1.0:1787827167.380121:0:91421:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.380123:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00020000:00000080:1.0:1787827167.380125:0:91421:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.380126:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00020000:00000080:1.0:1787827167.380128:0:91421:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.380130:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00020000:00000080:1.0:1787827167.380132:0:91421:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.380134:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00020000:00000080:1.0:1787827167.380135:0:91421:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.380137:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00020000:00000080:1.0:1787827167.380139:0:91421:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.380141:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00020000:00000080:1.0:1787827167.380142:0:91421:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.380144:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00020000:00000080:1.0:1787827167.380146:0:91421:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.380148:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00020000:00000080:1.0:1787827167.380150:0:91421:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.380151:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00020000:00000080:1.0:1787827167.380153:0:91421:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.380155:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00020000:00000080:1.0:1787827167.380157:0:91421:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.380159:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00020000:00000080:1.0:1787827167.380161:0:91421:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.380163:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00020000:00000080:1.0:1787827167.380165:0:91421:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.380167:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00020000:00000080:1.0:1787827167.380168:0:91421:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.380170:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00020000:00000080:1.0:1787827167.380180:0:91421:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.380183:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00020000:00000080:1.0:1787827167.380185:0:91421:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.380187:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00020000:00000080:1.0:1787827167.380189:0:91421:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.380191:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00020000:00000080:1.0:1787827167.380192:0:91421:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.380194:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00020000:00000080:1.0:1787827167.380196:0:91421:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.380198:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00020000:00000080:1.0:1787827167.380200:0:91421:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.380202:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00020000:00000080:1.0:1787827167.380203:0:91421:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.380205:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00020000:00000080:1.0:1787827167.380207:0:91421:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.380209:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00020000:00000080:1.0:1787827167.380210:0:91421:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.380213:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00020000:00000080:1.0:1787827167.380215:0:91421:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.380217:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00020000:00000080:1.0:1787827167.380219:0:91421:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827167.380221:0:91421:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffe3ce69d50 00020000:00000080:1.0:1787827167.380222:0:91421:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200008:1.0:1787827167.380231:0:91421:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 100000 00000020:00000080:0.0:1787827167.976067:0:91439:0:(class_obd.c:336:class_handle_ioctl()) obdclass: cmd=40086685 len=0 uarg=00007ffeca4e3660 00000002:00000080:0.0:1787827167.976089:0:91439:0:(mdc_request.c:2317:mdc_iocontrol()) lustre-MDT0000-mdc-ffff985602bc9800: cmd=40086685 len=576 karg=ffff985608614800 uarg=0000000000000000 00000100:00080000:0.0:1787827167.976097:0:91439:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from FULL to DISCONN 00000100:02000400:0.0:1787827167.976105:0:91439:0:(import.c:182:ptlrpc_set_import_discon()) lustre-MDT0000-mdc-ffff985602bc9800: Connection to lustre-MDT0000 (at 192.168.203.114@tcp) was lost; in progress operations using this service will wait for recovery to complete 00000100:00080000:0.0:1787827167.984363:0:91439:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827167.984375:0:91439:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3312 00000100:00080000:0.0:1787827167.984391:0:91439:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787827167.984402:0:91439:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827167.984435:0:91439:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5e00 x1874669228212352/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827167.984468:0:91439:0:(recover.c:337:ptlrpc_recover_import()) lustre-MDT0000_UUID: recovery started, waiting 20 seconds 00000100:00100000:3.0:1787827167.984556:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228212352:192.168.203.114@tcp:38:lctl.0 00010000:00010000:0.0:1787827168.052842:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827169.076925:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827170.100849:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827170.548896:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827170.548902:0:54101:0:(pinger.c:247:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:1.0:1787827170.548945:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2383:1874669228212480:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787827170.548970:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669228212608:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827170.549038:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669228212736:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787827170.549040:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669228212864:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787827170.550194:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669228212608:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827170.550195:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2383:1874669228212480:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787827170.550207:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669228212864:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827170.550219:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669228212736:192.168.203.114@tcp:400:kworker.0 00010000:00010000:1.0:1787827171.124885:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827172.148878:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827172.599711:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827172.599722:0:38:0:(pinger.c:247:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1787827172.599743:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2385:1874669228212992:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787827172.599765:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228213120:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787827172.599848:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669228213248:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787827172.600250:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228213376:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787827172.602012:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228213120:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787827172.602013:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669228213248:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787827172.602040:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2385:1874669228212992:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787827172.602050:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228213376:192.168.203.114@tcp:400:kworker.0 00000100:00080000:3.0:1787827172.996803:0:2382:0:(import.c:1132:ptlrpc_connect_interpret()) lustre-MDT0000-mdc-ffff985602bc9800: connect to target with instance 52 00000100:00080000:3.0:1787827172.996840:0:2382:0:(import.c:1297:ptlrpc_connect_interpret()) reconnected to lustre-MDT0000_UUID@192.168.203.114@tcp after partition 00000100:00080000:3.0:1787827172.996844:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to RECOVER 00000100:00080000:3.0:1787827172.996850:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from RECOVER to FULL 00000100:02000000:0.0:1787827172.997182:0:2382:0:(import.c:1743:ptlrpc_import_recovery_state_machine()) lustre-MDT0000-mdc-ffff985602bc9800: Connection restored to 192.168.203.114@tcp (at 192.168.203.114@tcp) 00000100:00100000:0.0:1787827172.997231:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228212352:192.168.203.114@tcp:38:lctl.0 00000100:00080000:1.0:1787827172.997345:0:91439:0:(recover.c:348:ptlrpc_recover_import()) lustre-MDT0000_UUID: recovery finished FULL, rc = 0 00000080:00200008:1.0:1787827173.002478:0:91442:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:1.0:1787827173.002490:0:91442:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787827173.002512:0:91442:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827173.002531:0:91442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 27 lru_type: 1 00010000:00010000:1.0:1787827173.002548:0:91442:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 27 lru_type: 1 00000080:00200000:1.0:1787827173.002561:0:91442:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827173.002568:0:91442:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787827173.002571:0:91442:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee56d 00000080:00010000:1.0:1787827173.002578:0:91442:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee56d from it ffffab3ac0e33cb8 00010000:00010000:1.0:1787827173.002582:0:91442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 27 lru_type: 1 00010000:00010000:1.0:1787827173.002590:0:91442:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 28 lru_type: 1 00000080:00200008:1.0:1787827173.003139:0:91442:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 0, flags 0, foreign 0 00000080:00200000:1.0:1787827173.003144:0:91442:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787827173.003152:0:91442:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827173.003160:0:91442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 28 lru_type: 1 00010000:00010000:1.0:1787827173.003170:0:91442:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 28 lru_type: 1 00000080:00200000:1.0:1787827173.003177:0:91442:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827173.003180:0:91442:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787827173.003183:0:91442:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee56d 00000080:00010000:1.0:1787827173.003187:0:91442:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee56d from it ffffab3ac0e33cb8 00010000:00010000:1.0:1787827173.003190:0:91442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 28 lru_type: 1 00010000:00010000:1.0:1787827173.003203:0:91442:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 29 lru_type: 1 00000080:00200000:1.0:1787827173.003269:0:91442:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787827173.003276:0:91442:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827173.003281:0:91442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 29 lru_type: 1 00010000:00010000:1.0:1787827173.003287:0:91442:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 29 lru_type: 1 00000080:00200000:1.0:1787827173.003291:0:91442:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827173.003294:0:91442:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787827173.003296:0:91442:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee56d 00000080:00010000:1.0:1787827173.003299:0:91442:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee56d from it ffffab3ac0e33bb0 00010000:00010000:1.0:1787827173.003303:0:91442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 29 lru_type: 1 00010000:00010000:1.0:1787827173.003310:0:91442:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 30 lru_type: 1 00000080:00200000:1.0:1787827173.003318:0:91442:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 44 00000080:00200008:1.0:1787827173.003324:0:91442:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 100000 00000080:00200080:1.0:1787827173.003386:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00800000:00000080:1.0:1787827173.003398:0:91442:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00800000:00000004:1.0:1787827173.003403:0:91442:0:(obd_class.h:1091:obd_statfs()) lustre-MDT0000-mdc-ffff985602bc9800: age 3318, max_age 3322 00000100:00100000:1.0:1787827173.003436:0:91442:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3480 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:91442:1874669228213504:192.168.203.114@tcp:41:lfs.0 00000100:00100000:1.0:1787827173.003496:0:91442:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bd80 going to sleep for 12 seconds 00000100:00100000:1.0:1787827173.004590:0:91442:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3480 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:91442:1874669228213504:192.168.203.114@tcp:41:lfs.0 00800000:00000004:1.0:1787827173.004607:0:91442:0:(obd_class.h:1117:obd_statfs()) lustre-MDT0000-mdc-ffff985602bc9800: update ffff985606a337f0 cache blocks 317333/353529 objects 1023688/1024000 00000080:00200080:1.0:1787827173.004641:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00800000:00000080:1.0:1787827173.004647:0:91442:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00800000:00000004:1.0:1787827173.004651:0:91442:0:(obd_class.h:1091:obd_statfs()) lustre-MDT0001-mdc-ffff985602bc9800: age 3318, max_age 3322 00000100:00100000:1.0:1787827173.004666:0:91442:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3480 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:91442:1874669228213632:192.168.203.114@tcp:41:lfs.0 00000100:00100000:1.0:1787827173.004728:0:91442:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bd80 going to sleep for 12 seconds 00000100:00100000:1.0:1787827173.005502:0:91442:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3480 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:91442:1874669228213632:192.168.203.114@tcp:41:lfs.0 00800000:00000004:1.0:1787827173.005511:0:91442:0:(obd_class.h:1117:obd_statfs()) lustre-MDT0001-mdc-ffff985602bc9800: update ffff98560952b7f0 cache blocks 317401/353529 objects 1023737/1024000 00000080:00200080:1.0:1787827173.005535:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00800000:00000080:1.0:1787827173.005543:0:91442:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.005551:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00800000:00000080:1.0:1787827173.005555:0:91442:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.005561:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00800000:00000080:1.0:1787827173.005565:0:91442:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.005570:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00800000:00000080:1.0:1787827173.005573:0:91442:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.005578:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00800000:00000080:1.0:1787827173.005582:0:91442:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.005586:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00800000:00000080:1.0:1787827173.005588:0:91442:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.005591:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00800000:00000080:1.0:1787827173.005594:0:91442:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.005597:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00800000:00000080:1.0:1787827173.005599:0:91442:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.005602:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00800000:00000080:1.0:1787827173.005605:0:91442:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.005609:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00800000:00000080:1.0:1787827173.005611:0:91442:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.005614:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00800000:00000080:1.0:1787827173.005615:0:91442:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.005619:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00800000:00000080:1.0:1787827173.005621:0:91442:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.005625:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00800000:00000080:1.0:1787827173.005627:0:91442:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.005629:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00800000:00000080:1.0:1787827173.005631:0:91442:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.005633:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00800000:00000080:1.0:1787827173.005635:0:91442:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.005637:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00800000:00000080:1.0:1787827173.005639:0:91442:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.005641:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00800000:00000080:1.0:1787827173.005642:0:91442:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.005644:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00800000:00000080:1.0:1787827173.005646:0:91442:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.005648:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00800000:00000080:1.0:1787827173.005650:0:91442:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.005652:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00800000:00000080:1.0:1787827173.005654:0:91442:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.005656:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00800000:00000080:1.0:1787827173.005657:0:91442:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.005659:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00800000:00000080:1.0:1787827173.005662:0:91442:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.005666:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00800000:00000080:1.0:1787827173.005669:0:91442:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.005672:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00800000:00000080:1.0:1787827173.005675:0:91442:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.005679:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00800000:00000080:1.0:1787827173.005683:0:91442:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.005687:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00800000:00000080:1.0:1787827173.005722:0:91442:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.005728:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00800000:00000080:1.0:1787827173.005732:0:91442:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.005735:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00800000:00000080:1.0:1787827173.005737:0:91442:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.005741:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00800000:00000080:1.0:1787827173.005743:0:91442:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.005746:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00800000:00000080:1.0:1787827173.005748:0:91442:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.005750:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00800000:00000080:1.0:1787827173.005752:0:91442:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.005754:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00800000:00000080:1.0:1787827173.005758:0:91442:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.005761:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00800000:00000080:1.0:1787827173.005763:0:91442:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.005766:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00800000:00000080:1.0:1787827173.005768:0:91442:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.005770:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00800000:00000080:1.0:1787827173.005772:0:91442:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.005775:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00800000:00000080:1.0:1787827173.005777:0:91442:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.005779:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00800000:00000080:1.0:1787827173.005781:0:91442:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.005793:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00800000:00000080:1.0:1787827173.005796:0:91442:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.005799:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00800000:00000080:1.0:1787827173.005801:0:91442:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.005803:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00800000:00000080:1.0:1787827173.005805:0:91442:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.005807:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00800000:00000080:1.0:1787827173.005809:0:91442:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.005811:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00800000:00000080:1.0:1787827173.005812:0:91442:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.005814:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00800000:00000080:1.0:1787827173.005816:0:91442:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.005818:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00800000:00000080:1.0:1787827173.005820:0:91442:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.005822:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00800000:00000080:1.0:1787827173.005824:0:91442:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.005826:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00800000:00000080:1.0:1787827173.005828:0:91442:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.005830:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00800000:00000080:1.0:1787827173.005831:0:91442:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.005834:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00800000:00000080:1.0:1787827173.005837:0:91442:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.005841:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00800000:00000080:1.0:1787827173.005844:0:91442:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.005847:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00800000:00000080:1.0:1787827173.005850:0:91442:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.005853:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00800000:00000080:1.0:1787827173.005856:0:91442:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.005860:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00800000:00000080:1.0:1787827173.005864:0:91442:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.005868:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00800000:00000080:1.0:1787827173.005871:0:91442:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.005875:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00800000:00000080:1.0:1787827173.005879:0:91442:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.005883:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00800000:00000080:1.0:1787827173.005886:0:91442:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.005890:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00800000:00000080:1.0:1787827173.005892:0:91442:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.005894:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00800000:00000080:1.0:1787827173.005896:0:91442:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.005899:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00800000:00000080:1.0:1787827173.005901:0:91442:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.005903:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00800000:00000080:1.0:1787827173.005905:0:91442:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.005907:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00800000:00000080:1.0:1787827173.005908:0:91442:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.005910:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00800000:00000080:1.0:1787827173.005912:0:91442:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.005915:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00800000:00000080:1.0:1787827173.005919:0:91442:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.005922:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00800000:00000080:1.0:1787827173.005925:0:91442:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.005929:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00020000:00000080:1.0:1787827173.005947:0:91442:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00020000:00000004:1.0:1787827173.005955:0:91442:0:(obd_class.h:1091:obd_statfs()) lustre-OST0000-osc-ffff985602bc9800: age 3318, max_age 3322 00000100:00100000:1.0:1787827173.005972:0:91442:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3480 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:91442:1874669228213760:192.168.203.114@tcp:13:lfs.0 00000100:00100000:1.0:1787827173.006012:0:91442:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bd80 going to sleep for 11 seconds 00000100:00100000:1.0:1787827173.006839:0:91442:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3480 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:91442:1874669228213760:192.168.203.114@tcp:13:lfs.0 00020000:00000004:1.0:1787827173.006852:0:91442:0:(obd_class.h:1117:obd_statfs()) lustre-OST0000-osc-ffff985602bc9800: update ffff985609528450 cache blocks 901356/958279 objects 261727/262144 00000080:00200080:1.0:1787827173.006885:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00020000:00000080:1.0:1787827173.006891:0:91442:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00020000:00000004:1.0:1787827173.006895:0:91442:0:(obd_class.h:1091:obd_statfs()) lustre-OST0001-osc-ffff985602bc9800: age 3318, max_age 3322 00000100:00100000:1.0:1787827173.006908:0:91442:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3480 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:91442:1874669228213888:192.168.203.114@tcp:13:lfs.0 00000100:00100000:1.0:1787827173.006949:0:91442:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bd80 going to sleep for 11 seconds 00000100:00100000:1.0:1787827173.007792:0:91442:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3480 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:91442:1874669228213888:192.168.203.114@tcp:13:lfs.0 00020000:00000004:1.0:1787827173.007802:0:91442:0:(obd_class.h:1117:obd_statfs()) lustre-OST0001-osc-ffff985602bc9800: update ffff98560952d1c0 cache blocks 901363/958279 objects 261735/262144 00000080:00200080:1.0:1787827173.007824:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00020000:00000080:1.0:1787827173.007830:0:91442:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.007833:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00020000:00000080:1.0:1787827173.007837:0:91442:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.007843:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00020000:00000080:1.0:1787827173.007847:0:91442:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.007852:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00020000:00000080:1.0:1787827173.007857:0:91442:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.007862:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00020000:00000080:1.0:1787827173.007866:0:91442:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.007871:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00020000:00000080:1.0:1787827173.007875:0:91442:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.007881:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00020000:00000080:1.0:1787827173.007885:0:91442:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.007890:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00020000:00000080:1.0:1787827173.007892:0:91442:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.007895:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00020000:00000080:1.0:1787827173.007897:0:91442:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.007899:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00020000:00000080:1.0:1787827173.007901:0:91442:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.007903:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00020000:00000080:1.0:1787827173.007906:0:91442:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.007908:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00020000:00000080:1.0:1787827173.007910:0:91442:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.007912:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00020000:00000080:1.0:1787827173.007915:0:91442:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.007918:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00020000:00000080:1.0:1787827173.007922:0:91442:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.007925:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00020000:00000080:1.0:1787827173.007928:0:91442:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.007932:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00020000:00000080:1.0:1787827173.007936:0:91442:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.007939:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00020000:00000080:1.0:1787827173.007943:0:91442:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.007946:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00020000:00000080:1.0:1787827173.007950:0:91442:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.007954:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00020000:00000080:1.0:1787827173.007958:0:91442:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.007962:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00020000:00000080:1.0:1787827173.007966:0:91442:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.007970:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00020000:00000080:1.0:1787827173.007973:0:91442:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.007977:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00020000:00000080:1.0:1787827173.007980:0:91442:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.007984:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00020000:00000080:1.0:1787827173.007987:0:91442:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.007991:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00020000:00000080:1.0:1787827173.007995:0:91442:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.007999:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00020000:00000080:1.0:1787827173.008002:0:91442:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.008006:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00020000:00000080:1.0:1787827173.008010:0:91442:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.008014:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00020000:00000080:1.0:1787827173.008017:0:91442:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.008020:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00020000:00000080:1.0:1787827173.008022:0:91442:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.008024:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00020000:00000080:1.0:1787827173.008026:0:91442:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.008028:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00020000:00000080:1.0:1787827173.008030:0:91442:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.008032:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00020000:00000080:1.0:1787827173.008033:0:91442:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.008035:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00020000:00000080:1.0:1787827173.008037:0:91442:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.008039:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00020000:00000080:1.0:1787827173.008040:0:91442:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.008042:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00020000:00000080:1.0:1787827173.008044:0:91442:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.008046:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00020000:00000080:1.0:1787827173.008048:0:91442:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.008050:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00020000:00000080:1.0:1787827173.008052:0:91442:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.008054:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00020000:00000080:1.0:1787827173.008056:0:91442:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.008058:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00020000:00000080:1.0:1787827173.008060:0:91442:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.008062:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00020000:00000080:1.0:1787827173.008064:0:91442:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.008066:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00020000:00000080:1.0:1787827173.008067:0:91442:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.008069:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00020000:00000080:1.0:1787827173.008071:0:91442:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.008073:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00020000:00000080:1.0:1787827173.008075:0:91442:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.008076:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00020000:00000080:1.0:1787827173.008078:0:91442:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.008080:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00020000:00000080:1.0:1787827173.008082:0:91442:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.008084:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00020000:00000080:1.0:1787827173.008085:0:91442:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.008095:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00020000:00000080:1.0:1787827173.008097:0:91442:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.008100:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00020000:00000080:1.0:1787827173.008101:0:91442:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.008103:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00020000:00000080:1.0:1787827173.008105:0:91442:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.008108:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00020000:00000080:1.0:1787827173.008110:0:91442:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.008114:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00020000:00000080:1.0:1787827173.008117:0:91442:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.008121:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00020000:00000080:1.0:1787827173.008124:0:91442:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.008128:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00020000:00000080:1.0:1787827173.008131:0:91442:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.008134:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00020000:00000080:1.0:1787827173.008138:0:91442:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.008142:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00020000:00000080:1.0:1787827173.008146:0:91442:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.008150:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00020000:00000080:1.0:1787827173.008153:0:91442:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.008158:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00020000:00000080:1.0:1787827173.008160:0:91442:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.008163:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00020000:00000080:1.0:1787827173.008165:0:91442:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.008167:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00020000:00000080:1.0:1787827173.008169:0:91442:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.008171:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00020000:00000080:1.0:1787827173.008172:0:91442:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.008174:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00020000:00000080:1.0:1787827173.008176:0:91442:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.008178:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00020000:00000080:1.0:1787827173.008186:0:91442:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.008189:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00020000:00000080:1.0:1787827173.008191:0:91442:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200080:1.0:1787827173.008193:0:91442:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd937b2610 00020000:00000080:1.0:1787827173.008195:0:91442:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865e000 uarg=0000000000000000 00000080:00200008:1.0:1787827173.008205:0:91442:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 100000 00010000:00010000:0.0:1787827173.172822:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000080:1.0:1787827173.656031:0:91460:0:(class_obd.c:336:class_handle_ioctl()) obdclass: cmd=40086685 len=0 uarg=00007ffe5dc10d90 00000002:00000080:1.0:1787827173.656051:0:91460:0:(mdc_request.c:2317:mdc_iocontrol()) lustre-MDT0000-mdc-ffff985602bc9800: cmd=40086685 len=576 karg=ffff98560865d800 uarg=0000000000000000 00000100:00080000:1.0:1787827173.656059:0:91460:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from FULL to DISCONN 00000100:02000400:1.0:1787827173.656065:0:91460:0:(import.c:182:ptlrpc_set_import_discon()) lustre-MDT0000-mdc-ffff985602bc9800: Connection to lustre-MDT0000 (at 192.168.203.114@tcp) was lost; in progress operations using this service will wait for recovery to complete 00000100:00080000:1.0:1787827173.656076:0:91460:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827173.656081:0:91460:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3318 00000100:00080000:1.0:1787827173.656095:0:91460:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787827173.656105:0:91460:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827173.656143:0:91460:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a21c0 x1874669228214016/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827173.656170:0:91460:0:(recover.c:337:ptlrpc_recover_import()) lustre-MDT0000_UUID: recovery started, waiting 20 seconds 00000100:00100000:0.0:1787827173.656265:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228214016:192.168.203.114@tcp:38:lctl.0 00010000:00010000:3.0:1787827174.196903:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827175.220876:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827176.244845:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827177.268882:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827177.716883:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827177.716895:0:38:0:(pinger.c:247:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:1.0:1787827177.716925:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2383:1874669228214144:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787827177.716941:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669228214272:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787827177.717013:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669228214528:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827177.717023:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669228214400:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827177.718232:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2383:1874669228214144:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827177.718257:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669228214400:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787827177.718303:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669228214272:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787827177.718317:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669228214528:192.168.203.114@tcp:400:kworker.0 00010000:00010000:0.0:1787827178.292839:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827179.135811:0:2382:0:(import.c:1132:ptlrpc_connect_interpret()) lustre-MDT0000-mdc-ffff985602bc9800: connect to target with instance 52 00000100:00080000:0.0:1787827179.135842:0:2382:0:(import.c:1297:ptlrpc_connect_interpret()) reconnected to lustre-MDT0000_UUID@192.168.203.114@tcp after partition 00000100:00080000:0.0:1787827179.135846:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to RECOVER 00000100:00080000:0.0:1787827179.135852:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from RECOVER to FULL 00000100:02000000:0.0:1787827179.136060:0:2382:0:(import.c:1743:ptlrpc_import_recovery_state_machine()) lustre-MDT0000-mdc-ffff985602bc9800: Connection restored to 192.168.203.114@tcp (at 192.168.203.114@tcp) 00000100:00100000:0.0:1787827179.136072:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228214016:192.168.203.114@tcp:38:lctl.0 00000100:00080000:1.0:1787827179.136100:0:91460:0:(recover.c:348:ptlrpc_recover_import()) lustre-MDT0000_UUID: recovery finished FULL, rc = 0 00000080:00200008:0.0:1787827179.141307:0:91463:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:0.0:1787827179.141318:0:91463:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:0.0:1787827179.141340:0:91463:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827179.141358:0:91463:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 30 lru_type: 1 00010000:00010000:0.0:1787827179.141376:0:91463:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 30 lru_type: 1 00000080:00200000:0.0:1787827179.141382:0:91463:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827179.141388:0:91463:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:0.0:1787827179.141391:0:91463:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee56d 00000080:00010000:0.0:1787827179.141397:0:91463:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee56d from it ffffab3ac1ef7cb8 00010000:00010000:0.0:1787827179.141400:0:91463:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 30 lru_type: 1 00010000:00010000:0.0:1787827179.141406:0:91463:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 31 lru_type: 1 00000080:00200008:0.0:1787827179.141906:0:91463:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 0, flags 0, foreign 0 00000080:00200000:0.0:1787827179.141910:0:91463:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:0.0:1787827179.141916:0:91463:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827179.141921:0:91463:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 31 lru_type: 1 00010000:00010000:0.0:1787827179.141927:0:91463:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 31 lru_type: 1 00000080:00200000:0.0:1787827179.141932:0:91463:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827179.141934:0:91463:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:0.0:1787827179.141936:0:91463:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee56d 00000080:00010000:0.0:1787827179.141939:0:91463:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee56d from it ffffab3ac1ef7cb8 00010000:00010000:0.0:1787827179.141942:0:91463:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 31 lru_type: 1 00010000:00010000:0.0:1787827179.141947:0:91463:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 32 lru_type: 1 00000080:00200000:0.0:1787827179.142013:0:91463:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:0.0:1787827179.142019:0:91463:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827179.142023:0:91463:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 32 lru_type: 1 00010000:00010000:0.0:1787827179.142028:0:91463:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 32 lru_type: 1 00000080:00200000:0.0:1787827179.142033:0:91463:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827179.142035:0:91463:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:0.0:1787827179.142037:0:91463:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee56d 00000080:00010000:0.0:1787827179.142039:0:91463:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee56d from it ffffab3ac1ef7bb0 00010000:00010000:0.0:1787827179.142041:0:91463:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 32 lru_type: 1 00010000:00010000:0.0:1787827179.142046:0:91463:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 33 lru_type: 1 00000080:00200000:0.0:1787827179.142050:0:91463:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 44 00000080:00200008:0.0:1787827179.142055:0:91463:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 100000 00000080:00200080:0.0:1787827179.142120:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00800000:00000080:0.0:1787827179.142134:0:91463:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00800000:00000004:0.0:1787827179.142140:0:91463:0:(obd_class.h:1091:obd_statfs()) lustre-MDT0000-mdc-ffff985602bc9800: age 3323, max_age 3328 00000100:00100000:0.0:1787827179.142182:0:91463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4f00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:91463:1874669228214656:192.168.203.114@tcp:41:lfs.0 00000100:00100000:0.0:1787827179.142265:0:91463:0:(client.c:2715:ptlrpc_set_wait()) set ffff98560860be40 going to sleep for 12 seconds 00000100:00100000:0.0:1787827179.143399:0:91463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4f00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:91463:1874669228214656:192.168.203.114@tcp:41:lfs.0 00800000:00000004:0.0:1787827179.143421:0:91463:0:(obd_class.h:1117:obd_statfs()) lustre-MDT0000-mdc-ffff985602bc9800: update ffff985606a337f0 cache blocks 317333/353529 objects 1023688/1024000 00000080:00200080:0.0:1787827179.143462:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00800000:00000080:0.0:1787827179.143471:0:91463:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00800000:00000004:0.0:1787827179.143477:0:91463:0:(obd_class.h:1091:obd_statfs()) lustre-MDT0001-mdc-ffff985602bc9800: age 3323, max_age 3328 00000100:00100000:0.0:1787827179.143496:0:91463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4f00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:91463:1874669228214784:192.168.203.114@tcp:41:lfs.0 00000100:00100000:0.0:1787827179.143536:0:91463:0:(client.c:2715:ptlrpc_set_wait()) set ffff98560860be40 going to sleep for 12 seconds 00000100:00100000:0.0:1787827179.144294:0:91463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4f00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:91463:1874669228214784:192.168.203.114@tcp:41:lfs.0 00800000:00000004:0.0:1787827179.144307:0:91463:0:(obd_class.h:1117:obd_statfs()) lustre-MDT0001-mdc-ffff985602bc9800: update ffff98560952b7f0 cache blocks 317401/353529 objects 1023737/1024000 00000080:00200080:0.0:1787827179.144336:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00800000:00000080:0.0:1787827179.144345:0:91463:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.144352:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00800000:00000080:0.0:1787827179.144357:0:91463:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.144363:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00800000:00000080:0.0:1787827179.144368:0:91463:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.144373:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00800000:00000080:0.0:1787827179.144378:0:91463:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.144382:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00800000:00000080:0.0:1787827179.144386:0:91463:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.144390:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00800000:00000080:0.0:1787827179.144394:0:91463:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.144399:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00800000:00000080:0.0:1787827179.144402:0:91463:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.144407:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00800000:00000080:0.0:1787827179.144410:0:91463:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.144414:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00800000:00000080:0.0:1787827179.144417:0:91463:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.144421:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00800000:00000080:0.0:1787827179.144424:0:91463:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.144428:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00800000:00000080:0.0:1787827179.144431:0:91463:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.144434:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00800000:00000080:0.0:1787827179.144437:0:91463:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.144440:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00800000:00000080:0.0:1787827179.144443:0:91463:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.144446:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00800000:00000080:0.0:1787827179.144449:0:91463:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.144453:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00800000:00000080:0.0:1787827179.144456:0:91463:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.144460:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00800000:00000080:0.0:1787827179.144464:0:91463:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.144468:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00800000:00000080:0.0:1787827179.144472:0:91463:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.144476:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00800000:00000080:0.0:1787827179.144480:0:91463:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.144484:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00800000:00000080:0.0:1787827179.144488:0:91463:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.144492:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00800000:00000080:0.0:1787827179.144496:0:91463:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.144500:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00800000:00000080:0.0:1787827179.144504:0:91463:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.144508:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00800000:00000080:0.0:1787827179.144512:0:91463:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.144516:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00800000:00000080:0.0:1787827179.144520:0:91463:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.144524:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00800000:00000080:0.0:1787827179.144528:0:91463:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.144533:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00800000:00000080:0.0:1787827179.144536:0:91463:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.144540:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00800000:00000080:0.0:1787827179.144544:0:91463:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.144548:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00800000:00000080:0.0:1787827179.144553:0:91463:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.144557:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00800000:00000080:0.0:1787827179.144561:0:91463:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.144565:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00800000:00000080:0.0:1787827179.144569:0:91463:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.144574:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00800000:00000080:0.0:1787827179.144578:0:91463:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.144582:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00800000:00000080:0.0:1787827179.144586:0:91463:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.144591:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00800000:00000080:0.0:1787827179.144595:0:91463:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.144598:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00800000:00000080:0.0:1787827179.144602:0:91463:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.144606:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00800000:00000080:0.0:1787827179.144609:0:91463:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.144613:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00800000:00000080:0.0:1787827179.144618:0:91463:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.144623:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00800000:00000080:0.0:1787827179.144627:0:91463:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.144631:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00800000:00000080:0.0:1787827179.144635:0:91463:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.144639:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00800000:00000080:0.0:1787827179.144643:0:91463:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.144647:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00800000:00000080:0.0:1787827179.144651:0:91463:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.144655:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00800000:00000080:0.0:1787827179.144659:0:91463:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.144663:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00800000:00000080:0.0:1787827179.144667:0:91463:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.144671:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00800000:00000080:0.0:1787827179.144675:0:91463:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.144680:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00800000:00000080:0.0:1787827179.144683:0:91463:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.144687:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00800000:00000080:0.0:1787827179.144715:0:91463:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.144723:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00800000:00000080:0.0:1787827179.144727:0:91463:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.144732:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00800000:00000080:0.0:1787827179.144736:0:91463:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.144754:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00800000:00000080:0.0:1787827179.144760:0:91463:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.144765:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00800000:00000080:0.0:1787827179.144768:0:91463:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.144773:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00800000:00000080:0.0:1787827179.144776:0:91463:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.144780:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00800000:00000080:0.0:1787827179.144784:0:91463:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.144788:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00800000:00000080:0.0:1787827179.144792:0:91463:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.144796:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00800000:00000080:0.0:1787827179.144800:0:91463:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.144805:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00800000:00000080:0.0:1787827179.144808:0:91463:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.144813:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00800000:00000080:0.0:1787827179.144817:0:91463:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.144821:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00800000:00000080:0.0:1787827179.144825:0:91463:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.144829:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00800000:00000080:0.0:1787827179.144832:0:91463:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.144838:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00800000:00000080:0.0:1787827179.144842:0:91463:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.144846:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00800000:00000080:0.0:1787827179.144850:0:91463:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.144854:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00800000:00000080:0.0:1787827179.144858:0:91463:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.144862:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00800000:00000080:0.0:1787827179.144867:0:91463:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.144871:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00800000:00000080:0.0:1787827179.144875:0:91463:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.144879:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00800000:00000080:0.0:1787827179.144883:0:91463:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.144887:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00800000:00000080:0.0:1787827179.144891:0:91463:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.144896:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00020000:00000080:0.0:1787827179.144902:0:91463:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00020000:00000004:0.0:1787827179.144910:0:91463:0:(obd_class.h:1091:obd_statfs()) lustre-OST0000-osc-ffff985602bc9800: age 3323, max_age 3328 00000100:00100000:0.0:1787827179.144933:0:91463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4f00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:91463:1874669228214912:192.168.203.114@tcp:13:lfs.0 00000100:00100000:0.0:1787827179.144978:0:91463:0:(client.c:2715:ptlrpc_set_wait()) set ffff98560860be40 going to sleep for 11 seconds 00000100:00100000:0.0:1787827179.145636:0:91463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4f00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:91463:1874669228214912:192.168.203.114@tcp:13:lfs.0 00020000:00000004:0.0:1787827179.145648:0:91463:0:(obd_class.h:1117:obd_statfs()) lustre-OST0000-osc-ffff985602bc9800: update ffff985609528450 cache blocks 901356/958279 objects 261727/262144 00000080:00200080:0.0:1787827179.145678:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00020000:00000080:0.0:1787827179.145686:0:91463:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00020000:00000004:0.0:1787827179.145710:0:91463:0:(obd_class.h:1091:obd_statfs()) lustre-OST0001-osc-ffff985602bc9800: age 3323, max_age 3328 00000100:00100000:0.0:1787827179.145732:0:91463:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4f00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:91463:1874669228215040:192.168.203.114@tcp:13:lfs.0 00000100:00100000:0.0:1787827179.145789:0:91463:0:(client.c:2715:ptlrpc_set_wait()) set ffff98560860be40 going to sleep for 11 seconds 00000100:00100000:0.0:1787827179.146415:0:91463:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4f00 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:91463:1874669228215040:192.168.203.114@tcp:13:lfs.0 00020000:00000004:0.0:1787827179.146426:0:91463:0:(obd_class.h:1117:obd_statfs()) lustre-OST0001-osc-ffff985602bc9800: update ffff98560952d1c0 cache blocks 901363/958279 objects 261735/262144 00000080:00200080:0.0:1787827179.146453:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00020000:00000080:0.0:1787827179.146461:0:91463:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.146467:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00020000:00000080:0.0:1787827179.146472:0:91463:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.146478:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00020000:00000080:0.0:1787827179.146483:0:91463:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.146488:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00020000:00000080:0.0:1787827179.146492:0:91463:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.146497:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00020000:00000080:0.0:1787827179.146501:0:91463:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.146505:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00020000:00000080:0.0:1787827179.146508:0:91463:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.146512:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00020000:00000080:0.0:1787827179.146515:0:91463:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.146519:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00020000:00000080:0.0:1787827179.146522:0:91463:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.146526:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00020000:00000080:0.0:1787827179.146529:0:91463:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.146532:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00020000:00000080:0.0:1787827179.146536:0:91463:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.146543:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00020000:00000080:0.0:1787827179.146547:0:91463:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.146551:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00020000:00000080:0.0:1787827179.146556:0:91463:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.146560:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00020000:00000080:0.0:1787827179.146563:0:91463:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.146567:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00020000:00000080:0.0:1787827179.146570:0:91463:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.146574:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00020000:00000080:0.0:1787827179.146577:0:91463:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.146580:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00020000:00000080:0.0:1787827179.146583:0:91463:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.146587:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00020000:00000080:0.0:1787827179.146590:0:91463:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.146593:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00020000:00000080:0.0:1787827179.146596:0:91463:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.146600:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00020000:00000080:0.0:1787827179.146604:0:91463:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.146606:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00020000:00000080:0.0:1787827179.146608:0:91463:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.146611:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00020000:00000080:0.0:1787827179.146613:0:91463:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.146615:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00020000:00000080:0.0:1787827179.146617:0:91463:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.146619:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00020000:00000080:0.0:1787827179.146621:0:91463:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.146623:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00020000:00000080:0.0:1787827179.146625:0:91463:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.146627:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00020000:00000080:0.0:1787827179.146629:0:91463:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.146631:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00020000:00000080:0.0:1787827179.146633:0:91463:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.146635:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00020000:00000080:0.0:1787827179.146636:0:91463:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.146638:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00020000:00000080:0.0:1787827179.146640:0:91463:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.146642:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00020000:00000080:0.0:1787827179.146644:0:91463:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.146646:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00020000:00000080:0.0:1787827179.146647:0:91463:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.146649:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00020000:00000080:0.0:1787827179.146651:0:91463:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.146655:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00020000:00000080:0.0:1787827179.146658:0:91463:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.146661:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00020000:00000080:0.0:1787827179.146665:0:91463:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.146668:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00020000:00000080:0.0:1787827179.146672:0:91463:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.146676:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00020000:00000080:0.0:1787827179.146679:0:91463:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.146684:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00020000:00000080:0.0:1787827179.146687:0:91463:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.146713:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00020000:00000080:0.0:1787827179.146718:0:91463:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.146723:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00020000:00000080:0.0:1787827179.146727:0:91463:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.146732:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00020000:00000080:0.0:1787827179.146735:0:91463:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.146739:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00020000:00000080:0.0:1787827179.146755:0:91463:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.146759:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00020000:00000080:0.0:1787827179.146763:0:91463:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.146766:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00020000:00000080:0.0:1787827179.146769:0:91463:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.146773:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00020000:00000080:0.0:1787827179.146777:0:91463:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.146781:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00020000:00000080:0.0:1787827179.146783:0:91463:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.146786:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00020000:00000080:0.0:1787827179.146787:0:91463:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.146789:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00020000:00000080:0.0:1787827179.146791:0:91463:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.146793:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00020000:00000080:0.0:1787827179.146795:0:91463:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.146797:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00020000:00000080:0.0:1787827179.146799:0:91463:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.146802:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00020000:00000080:0.0:1787827179.146804:0:91463:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.146806:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00020000:00000080:0.0:1787827179.146808:0:91463:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.146810:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00020000:00000080:0.0:1787827179.146811:0:91463:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.146813:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00020000:00000080:0.0:1787827179.146815:0:91463:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.146817:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00020000:00000080:0.0:1787827179.146818:0:91463:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.146820:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00020000:00000080:0.0:1787827179.146822:0:91463:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.146824:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00020000:00000080:0.0:1787827179.146826:0:91463:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.146828:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00020000:00000080:0.0:1787827179.146829:0:91463:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.146831:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00020000:00000080:0.0:1787827179.146833:0:91463:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.146835:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00020000:00000080:0.0:1787827179.146836:0:91463:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.146838:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00020000:00000080:0.0:1787827179.146840:0:91463:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.146842:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00020000:00000080:0.0:1787827179.146843:0:91463:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.146846:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00020000:00000080:0.0:1787827179.146848:0:91463:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.146850:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00020000:00000080:0.0:1787827179.146851:0:91463:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200080:0.0:1787827179.146853:0:91463:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fffb24f2f90 00020000:00000080:0.0:1787827179.146855:0:91463:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff985608615000 uarg=0000000000000000 00000080:00200008:0.0:1787827179.146865:0:91463:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 100000 00010000:00010000:3.0:1787827179.316886:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000080:1.0:1787827179.707406:0:91482:0:(class_obd.c:336:class_handle_ioctl()) obdclass: cmd=40086685 len=0 uarg=00007ffca3344cd0 00000002:00000080:1.0:1787827179.707431:0:91482:0:(mdc_request.c:2317:mdc_iocontrol()) lustre-MDT0000-mdc-ffff985602bc9800: cmd=40086685 len=576 karg=ffff98560865d800 uarg=0000000000000000 00000100:00080000:1.0:1787827179.707442:0:91482:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from FULL to DISCONN 00000100:02000400:1.0:1787827179.707452:0:91482:0:(import.c:182:ptlrpc_set_import_discon()) lustre-MDT0000-mdc-ffff985602bc9800: Connection to lustre-MDT0000 (at 192.168.203.114@tcp) was lost; in progress operations using this service will wait for recovery to complete 00000100:00080000:1.0:1787827179.707460:0:91482:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827179.707465:0:91482:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3324 00000100:00080000:1.0:1787827179.707485:0:91482:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787827179.707499:0:91482:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827179.707534:0:91482:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2940 x1874669228215168/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827179.707563:0:91482:0:(recover.c:337:ptlrpc_recover_import()) lustre-MDT0000_UUID: recovery started, waiting 20 seconds 00000100:00100000:0.0:1787827179.707594:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228215168:192.168.203.114@tcp:38:lctl.0 00010000:00010000:3.0:1787827180.340916:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827181.364926:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827182.388839:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827182.836831:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827182.836836:0:38:0:(pinger.c:247:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1787827182.836876:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2385:1874669228215296:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787827182.836887:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228215424:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787827182.836959:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669228215552:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787827182.836960:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228215680:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787827182.838158:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2385:1874669228215296:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787827182.838172:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228215424:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787827182.838178:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669228215552:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787827182.838190:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228215680:192.168.203.114@tcp:400:kworker.0 00010000:00010000:0.0:1787827183.412848:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827183.860905:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827183.860911:0:38:0:(pinger.c:247:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:1.0:1787827183.860935:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2383:1874669228215808:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787827183.860959:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669228215936:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787827183.861011:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669228216192:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827183.861021:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669228216064:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827183.862143:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669228216064:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787827183.862144:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669228215936:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827183.862155:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2383:1874669228215808:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787827183.862177:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669228216192:192.168.203.114@tcp:400:kworker.0 00010000:00010000:3.0:1787827184.436817:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827184.772453:0:2382:0:(import.c:1132:ptlrpc_connect_interpret()) lustre-MDT0000-mdc-ffff985602bc9800: connect to target with instance 52 00000100:00080000:0.0:1787827184.772491:0:2382:0:(import.c:1297:ptlrpc_connect_interpret()) reconnected to lustre-MDT0000_UUID@192.168.203.114@tcp after partition 00000100:00080000:0.0:1787827184.772495:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to RECOVER 00000100:00080000:0.0:1787827184.772503:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from RECOVER to FULL 00000100:02000000:0.0:1787827184.772785:0:2382:0:(import.c:1743:ptlrpc_import_recovery_state_machine()) lustre-MDT0000-mdc-ffff985602bc9800: Connection restored to 192.168.203.114@tcp (at 192.168.203.114@tcp) 00000100:00100000:0.0:1787827184.772801:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228215168:192.168.203.114@tcp:38:lctl.0 00000100:00080000:1.0:1787827184.772825:0:91482:0:(recover.c:348:ptlrpc_recover_import()) lustre-MDT0000_UUID: recovery finished FULL, rc = 0 00000080:00200008:1.0:1787827184.777957:0:91485:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:1.0:1787827184.777967:0:91485:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787827184.777987:0:91485:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827184.778005:0:91485:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 33 lru_type: 1 00010000:00010000:1.0:1787827184.778018:0:91485:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 33 lru_type: 1 00000080:00200000:1.0:1787827184.778024:0:91485:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827184.778030:0:91485:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787827184.778032:0:91485:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee56d 00000080:00010000:1.0:1787827184.778038:0:91485:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee56d from it ffffab3ac1eb7cb8 00010000:00010000:1.0:1787827184.778041:0:91485:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 33 lru_type: 1 00010000:00010000:1.0:1787827184.778047:0:91485:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 34 lru_type: 1 00000080:00200008:1.0:1787827184.778486:0:91485:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 0, flags 0, foreign 0 00000080:00200000:1.0:1787827184.778489:0:91485:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787827184.778495:0:91485:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827184.778500:0:91485:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 34 lru_type: 1 00010000:00010000:1.0:1787827184.778506:0:91485:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 34 lru_type: 1 00000080:00200000:1.0:1787827184.778511:0:91485:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827184.778513:0:91485:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787827184.778515:0:91485:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee56d 00000080:00010000:1.0:1787827184.778518:0:91485:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee56d from it ffffab3ac1eb7cb8 00010000:00010000:1.0:1787827184.778520:0:91485:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 34 lru_type: 1 00010000:00010000:1.0:1787827184.778526:0:91485:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 35 lru_type: 1 00000080:00200000:1.0:1787827184.778587:0:91485:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787827184.778593:0:91485:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827184.778597:0:91485:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 35 lru_type: 1 00010000:00010000:1.0:1787827184.778602:0:91485:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 35 lru_type: 1 00000080:00200000:1.0:1787827184.778607:0:91485:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827184.778609:0:91485:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787827184.778611:0:91485:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee56d 00000080:00010000:1.0:1787827184.778613:0:91485:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee56d from it ffffab3ac1eb7bb0 00010000:00010000:1.0:1787827184.778615:0:91485:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 35 lru_type: 1 00010000:00010000:1.0:1787827184.778620:0:91485:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 36 lru_type: 1 00000080:00200000:1.0:1787827184.778625:0:91485:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 44 00000080:00200008:1.0:1787827184.778631:0:91485:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 100000 00000080:00200080:1.0:1787827184.778759:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00800000:00000080:1.0:1787827184.778772:0:91485:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00800000:00000004:1.0:1787827184.778779:0:91485:0:(obd_class.h:1091:obd_statfs()) lustre-MDT0000-mdc-ffff985602bc9800: age 3329, max_age 3334 00000100:00100000:1.0:1787827184.778819:0:91485:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2580 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:91485:1874669228216320:192.168.203.114@tcp:41:lfs.0 00000100:00100000:1.0:1787827184.778894:0:91485:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5b6c0 going to sleep for 12 seconds 00000100:00100000:1.0:1787827184.779932:0:91485:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2580 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:91485:1874669228216320:192.168.203.114@tcp:41:lfs.0 00800000:00000004:1.0:1787827184.779947:0:91485:0:(obd_class.h:1117:obd_statfs()) lustre-MDT0000-mdc-ffff985602bc9800: update ffff985606a337f0 cache blocks 317333/353529 objects 1023688/1024000 00000080:00200080:1.0:1787827184.779976:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00800000:00000080:1.0:1787827184.779982:0:91485:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00800000:00000004:1.0:1787827184.779986:0:91485:0:(obd_class.h:1091:obd_statfs()) lustre-MDT0001-mdc-ffff985602bc9800: age 3329, max_age 3334 00000100:00100000:1.0:1787827184.779998:0:91485:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2580 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:91485:1874669228216448:192.168.203.114@tcp:41:lfs.0 00000100:00100000:1.0:1787827184.780029:0:91485:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5b6c0 going to sleep for 12 seconds 00000100:00100000:1.0:1787827184.780675:0:91485:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2580 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:91485:1874669228216448:192.168.203.114@tcp:41:lfs.0 00800000:00000004:1.0:1787827184.780684:0:91485:0:(obd_class.h:1117:obd_statfs()) lustre-MDT0001-mdc-ffff985602bc9800: update ffff98560952b7f0 cache blocks 317401/353529 objects 1023737/1024000 00000080:00200080:1.0:1787827184.780737:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00800000:00000080:1.0:1787827184.780743:0:91485:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.780748:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00800000:00000080:1.0:1787827184.780751:0:91485:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.780754:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00800000:00000080:1.0:1787827184.780756:0:91485:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.780759:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00800000:00000080:1.0:1787827184.780761:0:91485:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.780764:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00800000:00000080:1.0:1787827184.780765:0:91485:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.780767:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00800000:00000080:1.0:1787827184.780769:0:91485:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.780772:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00800000:00000080:1.0:1787827184.780774:0:91485:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.780776:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00800000:00000080:1.0:1787827184.780778:0:91485:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.780780:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00800000:00000080:1.0:1787827184.780782:0:91485:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.780784:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00800000:00000080:1.0:1787827184.780785:0:91485:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.780787:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00800000:00000080:1.0:1787827184.780789:0:91485:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.780791:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00800000:00000080:1.0:1787827184.780793:0:91485:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.780795:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00800000:00000080:1.0:1787827184.780796:0:91485:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.780798:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00800000:00000080:1.0:1787827184.780800:0:91485:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.780802:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00800000:00000080:1.0:1787827184.780803:0:91485:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.780805:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00800000:00000080:1.0:1787827184.780807:0:91485:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.780809:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00800000:00000080:1.0:1787827184.780811:0:91485:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.780812:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00800000:00000080:1.0:1787827184.780814:0:91485:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.780816:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00800000:00000080:1.0:1787827184.780818:0:91485:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.780821:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00800000:00000080:1.0:1787827184.780823:0:91485:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.780825:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00800000:00000080:1.0:1787827184.780827:0:91485:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.780828:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00800000:00000080:1.0:1787827184.780830:0:91485:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.780832:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00800000:00000080:1.0:1787827184.780834:0:91485:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.780836:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00800000:00000080:1.0:1787827184.780837:0:91485:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.780839:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00800000:00000080:1.0:1787827184.780841:0:91485:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.780843:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00800000:00000080:1.0:1787827184.780844:0:91485:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.780846:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00800000:00000080:1.0:1787827184.780848:0:91485:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.780850:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00800000:00000080:1.0:1787827184.780852:0:91485:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.780853:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00800000:00000080:1.0:1787827184.780855:0:91485:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.780857:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00800000:00000080:1.0:1787827184.780859:0:91485:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.780861:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00800000:00000080:1.0:1787827184.780862:0:91485:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.780865:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00800000:00000080:1.0:1787827184.780867:0:91485:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.780869:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00800000:00000080:1.0:1787827184.780871:0:91485:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.780873:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00800000:00000080:1.0:1787827184.780875:0:91485:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.780877:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00800000:00000080:1.0:1787827184.780878:0:91485:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.780880:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00800000:00000080:1.0:1787827184.780882:0:91485:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.780884:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00800000:00000080:1.0:1787827184.780885:0:91485:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.780887:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00800000:00000080:1.0:1787827184.780889:0:91485:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.780891:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00800000:00000080:1.0:1787827184.780893:0:91485:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.780895:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00800000:00000080:1.0:1787827184.780896:0:91485:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.780898:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00800000:00000080:1.0:1787827184.780900:0:91485:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.780902:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00800000:00000080:1.0:1787827184.780903:0:91485:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.780905:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00800000:00000080:1.0:1787827184.780907:0:91485:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.780909:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00800000:00000080:1.0:1787827184.780911:0:91485:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.780913:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00800000:00000080:1.0:1787827184.780915:0:91485:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.780917:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00800000:00000080:1.0:1787827184.780919:0:91485:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.780921:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00800000:00000080:1.0:1787827184.780922:0:91485:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.780924:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00800000:00000080:1.0:1787827184.780926:0:91485:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.780928:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00800000:00000080:1.0:1787827184.780929:0:91485:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.780931:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00800000:00000080:1.0:1787827184.780933:0:91485:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.780935:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00800000:00000080:1.0:1787827184.780937:0:91485:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.780939:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00800000:00000080:1.0:1787827184.780940:0:91485:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.780942:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00800000:00000080:1.0:1787827184.780944:0:91485:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.780946:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00800000:00000080:1.0:1787827184.780947:0:91485:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.780949:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00800000:00000080:1.0:1787827184.780951:0:91485:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.780953:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00800000:00000080:1.0:1787827184.780955:0:91485:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.780957:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00800000:00000080:1.0:1787827184.780959:0:91485:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.780961:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00800000:00000080:1.0:1787827184.780963:0:91485:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.780964:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00800000:00000080:1.0:1787827184.780966:0:91485:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.780968:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00800000:00000080:1.0:1787827184.780970:0:91485:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.780972:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00800000:00000080:1.0:1787827184.780973:0:91485:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.780975:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00800000:00000080:1.0:1787827184.780977:0:91485:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.780979:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00800000:00000080:1.0:1787827184.780981:0:91485:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.780983:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00020000:00000080:1.0:1787827184.780987:0:91485:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00020000:00000004:1.0:1787827184.780991:0:91485:0:(obd_class.h:1091:obd_statfs()) lustre-OST0000-osc-ffff985602bc9800: age 3329, max_age 3334 00000100:00100000:1.0:1787827184.781004:0:91485:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2580 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:91485:1874669228216576:192.168.203.114@tcp:13:lfs.0 00000100:00100000:1.0:1787827184.781031:0:91485:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5b6c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827184.781577:0:91485:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2580 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:91485:1874669228216576:192.168.203.114@tcp:13:lfs.0 00020000:00000004:1.0:1787827184.781584:0:91485:0:(obd_class.h:1117:obd_statfs()) lustre-OST0000-osc-ffff985602bc9800: update ffff985609528450 cache blocks 901356/958279 objects 261727/262144 00000080:00200080:1.0:1787827184.781604:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00020000:00000080:1.0:1787827184.781609:0:91485:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00020000:00000004:1.0:1787827184.781611:0:91485:0:(obd_class.h:1091:obd_statfs()) lustre-OST0001-osc-ffff985602bc9800: age 3329, max_age 3334 00000100:00100000:1.0:1787827184.781623:0:91485:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2580 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:91485:1874669228216704:192.168.203.114@tcp:13:lfs.0 00000100:00100000:1.0:1787827184.781650:0:91485:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5b6c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827184.782211:0:91485:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2580 pname:cluuid:pid:xid:nid:opc:job lfs:7b4e8535-9db7-485d-abf6-bda966b63320:91485:1874669228216704:192.168.203.114@tcp:13:lfs.0 00020000:00000004:1.0:1787827184.782221:0:91485:0:(obd_class.h:1117:obd_statfs()) lustre-OST0001-osc-ffff985602bc9800: update ffff98560952d1c0 cache blocks 901363/958279 objects 261735/262144 00000080:00200080:1.0:1787827184.782241:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00020000:00000080:1.0:1787827184.782246:0:91485:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.782249:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00020000:00000080:1.0:1787827184.782251:0:91485:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.782254:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00020000:00000080:1.0:1787827184.782256:0:91485:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.782259:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00020000:00000080:1.0:1787827184.782262:0:91485:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.782264:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00020000:00000080:1.0:1787827184.782267:0:91485:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.782271:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00020000:00000080:1.0:1787827184.782273:0:91485:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.782279:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00020000:00000080:1.0:1787827184.782281:0:91485:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.782285:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00020000:00000080:1.0:1787827184.782286:0:91485:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.782288:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00020000:00000080:1.0:1787827184.782290:0:91485:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.782292:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00020000:00000080:1.0:1787827184.782294:0:91485:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.782297:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00020000:00000080:1.0:1787827184.782299:0:91485:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.782300:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00020000:00000080:1.0:1787827184.782302:0:91485:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.782304:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00020000:00000080:1.0:1787827184.782306:0:91485:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.782307:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00020000:00000080:1.0:1787827184.782309:0:91485:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.782311:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00020000:00000080:1.0:1787827184.782313:0:91485:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.782315:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00020000:00000080:1.0:1787827184.782316:0:91485:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.782318:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00020000:00000080:1.0:1787827184.782320:0:91485:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.782322:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00020000:00000080:1.0:1787827184.782323:0:91485:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.782325:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00020000:00000080:1.0:1787827184.782327:0:91485:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.782329:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00020000:00000080:1.0:1787827184.782330:0:91485:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.782332:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00020000:00000080:1.0:1787827184.782334:0:91485:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.782336:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00020000:00000080:1.0:1787827184.782337:0:91485:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.782339:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00020000:00000080:1.0:1787827184.782342:0:91485:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.782343:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00020000:00000080:1.0:1787827184.782345:0:91485:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.782347:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00020000:00000080:1.0:1787827184.782349:0:91485:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.782350:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00020000:00000080:1.0:1787827184.782352:0:91485:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.782354:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00020000:00000080:1.0:1787827184.782356:0:91485:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.782357:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00020000:00000080:1.0:1787827184.782359:0:91485:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.782361:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00020000:00000080:1.0:1787827184.782363:0:91485:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.782364:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00020000:00000080:1.0:1787827184.782366:0:91485:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.782368:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00020000:00000080:1.0:1787827184.782370:0:91485:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.782371:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00020000:00000080:1.0:1787827184.782373:0:91485:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.782375:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00020000:00000080:1.0:1787827184.782377:0:91485:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.782379:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00020000:00000080:1.0:1787827184.782380:0:91485:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.782382:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00020000:00000080:1.0:1787827184.782384:0:91485:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.782386:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00020000:00000080:1.0:1787827184.782388:0:91485:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.782390:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00020000:00000080:1.0:1787827184.782392:0:91485:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.782394:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00020000:00000080:1.0:1787827184.782395:0:91485:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.782397:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00020000:00000080:1.0:1787827184.782399:0:91485:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.782401:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00020000:00000080:1.0:1787827184.782402:0:91485:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.782404:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00020000:00000080:1.0:1787827184.782406:0:91485:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.782408:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00020000:00000080:1.0:1787827184.782409:0:91485:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.782411:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00020000:00000080:1.0:1787827184.782413:0:91485:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.782415:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00020000:00000080:1.0:1787827184.782416:0:91485:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.782418:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00020000:00000080:1.0:1787827184.782420:0:91485:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.782422:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00020000:00000080:1.0:1787827184.782424:0:91485:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.782425:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00020000:00000080:1.0:1787827184.782427:0:91485:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.782429:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00020000:00000080:1.0:1787827184.782431:0:91485:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.782433:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00020000:00000080:1.0:1787827184.782435:0:91485:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.782437:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00020000:00000080:1.0:1787827184.782438:0:91485:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.782440:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00020000:00000080:1.0:1787827184.782442:0:91485:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.782444:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00020000:00000080:1.0:1787827184.782446:0:91485:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.782447:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00020000:00000080:1.0:1787827184.782449:0:91485:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.782451:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00020000:00000080:1.0:1787827184.782453:0:91485:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.782455:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00020000:00000080:1.0:1787827184.782456:0:91485:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.782458:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00020000:00000080:1.0:1787827184.782460:0:91485:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.782462:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00020000:00000080:1.0:1787827184.782463:0:91485:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.782465:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00020000:00000080:1.0:1787827184.782467:0:91485:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.782469:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00020000:00000080:1.0:1787827184.782470:0:91485:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.782472:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00020000:00000080:1.0:1787827184.782474:0:91485:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.782476:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00020000:00000080:1.0:1787827184.782478:0:91485:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.782480:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00020000:00000080:1.0:1787827184.782482:0:91485:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200080:1.0:1787827184.782483:0:91485:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7fff612986f0 00020000:00000080:1.0:1787827184.782485:0:91485:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865d800 uarg=0000000000000000 00000080:00200008:1.0:1787827184.782492:0:91485:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 100000 00010000:00010000:2.0:1787827185.460867:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827186.485802:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:3.0:1787827187.275649:0:92140:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == replay-single test 44b: race in target handle connect ========================================================== 06:39:46 (1787827186) 00010000:00010000:2.0:1787827187.508865:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000080:1.0:1787827187.863625:0:92176:0:(class_obd.c:336:class_handle_ioctl()) obdclass: cmd=40086685 len=0 uarg=00007fff3c2da0b0 00000002:00000080:1.0:1787827187.863643:0:92176:0:(mdc_request.c:2317:mdc_iocontrol()) lustre-MDT0000-mdc-ffff985602bc9800: cmd=40086685 len=576 karg=ffff98560865d000 uarg=0000000000000000 00000100:00080000:1.0:1787827187.863651:0:92176:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from FULL to DISCONN 00000100:02000400:1.0:1787827187.863658:0:92176:0:(import.c:182:ptlrpc_set_import_discon()) lustre-MDT0000-mdc-ffff985602bc9800: Connection to lustre-MDT0000 (at 192.168.203.114@tcp) was lost; in progress operations using this service will wait for recovery to complete 00000100:00080000:1.0:1787827187.863664:0:92176:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827187.863668:0:92176:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3330 00000100:00080000:1.0:1787827187.863712:0:92176:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787827187.863723:0:92176:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827187.863754:0:92176:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2d00 x1874669228216832/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827187.863773:0:92176:0:(recover.c:337:ptlrpc_recover_import()) lustre-MDT0000_UUID: recovery started, waiting 20 seconds 00000100:00100000:0.0:1787827187.863800:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228216832:192.168.203.114@tcp:38:lctl.0 00010000:00010000:0.0:1787827188.532813:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827188.980888:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827188.980896:0:54101:0:(pinger.c:247:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:0.0:1787827188.980915:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2383:1874669228216960:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827188.980968:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669228217088:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827188.981024:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669228217344:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787827188.981035:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669228217216:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827188.982439:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669228217088:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787827188.982442:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669228217216:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827188.982457:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669228217344:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787827188.982462:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2383:1874669228216960:192.168.203.114@tcp:400:kworker.0 00010000:00010000:3.0:1787827189.556905:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827190.004822:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827190.004826:0:54101:0:(pinger.c:247:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1787827190.004862:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228217600:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787827190.004863:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2385:1874669228217472:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787827190.004931:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228217856:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787827190.004939:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669228217728:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787827190.006423:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228217600:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787827190.006444:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228217856:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787827190.006499:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2385:1874669228217472:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787827190.006519:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669228217728:192.168.203.114@tcp:400:kworker.0 00010000:00010000:2.0:1787827190.580915:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827191.604875:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827192.628855:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827193.652786:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827194.676856:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827195.124902:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827195.124907:0:38:0:(pinger.c:247:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1787827195.124941:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228218112:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787827195.124942:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2385:1874669228217984:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787827195.125029:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228218368:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787827195.125032:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669228218240:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787827195.126350:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2385:1874669228217984:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787827195.126393:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228218112:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787827195.126404:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228218368:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787827195.126438:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669228218240:192.168.203.114@tcp:400:kworker.0 00010000:00010000:2.0:1787827195.700819:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827196.724869:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827197.172865:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787827187/real 1787827187] req@ffff9856085a2d00 x1874669228216832/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827197 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'lctl.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827197.172907:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2d00 x1874669228216832/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827197 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'lctl.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827197.172923:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827197.172948:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827197.172958:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228216832:192.168.203.114@tcp:38:lctl.0 00000100:00080000:3.0:1787827197.173097:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827197.173104:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827197.173109:0:38:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827197.173119:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3338 00000100:00080000:3.0:1787827197.173134:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 5s 00000100:00080000:3.0:1787827197.173144:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787827197.173155:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2385:1874669228218496:192.168.203.114@tcp:400:kworker.0 00000008:00100000:3.0:1787827197.173157:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827197.173178:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f830c0 x1874669228218624/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827197.173263:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669228218880:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787827197.173278:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228218752:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787827197.173303:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228218624:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787827197.173329:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228219008:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787827197.174657:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2385:1874669228218496:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787827197.174687:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228218752:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787827197.174734:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228219008:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787827197.174746:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669228218880:192.168.203.114@tcp:400:kworker.0 00000100:00080000:0.0:1787827197.176991:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827197.177008:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-114) 00000100:00100000:0.0:1787827197.177018:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228218624:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787827197.748902:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827198.772831:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827199.796926:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827200.820907:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827201.844880:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827202.292885:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827202.292891:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827202.292896:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827202.292907:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3347 00000100:00100000:1.0:1787827202.292912:0:2384:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00080000:0.0:1787827202.292919:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 10s 00000100:00080000:0.0:1787827202.292928:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787827202.292936:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:3c946890-15ca-497a-b838-fe1cba61f4dd:2384:1874669228219136:192.168.203.114@tcp:400:kworker.0 00000008:00100000:0.0:1787827202.292940:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827202.292957:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6d00 x1874669228219264/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827202.293040:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228219264:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787827202.293042:0:2383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669228219520:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827202.293055:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669228219392:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827202.293117:0:2384:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669228219648:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827202.294387:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:3c946890-15ca-497a-b838-fe1cba61f4dd:2384:1874669228219136:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787827202.294410:0:2383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7b4e8535-9db7-485d-abf6-bda966b63320:2383:1874669228219520:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827202.294420:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669228219648:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827202.294548:0:2384:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7b4e8535-9db7-485d-abf6-bda966b63320:2384:1874669228219392:192.168.203.114@tcp:400:kworker.0 00000100:00080000:3.0:1787827202.297298:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827202.297305:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-114) 00000100:00100000:3.0:1787827202.297316:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228219264:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787827202.868881:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827203.892818:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827204.916937:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827205.940885:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827206.964893:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827207.412879:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827207.412886:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827207.412889:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827207.412899:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3353 00000100:00080000:3.0:1787827207.412910:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 15s 00000100:00080000:3.0:1787827207.412918:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787827207.412929:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2385:1874669228219776:192.168.203.114@tcp:400:kworker.0 00000008:00100000:3.0:1787827207.412930:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827207.412951:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81a40 x1874669228219904/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827207.412998:0:54101:0:(import.c:65:import_set_state_nolock()) ffff9856110b0000 lustre-OST0000_UUID: changing import state from FULL to CONNECTING 00000100:00080000:3.0:1787827207.413001:0:54101:0:(import.c:2101:ptlrpc_disconnect_and_idle_import()) lustre-OST0000-osc-ffff985602bc9800: disconnect after 23s idle 00000100:00080000:3.0:1787827207.413011:0:54101:0:(import.c:65:import_set_state_nolock()) ffff9856110b1000 lustre-OST0001_UUID: changing import state from FULL to CONNECTING 00000100:00080000:3.0:1787827207.413015:0:54101:0:(import.c:2101:ptlrpc_disconnect_and_idle_import()) lustre-OST0001-osc-ffff985602bc9800: disconnect after 23s idle 00000100:00100000:0.0:1787827207.413026:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228219904:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827207.413033:0:2385:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1787827207.413045:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669228220032:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787827207.413075:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228220160:192.168.203.114@tcp:9:kworker.0 00000100:00100000:0.0:1787827207.413113:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228220288:192.168.203.114@tcp:9:kworker.0 00000100:00100000:3.0:1787827207.414359:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669228220032:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787827207.414409:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2385:1874669228219776:192.168.203.114@tcp:400:kworker.0 00000100:00080000:0.0:1787827207.414462:0:2382:0:(import.c:2011:ptlrpc_disconnect_idle_interpret()) @@@ inflight=1, refcount=3: rc = 0 req@ffff985610f812c0 x1874669228220288/t0(0) o9->lustre-OST0001-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827217 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827207.414473:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9856110b1000 lustre-OST0001_UUID: changing import state from CONNECTING to IDLE 00000100:00100000:0.0:1787827207.414477:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228220288:192.168.203.114@tcp:9:kworker.0 00000100:00080000:0.0:1787827207.414492:0:2382:0:(import.c:2011:ptlrpc_disconnect_idle_interpret()) @@@ inflight=1, refcount=3: rc = 0 req@ffff985610f82d00 x1874669228220160/t0(0) o9->lustre-OST0000-osc-ffff985602bc9800@192.168.203.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787827217 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827207.414500:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9856110b0000 lustre-OST0000_UUID: changing import state from CONNECTING to IDLE 00000100:00100000:0.0:1787827207.414506:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228220160:192.168.203.114@tcp:9:kworker.0 00000100:00080000:0.0:1787827207.416680:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827207.416686:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-114) 00000100:00100000:0.0:1787827207.416714:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228219904:192.168.203.114@tcp:38:kworker.0 00000100:00080000:1.0:1787827207.868781:0:92176:0:(recover.c:348:ptlrpc_recover_import()) lustre-MDT0000_UUID: recovery finished DISCONN, rc = -110 00000080:00200008:1.0:1787827207.873074:0:92181:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:1.0:1787827207.873086:0:92181:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787827207.873110:0:92181:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827207.873130:0:92181:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 36 lru_type: 1 00010000:00010000:1.0:1787827207.873146:0:92181:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 36 lru_type: 1 00000080:00200000:1.0:1787827207.873159:0:92181:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827207.873166:0:92181:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787827207.873170:0:92181:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee56d 00000080:00010000:1.0:1787827207.873178:0:92181:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee56d from it ffffab3ac0dd3cb8 00010000:00010000:1.0:1787827207.873181:0:92181:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 36 lru_type: 1 00010000:00010000:1.0:1787827207.873189:0:92181:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 37 lru_type: 1 00000080:00200000:1.0:1787827207.873212:0:92181:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787827207.873216:0:92181:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827207.873222:0:92181:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 37 lru_type: 1 00010000:00010000:1.0:1787827207.873232:0:92181:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 37 lru_type: 1 00000080:00200000:1.0:1787827207.873239:0:92181:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827207.873244:0:92181:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787827207.873246:0:92181:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee56d 00000080:00010000:1.0:1787827207.873251:0:92181:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee56d from it ffffab3ac0dd3bb0 00010000:00010000:1.0:1787827207.873255:0:92181:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 37 lru_type: 1 00010000:00010000:1.0:1787827207.873265:0:92181:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 38 lru_type: 1 00000080:00200000:1.0:1787827207.873275:0:92181:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 44 00000080:00200008:1.0:1787827207.873283:0:92181:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 100400 00000080:00200008:1.0:1787827207.873296:0:92181:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 100000 00000080:00200008:1.0:1787827207.873963:0:92181:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:1.0:1787827207.873968:0:92181:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787827207.873975:0:92181:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827207.873981:0:92181:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 38 lru_type: 1 00010000:00010000:1.0:1787827207.873988:0:92181:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 38 lru_type: 1 00000080:00200000:1.0:1787827207.873994:0:92181:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827207.873997:0:92181:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787827207.873999:0:92181:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee56d 00000080:00010000:1.0:1787827207.874002:0:92181:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee56d from it ffffab3ac0dd3cb8 00010000:00010000:1.0:1787827207.874005:0:92181:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 38 lru_type: 1 00010000:00010000:1.0:1787827207.874011:0:92181:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 39 lru_type: 1 00000080:00200008:1.0:1787827207.874028:0:92181:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:1.0:1787827207.874031:0:92181:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787827207.874034:0:92181:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827207.874038:0:92181:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 39 lru_type: 1 00010000:00010000:1.0:1787827207.874045:0:92181:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 39 lru_type: 1 00000080:00200000:1.0:1787827207.874049:0:92181:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827207.874052:0:92181:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787827207.874053:0:92181:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee56d 00000080:00010000:1.0:1787827207.874055:0:92181:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee56d from it ffffab3ac0dd3cb8 00010000:00010000:1.0:1787827207.874057:0:92181:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 39 lru_type: 1 00010000:00010000:1.0:1787827207.874062:0:92181:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 40 lru_type: 1 00000080:00200000:1.0:1787827207.874076:0:92181:0:(llite_lib.c:2903:ll_statfs()) VFS Op:sb=lustre (ffff985602bc9800) / proj=0 00000080:00000004:1.0:1787827207.874081:0:92181:0:(obd_class.h:1091:obd_statfs()) lustre-clilmv-ffff985602bc9800: age 3159, max_age 3357 00800000:00000004:1.0:1787827207.874091:0:92181:0:(obd_class.h:1091:obd_statfs()) lustre-MDT0000-mdc-ffff985602bc9800: age 3335, max_age 3357 00000100:00100000:1.0:1787827207.874129:0:92181:0:(client.c:2715:ptlrpc_set_wait()) set ffff985610d3e300 going to sleep for 0 seconds 00000100:00100000:1.0:1787827207.874141:0:92181:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0780 pname:cluuid:pid:xid:nid:opc:job df:7b4e8535-9db7-485d-abf6-bda966b63320:92181:1874669228220416:192.168.203.114@tcp:41:df.0 00800000:00020000:1.0:1787827207.874155:0:92181:0:(lmv_obd.c:1471:lmv_statfs()) lustre-MDT0000-mdc-ffff985602bc9800: can't stat MDS #0: rc = -114 00800000:00000004:1.0:1787827207.877108:0:92181:0:(obd_class.h:1091:obd_statfs()) lustre-MDT0001-mdc-ffff985602bc9800: age 3335, max_age 3357 00000100:00100000:1.0:1787827207.877129:0:92181:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0780 pname:cluuid:pid:xid:nid:opc:job df:7b4e8535-9db7-485d-abf6-bda966b63320:92181:1874669228220544:192.168.203.114@tcp:41:df.0 00000100:00100000:1.0:1787827207.877219:0:92181:0:(client.c:2715:ptlrpc_set_wait()) set ffff985610d3e300 going to sleep for 12 seconds 00000100:00100000:1.0:1787827207.878420:0:92181:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0780 pname:cluuid:pid:xid:nid:opc:job df:7b4e8535-9db7-485d-abf6-bda966b63320:92181:1874669228220544:192.168.203.114@tcp:41:df.0 00800000:00000004:1.0:1787827207.878432:0:92181:0:(obd_class.h:1117:obd_statfs()) lustre-MDT0001-mdc-ffff985602bc9800: update ffff98560952b7f0 cache blocks 1802719/1916558 objects 523462/524037 00000080:00000004:1.0:1787827207.878436:0:92181:0:(obd_class.h:1117:obd_statfs()) lustre-clilmv-ffff985602bc9800: update ffff985606a31e20 cache blocks 1802719/1916558 objects 523462/524037 00000080:00000004:1.0:1787827207.878440:0:92181:0:(llite_lib.c:2755:ll_statfs_internal()) MDC blocks 1802719/1916558 objects 523462/524037 00010000:00010000:2.0:1787827207.988920:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000080:3.0:1787827208.654298:0:92199:0:(class_obd.c:336:class_handle_ioctl()) obdclass: cmd=40086685 len=0 uarg=00007ffed05a9950 00000002:00000080:3.0:1787827208.654314:0:92199:0:(mdc_request.c:2317:mdc_iocontrol()) lustre-MDT0000-mdc-ffff985602bc9800: cmd=40086685 len=576 karg=ffff985608491000 uarg=0000000000000000 00000100:00080000:3.0:1787827208.654320:0:92199:0:(import.c:214:ptlrpc_set_import_discon()) mdc: import ffff985611182000 already not connected (conn 313, was 0): DISCONN 00000100:00080000:3.0:1787827208.654325:0:92199:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827208.654333:0:92199:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3358 00000100:00080000:3.0:1787827208.654343:0:92199:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827208.654351:0:92199:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827208.654363:0:92199:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827208.654390:0:92199:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80f00 x1874669228220672/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827208.654411:0:92199:0:(recover.c:337:ptlrpc_recover_import()) lustre-MDT0000_UUID: recovery started, waiting 20 seconds 00000100:00100000:0.0:1787827208.654439:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228220672:192.168.203.114@tcp:38:lctl.0 00000100:00080000:0.0:1787827208.655564:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827208.655570:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-114) 00000100:00100000:0.0:1787827208.655580:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228220672:192.168.203.114@tcp:38:lctl.0 00010000:00010000:0.0:1787827209.012851:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827210.036856:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827211.060847:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827212.084866:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827212.532848:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827212.532854:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827212.532857:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827212.532865:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3359 00000100:00080000:3.0:1787827212.532873:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827212.532879:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827212.532887:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827212.532888:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f821c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2385:1874669228220800:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787827212.532905:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81e00 x1874669228220928/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827212.532957:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228221056:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787827212.532974:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228220928:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827212.534141:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228221056:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787827212.534176:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f821c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2385:1874669228220800:192.168.203.114@tcp:400:kworker.0 00000100:00080000:0.0:1787827212.538213:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827212.538219:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-114) 00000100:00100000:0.0:1787827212.538230:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228220928:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787827213.108856:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827214.132814:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827215.156856:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827216.180859:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827217.204823:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827217.652844:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827217.652851:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827217.652854:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827217.652864:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3363 00000100:00080000:3.0:1787827217.652874:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827217.652881:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787827217.652891:0:2386:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000008:00100000:3.0:1787827217.652892:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827217.652911:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83480 x1874669228221312/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827217.652913:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:3c946890-15ca-497a-b838-fe1cba61f4dd:2386:1874669228221184:192.168.203.114@tcp:400:kworker.0 00000100:00100000:0.0:1787827217.652979:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228221312:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787827217.652984:0:2385:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1787827217.652998:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669228221440:192.168.203.114@tcp:400:kworker.0 00010000:00010000:2.0:1787827218.228852:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827219.252787:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827220.276864:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827221.300827:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827222.324863:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827222.772882:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827222.772890:0:38:0:(pinger.c:247:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1787827222.772917:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2385:1874669228221568:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787827222.772935:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228221696:192.168.203.114@tcp:400:kworker.0 00010000:00010000:2.0:1787827223.348817:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827224.372831:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827225.396913:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827226.420872:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827227.444873:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827227.892851:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827227.892856:0:54101:0:(pinger.c:247:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1787827227.892888:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228221952:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787827227.892901:0:2385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2385:1874669228221824:192.168.203.114@tcp:400:kworker.0 00010000:00010000:2.0:1787827228.468898:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827228.916819:0:92199:0:(recover.c:348:ptlrpc_recover_import()) lustre-MDT0000_UUID: recovery finished CONNECTING, rc = -110 00000080:00200008:3.0:1787827228.920608:0:92203:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:3.0:1787827228.920619:0:92203:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:3.0:1787827228.920641:0:92203:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787827228.920660:0:92203:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 40 lru_type: 1 00010000:00010000:3.0:1787827228.920677:0:92203:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 40 lru_type: 1 00000080:00200000:3.0:1787827228.920686:0:92203:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787827228.920712:0:92203:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:3.0:1787827228.920716:0:92203:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee56d 00000080:00010000:3.0:1787827228.920723:0:92203:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee56d from it ffffab3ac151bcb8 00010000:00010000:3.0:1787827228.920726:0:92203:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 40 lru_type: 1 00010000:00010000:3.0:1787827228.920734:0:92203:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 41 lru_type: 1 00000080:00200000:3.0:1787827228.920756:0:92203:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:3.0:1787827228.920761:0:92203:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787827228.920767:0:92203:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 41 lru_type: 1 00010000:00010000:3.0:1787827228.920773:0:92203:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 41 lru_type: 1 00000080:00200000:3.0:1787827228.920778:0:92203:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787827228.920781:0:92203:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:3.0:1787827228.920782:0:92203:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee56d 00000080:00010000:3.0:1787827228.920784:0:92203:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee56d from it ffffab3ac151bbb0 00010000:00010000:3.0:1787827228.920787:0:92203:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 41 lru_type: 1 00010000:00010000:3.0:1787827228.920792:0:92203:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 42 lru_type: 1 00000080:00200000:3.0:1787827228.920797:0:92203:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 44 00000080:00200008:3.0:1787827228.920802:0:92203:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 100400 00000080:00200008:3.0:1787827228.920811:0:92203:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 100000 00000080:00200008:3.0:1787827228.921357:0:92203:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:3.0:1787827228.921361:0:92203:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:3.0:1787827228.921368:0:92203:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787827228.921374:0:92203:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 42 lru_type: 1 00010000:00010000:3.0:1787827228.921381:0:92203:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 42 lru_type: 1 00000080:00200000:3.0:1787827228.921390:0:92203:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787827228.921393:0:92203:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:3.0:1787827228.921410:0:92203:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee56d 00000080:00010000:3.0:1787827228.921414:0:92203:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee56d from it ffffab3ac151bcb8 00010000:00010000:3.0:1787827228.921417:0:92203:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 42 lru_type: 1 00010000:00010000:3.0:1787827228.921422:0:92203:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 43 lru_type: 1 00000080:00200008:3.0:1787827228.921441:0:92203:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:3.0:1787827228.921444:0:92203:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:3.0:1787827228.921448:0:92203:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787827228.921453:0:92203:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 43 lru_type: 1 00010000:00010000:3.0:1787827228.921458:0:92203:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 43 lru_type: 1 00000080:00200000:3.0:1787827228.921463:0:92203:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787827228.921465:0:92203:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:3.0:1787827228.921467:0:92203:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee56d 00000080:00010000:3.0:1787827228.921469:0:92203:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee56d from it ffffab3ac151bcb8 00010000:00010000:3.0:1787827228.921471:0:92203:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 43 lru_type: 1 00010000:00010000:3.0:1787827228.921480:0:92203:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 44 lru_type: 1 00000080:00200000:3.0:1787827228.921493:0:92203:0:(llite_lib.c:2903:ll_statfs()) VFS Op:sb=lustre (ffff985602bc9800) / proj=0 00000080:00000004:3.0:1787827228.921496:0:92203:0:(obd_class.h:1091:obd_statfs()) lustre-clilmv-ffff985602bc9800: age 3358, max_age 3378 00800000:00000004:3.0:1787827228.921505:0:92203:0:(obd_class.h:1091:obd_statfs()) lustre-MDT0000-mdc-ffff985602bc9800: age 3335, max_age 3378 00000100:00100000:3.0:1787827228.921532:0:92203:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5b6c0 going to sleep for 0 seconds 00000100:00100000:3.0:1787827228.921540:0:92203:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f821c0 pname:cluuid:pid:xid:nid:opc:job df:7b4e8535-9db7-485d-abf6-bda966b63320:92203:1874669228222080:192.168.203.114@tcp:41:df.0 00800000:00020000:3.0:1787827228.921551:0:92203:0:(lmv_obd.c:1471:lmv_statfs()) lustre-MDT0000-mdc-ffff985602bc9800: can't stat MDS #0: rc = -114 00800000:00000004:3.0:1787827228.924836:0:92203:0:(obd_class.h:1091:obd_statfs()) lustre-MDT0001-mdc-ffff985602bc9800: age 3358, max_age 3378 00000100:00100000:3.0:1787827228.924873:0:92203:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f821c0 pname:cluuid:pid:xid:nid:opc:job df:7b4e8535-9db7-485d-abf6-bda966b63320:92203:1874669228222208:192.168.203.114@tcp:41:df.0 00000100:00100000:3.0:1787827228.924959:0:92203:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5b6c0 going to sleep for 12 seconds 00010000:00010000:0.0:1787827229.492805:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827230.516879:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827231.540889:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827232.564859:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827233.012862:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000400:3.0:1787827233.012862:0:2386:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787827217/real 1787827217] req@ffff985610f82940 x1874669228221184/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 1 dl 1787827233 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827233.012875:0:54101:0:(pinger.c:247:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1787827233.012895:0:2386:0:(client.c:2547:ptlrpc_expire_one_request()) @@@ TIMEOUT-NORESEND: req@ffff985610f82940 x1874669228221184/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 1 dl 1787827233 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827233.012903:0:2386:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from FULL to DISCONN 00000100:02020000:3.0:1787827233.012907:0:2386:0:(import.c:186:ptlrpc_set_import_discon()) MGC192.168.203.114@tcp: Connection to MGS (at 192.168.203.114@tcp) was lost; in progress operations using this service will fail 00000100:00080000:3.0:1787827233.017501:0:2386:0:(import.c:192:ptlrpc_set_import_discon()) import MGS@192.168.203.114@tcp for MGC192.168.203.114@tcp not replayable, auto-deactivating 00000100:00080000:3.0:1787827233.017504:0:2386:0:(import.c:138:ptlrpc_deactivate_import_nolock()) setting import MGS INVALID 00000100:00100000:3.0:1787827233.017510:0:2386:0:(client.c:3589:ptlrpc_abort_inflight()) @@@ inflight req@ffff985610f82940 x1874669228221184/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 1 dl 1787827233 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827233.017520:0:2386:0:(client.c:3589:ptlrpc_abort_inflight()) @@@ inflight req@ffff9856140d8b40 x1874669228221568/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827238 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827233.017527:0:2386:0:(client.c:3589:ptlrpc_abort_inflight()) @@@ inflight req@ffff9856140d8780 x1874669228221824/t0(0) o400->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827243 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 10000000:01000000:3.0:1787827233.017542:0:2386:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808001 00000100:00080000:3.0:1787827233.017544:0:2386:0:(pinger.c:193:ptlrpc_pinger_ir_down()) IR down 10000000:01000000:3.0:1787827233.017545:0:2386:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808002 00000100:00080000:3.0:1787827233.017547:0:2386:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00100000:2.0:1787827233.017566:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:3c946890-15ca-497a-b838-fe1cba61f4dd:0:1874669228222336:192.168.203.114@tcp:400:kworker.0 00000100:00100000:3.0:1787827233.017573:0:2386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228222464:192.168.203.114@tcp:400:kworker.0 00000100:00100000:2.0:1787827233.017596:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2385:1874669228221824:192.168.203.114@tcp:400:kworker.0 00000100:00080000:1.0:1787827233.017607:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827233.017613:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827233.017617:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827233.017629:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3254 00000100:00100000:2.0:1787827233.017634:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:3c946890-15ca-497a-b838-fe1cba61f4dd:2385:1874669228221568:192.168.203.114@tcp:400:kworker.0 00000100:00080000:1.0:1787827233.017646:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787827233.017669:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:3c946890-15ca-497a-b838-fe1cba61f4dd:2386:1874669228221184:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827233.017685:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0000 x1874669228222592/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827233.017741:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827233.017745:0:54101:0:(pinger.c:247:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:1.0:1787827233.017802:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228222592:192.168.203.114@tcp:250:kworker.0 00010000:00010000:2.0:1787827233.588858:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000400:3.0:1787827234.036825:0:2385:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787827217/real 1787827217] req@ffff985610f82580 x1874669228221440/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 1 dl 1787827234 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827234.036854:0:2385:0:(client.c:2547:ptlrpc_expire_one_request()) @@@ TIMEOUT-NORESEND: req@ffff985610f82580 x1874669228221440/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 1 dl 1787827234 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827234.036863:0:2385:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from FULL to DISCONN 00000100:02000400:3.0:1787827234.036866:0:2385:0:(import.c:182:ptlrpc_set_import_discon()) lustre-MDT0001-mdc-ffff985602bc9800: Connection to lustre-MDT0001 (at 192.168.203.114@tcp) was lost; in progress operations using this service will wait for recovery to complete 00000100:00080000:3.0:1787827234.036871:0:2385:0:(import.c:418:ptlrpc_pinger_force()) lustre-MDT0001_UUID: waking up pinger s:DISCONN 00000100:00100000:3.0:1787827234.036905:0:2385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7b4e8535-9db7-485d-abf6-bda966b63320:2385:1874669228221440:192.168.203.114@tcp:400:kworker.0 00000100:00080000:0.0:1787827234.036912:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827234.036919:0:66417:0:(pinger.c:247:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1787827234.036923:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827234.036927:0:66417:0:(pinger.c:247:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1787827234.036931:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827234.036933:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827234.036936:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827234.036942:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 601 00000100:00080000:0.0:1787827234.036954:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787827234.036963:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827234.036991:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5e00 x1874669228222720/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827234.037042:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228222720:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787827234.612783:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827235.636875:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827236.660926:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827237.684869:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827238.068796:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827238.068804:0:66417:0:(pinger.c:247:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1787827238.068806:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827238.068809:0:66417:0:(pinger.c:247:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1787827238.068810:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827238.068813:0:66417:0:(pinger.c:247:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:2.0:1787827238.708889:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827239.092832:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827239.092844:0:66417:0:(pinger.c:247:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1787827239.092847:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827239.092850:0:66417:0:(pinger.c:247:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1787827239.092853:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827239.092857:0:66417:0:(pinger.c:247:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00000400:3.0:1787827239.156795:0:2386:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787827222/real 1787827222] req@ffff9856140dad00 x1874669228221696/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 1 dl 1787827239 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827239.170550:0:2386:0:(client.c:2547:ptlrpc_expire_one_request()) @@@ TIMEOUT-NORESEND: req@ffff9856140dad00 x1874669228221696/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 1 dl 1787827239 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827239.170565:0:2386:0:(import.c:214:ptlrpc_set_import_discon()) mdc: import ffff985611185800 already not connected (conn 2, was 1): CONNECTING 00000100:00100000:3.0:1787827239.170585:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228221696:192.168.203.114@tcp:400:kworker.0 00010000:00010000:0.0:1787827239.732804:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827240.756886:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827241.204757:0:92203:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5b6c0 going to sleep for 1 seconds 00010000:00010000:2.0:1787827241.780862:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827242.228769:0:92203:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5b6c0 going to sleep for 1 seconds 00000100:00100000:1.0:1787827242.228800:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787827217/real 1787827217] req@ffff985610f83480 x1874669228221312/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827242 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827242.228830:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83480 x1874669228221312/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827242 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827242.228847:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827242.228867:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827242.228872:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228221312:192.168.203.114@tcp:38:kworker.0 00000100:00080000:0.0:1787827242.228896:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827242.228903:0:66417:0:(pinger.c:247:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1787827242.228906:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827242.228910:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827242.228914:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827242.228922:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3368 00000100:00080000:0.0:1787827242.228931:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827242.228938:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787827242.228949:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827242.228980:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6580 x1874669228222848/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827242.229001:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827242.229005:0:66417:0:(pinger.c:247:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:1.0:1787827242.229031:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228222848:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787827242.804835:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827243.252754:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787827233/real 1787827233] req@ffff9856085a0000 x1874669228222592/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827243 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827243.252788:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0000 x1874669228222592/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827243 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827243.252808:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00100000:0.0:1787827243.252809:0:92203:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5b6c0 going to sleep for 1 seconds 00000100:00080000:1.0:1787827243.252841:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827243.252847:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228222592:192.168.203.114@tcp:250:kworker.0 00000100:00080000:2.0:1787827243.252864:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827243.252869:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827243.252871:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827243.252877:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3383 00000100:00080000:2.0:1787827243.252886:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 5s 00000100:00080000:2.0:1787827243.252892:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787827243.252918:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8b40 x1874669228222976/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827243.252937:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827243.252940:0:66417:0:(pinger.c:247:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1787827243.252942:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827243.252944:0:66417:0:(pinger.c:247:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:1.0:1787827243.252968:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228222976:192.168.203.114@tcp:250:kworker.0 00010000:00010000:2.0:1787827243.828941:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827244.276758:0:92203:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5b6c0 going to sleep for 1 seconds 00000100:00100000:1.0:1787827244.276760:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787827234/real 1787827234] req@ffff985607df5e00 x1874669228222720/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827244 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827244.276787:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5e00 x1874669228222720/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827244 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827244.276803:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827244.276823:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827244.276826:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228222720:192.168.203.114@tcp:38:kworker.0 00000100:00080000:0.0:1787827244.276856:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827244.276861:0:66417:0:(pinger.c:247:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1787827244.276864:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827244.276867:0:66417:0:(pinger.c:247:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1787827244.276872:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827244.276876:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827244.276879:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827244.276888:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3384 00000100:00080000:0.0:1787827244.276905:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 5s 00000100:00080000:0.0:1787827244.276914:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787827244.276926:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00000400:3.0:1787827244.276933:0:2386:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787827227/real 1787827227] req@ffff9856140db0c0 x1874669228221952/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 1 dl 1787827244 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827244.276953:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df43c0 x1874669228223104/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827244.277008:0:2386:0:(client.c:2547:ptlrpc_expire_one_request()) @@@ TIMEOUT-NORESEND: req@ffff9856140db0c0 x1874669228221952/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 1 dl 1787827244 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827244.277016:0:2386:0:(import.c:214:ptlrpc_set_import_discon()) mdc: import ffff985611185800 already not connected (conn 3, was 1): CONNECTING 00000100:00100000:3.0:1787827244.277025:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228221952:192.168.203.114@tcp:400:kworker.0 00000100:00100000:1.0:1787827244.277031:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228223104:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787827244.852822:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000400:0.0:1787827245.300790:0:92203:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787827228/real 1787827228] req@ffff985610f821c0 x1874669228222208/t0(0) o41->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 440/368 e 0 to 1 dl 1787827245 ref 2 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'df.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827245.300826:0:92203:0:(client.c:2547:ptlrpc_expire_one_request()) @@@ TIMEOUT-NORESEND: req@ffff985610f821c0 x1874669228222208/t0(0) o41->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 440/368 e 0 to 1 dl 1787827245 ref 2 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'df.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827245.300834:0:92203:0:(import.c:214:ptlrpc_set_import_discon()) mdc: import ffff985611185800 already not connected (conn 3, was 1): CONNECTING 00000100:00100000:0.0:1787827245.300838:0:92203:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5b6c0 going to sleep for 0 seconds 00000100:00100000:0.0:1787827245.300844:0:92203:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f821c0 pname:cluuid:pid:xid:nid:opc:job df:7b4e8535-9db7-485d-abf6-bda966b63320:92203:1874669228222208:192.168.203.114@tcp:41:df.0 00800000:00020000:0.0:1787827245.300877:0:92203:0:(lmv_obd.c:1471:lmv_statfs()) lustre-MDT0001-mdc-ffff985602bc9800: can't stat MDS #1: rc = -110 00010000:00010000:3.0:1787827245.876849:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827246.324831:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827246.324838:0:66417:0:(pinger.c:247:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1787827246.324841:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827246.324843:0:66417:0:(pinger.c:247:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1787827246.324845:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827246.324846:0:66417:0:(pinger.c:247:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:2.0:1787827246.900952:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827247.924860:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827248.372791:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827248.372798:0:54101:0:(pinger.c:247:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1787827248.372801:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827248.372804:0:54101:0:(pinger.c:247:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1787827248.372806:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827248.372808:0:54101:0:(pinger.c:247:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:2.0:1787827248.948914:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827249.396790:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827249.396800:0:54101:0:(pinger.c:247:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1787827249.396805:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827249.396808:0:54101:0:(pinger.c:247:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1787827249.396811:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827249.396815:0:54101:0:(pinger.c:247:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:0.0:1787827249.972823:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000400:3.0:1787827250.420740:0:2386:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787827233/real 1787827233] req@ffff9856140d83c0 x1874669228222464/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 1 dl 1787827250 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827250.420768:0:2386:0:(client.c:2547:ptlrpc_expire_one_request()) @@@ TIMEOUT-NORESEND: req@ffff9856140d83c0 x1874669228222464/t0(0) o400->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/224 e 0 to 1 dl 1787827250 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827250.420777:0:2386:0:(import.c:214:ptlrpc_set_import_discon()) mdc: import ffff985611185800 already not connected (conn 3, was 1): CONNECTING 00000100:00100000:3.0:1787827250.420794:0:2386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7b4e8535-9db7-485d-abf6-bda966b63320:2386:1874669228222464:192.168.203.114@tcp:400:kworker.0 00010000:00010000:3.0:1787827250.996893:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827252.020848:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827253.044926:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827254.068858:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827254.516832:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827254.516843:0:54101:0:(pinger.c:247:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1787827254.516848:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827254.516852:0:54101:0:(pinger.c:247:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1787827254.516855:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827254.516862:0:54101:0:(pinger.c:247:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000800:00000100:2.0:1787827254.708811:0:2374:0:(socklnd_cb.c:2254:ksocknal_find_timed_out_conn()) A connection with 12345-192.168.203.114@tcp (192.168.203.114:988) timed out; the network or node may be down. 00010000:00010000:0.0:1787827255.092829:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827256.116848:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827257.140944:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000080:2.0:1787827257.790993:0:92225:0:(class_obd.c:336:class_handle_ioctl()) obdclass: cmd=40086685 len=0 uarg=00007ffedc9885a0 00000002:00000080:2.0:1787827257.791011:0:92225:0:(mdc_request.c:2317:mdc_iocontrol()) lustre-MDT0000-mdc-ffff985602bc9800: cmd=40086685 len=576 karg=ffff9856110f2000 uarg=0000000000000000 00000100:00080000:2.0:1787827257.791018:0:92225:0:(import.c:214:ptlrpc_set_import_discon()) mdc: import ffff985611182000 already not connected (conn 317, was 0): CONNECTING 00000100:00080000:2.0:1787827257.791024:0:92225:0:(recover.c:337:ptlrpc_recover_import()) lustre-MDT0000_UUID: recovery started, waiting 20 seconds 00010000:00010000:1.0:1787827258.164863:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827258.548826:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787827243/real 1787827243] req@ffff9856140d8b40 x1874669228222976/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827258 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827258.548858:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8b40 x1874669228222976/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827258 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827258.548879:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827258.548896:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827258.548901:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228222976:192.168.203.114@tcp:250:kworker.0 00000100:00080000:2.0:1787827258.548924:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827258.548930:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827258.548933:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827258.548938:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3394 00000100:00080000:2.0:1787827258.548949:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 10s 00000100:00080000:2.0:1787827258.548956:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787827258.548982:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8780 x1874669228223232/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827258.549001:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level CONNECTING/4 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827258.549006:0:54101:0:(pinger.c:247:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1787827258.549009:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827258.549012:0:54101:0:(pinger.c:247:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:1.0:1787827258.549031:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228223232:192.168.203.114@tcp:250:kworker.0 00010000:00010000:2.0:1787827259.188861:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827259.572773:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787827244/real 1787827244] req@ffff985607df43c0 x1874669228223104/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827259 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827259.572809:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df43c0 x1874669228223104/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827259 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827259.572833:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827259.572855:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827259.572861:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228223104:192.168.203.114@tcp:38:kworker.0 00000100:00080000:0.0:1787827259.572912:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827259.572919:0:66417:0:(pinger.c:247:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1787827259.572923:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level CONNECTING/4 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827259.572927:0:66417:0:(pinger.c:247:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1787827259.572930:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827259.572933:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827259.572937:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827259.572943:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3395 00000100:00080000:0.0:1787827259.572953:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 10s 00000100:00080000:0.0:1787827259.572960:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787827259.572971:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827259.573000:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5680 x1874669228223360/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827259.573064:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228223360:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787827260.212854:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:3.0:1787827260.853027:0:2371:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1022 -> 192.168.203.114:988 00000400:00000100:3.0:1787827260.853072:0:2371:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.114@tcp at host 192.168.203.114 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:3.0:1787827260.853087:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:3.0:1787827260.853120:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000100:00100000:1.0:1787827260.853221:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827259/real 1787827260] req@ffff985607df5680 x1874669228223360/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827279 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827260.853260:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5680 x1874669228223360/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827279 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827260.853275:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827260.853281:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827260.853286:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228223360:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787827260.853297:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827258/real 1787827260] req@ffff9856140d8780 x1874669228223232/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827278 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827260.853311:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8780 x1874669228223232/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827278 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827260.853319:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827260.853324:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827260.853328:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228223232:192.168.203.114@tcp:250:kworker.0 00010000:00010000:3.0:1787827261.236860:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827262.260835:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827263.284865:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827263.604799:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827263.604810:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827263.604816:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827263.604827:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3409 00000100:00080000:2.0:1787827263.604841:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 15s 00000100:00080000:2.0:1787827263.604848:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787827263.604889:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d9a40 x1874669228223488/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827263.604914:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level CONNECTING/4 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827263.604917:0:66417:0:(pinger.c:247:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1787827263.604920:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827263.604922:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827263.604924:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827263.604926:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3410 00000100:00080000:2.0:1787827263.604944:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 15s 00000100:00080000:2.0:1787827263.604948:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787827263.604957:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827263.604970:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8780 x1874669228223616/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827263.605023:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228223488:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787827263.605129:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228223616:192.168.203.114@tcp:38:kworker.0 00000400:00000100:3.0:1787827263.925172:0:2372:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1022 -> 192.168.203.114:988 00000400:00000100:3.0:1787827263.925247:0:2372:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.114@tcp at host 192.168.203.114 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:3.0:1787827263.925267:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:3.0:1787827263.925311:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000100:00100000:1.0:1787827263.925361:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827263/real 1787827263] req@ffff9856140d9a40 x1874669228223488/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827288 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827263.925399:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d9a40 x1874669228223488/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827288 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827263.925411:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827263.925416:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827263.925421:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228223488:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787827263.925445:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827263/real 1787827263] req@ffff9856140d8780 x1874669228223616/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827288 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827263.925453:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8780 x1874669228223616/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827288 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827263.925458:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827263.925461:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827263.925463:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228223616:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787827264.308910:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827265.332869:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827266.356907:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827267.380868:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827267.700801:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787827242/real 1787827242] req@ffff985607df6580 x1874669228222848/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827267 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827267.700830:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6580 x1874669228222848/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827267 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827267.700847:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827267.700860:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827267.700869:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228222848:192.168.203.114@tcp:38:kworker.0 00000100:00080000:0.0:1787827267.700884:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827267.700891:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827267.700895:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827267.700903:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3414 00000100:00080000:0.0:1787827267.700914:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827267.700922:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787827267.700955:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5a40 x1874669228223744/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827267.700981:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827267.700985:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827267.700987:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827267.700992:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3392 00000100:00080000:0.0:1787827267.700995:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827267.701000:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787827267.701010:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827267.701022:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228223744:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787827267.701028:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df70c0 x1874669228223872/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827267.701042:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827267.701046:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827267.701048:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827267.701053:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3414 00000100:00080000:0.0:1787827267.701067:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827267.701072:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787827267.701077:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827267.701092:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6580 x1874669228224000/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827267.701099:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228223872:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787827267.701150:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228224000:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787827268.404836:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827269.428910:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827270.452844:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:3.0:1787827270.772913:0:2373:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1022 -> 192.168.203.114:988 00000400:00000100:3.0:1787827270.772946:0:2373:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.114@tcp at host 192.168.203.114 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:3.0:1787827270.772959:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:3.0:1787827270.772987:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:3.0:1787827270.772996:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000100:00100000:1.0:1787827270.773034:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827267/real 1787827270] req@ffff985607df6580 x1874669228224000/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827292 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827270.773061:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6580 x1874669228224000/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827292 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827270.773074:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827270.773079:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827270.773084:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228224000:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787827270.773093:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827267/real 1787827270] req@ffff985607df70c0 x1874669228223872/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827292 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827270.773119:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df70c0 x1874669228223872/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827292 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827270.773128:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827270.773133:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827270.773147:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228223872:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787827270.773154:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827267/real 1787827270] req@ffff985607df5a40 x1874669228223744/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827292 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827270.773168:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5a40 x1874669228223744/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827292 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827270.773186:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827270.773192:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827270.773196:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228223744:192.168.203.114@tcp:250:kworker.0 00010000:00010000:3.0:1787827271.476841:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827272.500798:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827272.948815:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827272.948824:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827272.948827:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827272.948836:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3418 00000100:00080000:2.0:1787827272.948846:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827272.948852:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787827272.948884:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db480 x1874669228224128/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827272.948906:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827272.948909:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827272.948911:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827272.948913:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3418 00000100:00080000:2.0:1787827272.948916:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827272.948918:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787827272.948926:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827272.948934:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d9680 x1874669228224256/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827272.948941:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827272.948944:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827272.948945:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827272.948948:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3418 00000100:00080000:2.0:1787827272.948950:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827272.948952:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787827272.948956:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827272.948958:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228224128:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827272.948965:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8b40 x1874669228224384/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827272.949014:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827272.949050:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827272/real 1787827272] req@ffff9856140db480 x1874669228224128/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827297 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827272.949069:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db480 x1874669228224128/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827297 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827272.949080:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827272.949085:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827272.949104:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228224128:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787827272.949112:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228224256:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787827272.949139:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827272.949153:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827272/real 1787827272] req@ffff9856140d9680 x1874669228224256/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827297 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827272.949167:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d9680 x1874669228224256/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827297 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827272.949176:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827272.949180:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827272.949194:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228224256:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787827272.949226:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228224384:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787827272.949251:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827272.949264:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827272/real 1787827272] req@ffff9856140d8b40 x1874669228224384/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827297 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827272.949278:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8b40 x1874669228224384/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827297 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827272.949287:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827272.949290:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827272.949294:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228224384:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787827273.524784:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827274.548950:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827275.572809:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827276.596844:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827277.620842:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827278.068773:0:92225:0:(recover.c:348:ptlrpc_recover_import()) lustre-MDT0000_UUID: recovery finished DISCONN, rc = -110 00000100:00080000:1.0:1787827278.068785:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827278.068790:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827278.068793:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827278.068801:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3423 00000100:00080000:1.0:1787827278.068810:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827278.068816:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787827278.068846:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0000 x1874669228224512/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827278.068867:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827278.068872:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827278.068874:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827278.068878:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3423 00000100:00080000:1.0:1787827278.068882:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827278.068891:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787827278.068902:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827278.068915:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0b40 x1874669228224640/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827278.068916:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228224512:192.168.203.114@tcp:250:kworker.0 00000100:00080000:1.0:1787827278.068926:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827278.068930:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827278.068932:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827278.068936:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3423 00000100:00080000:1.0:1787827278.068940:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827278.068943:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787827278.068949:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827278.068959:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3c00 x1874669228224768/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827278.069008:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228224640:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787827278.069036:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228224768:192.168.203.114@tcp:38:kworker.0 00000080:00200008:0.0:1787827278.073013:0:92229:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:0.0:1787827278.073021:0:92229:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:0.0:1787827278.073040:0:92229:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827278.073068:0:92229:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 44 lru_type: 1 00010000:00010000:0.0:1787827278.073081:0:92229:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 44 lru_type: 1 00000080:00200000:0.0:1787827278.073088:0:92229:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827278.073107:0:92229:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:0.0:1787827278.073111:0:92229:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee56d 00000080:00010000:0.0:1787827278.073119:0:92229:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee56d from it ffffab3ac1cc7cb8 00010000:00010000:0.0:1787827278.073123:0:92229:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 44 lru_type: 1 00010000:00010000:0.0:1787827278.073133:0:92229:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 45 lru_type: 1 00000080:00200000:0.0:1787827278.073161:0:92229:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:0.0:1787827278.073167:0:92229:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827278.073173:0:92229:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 45 lru_type: 1 00010000:00010000:0.0:1787827278.073179:0:92229:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 45 lru_type: 1 00000080:00200000:0.0:1787827278.073183:0:92229:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827278.073187:0:92229:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:0.0:1787827278.073188:0:92229:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee56d 00000080:00010000:0.0:1787827278.073191:0:92229:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee56d from it ffffab3ac1cc7bb0 00010000:00010000:0.0:1787827278.073193:0:92229:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 45 lru_type: 1 00010000:00010000:0.0:1787827278.073198:0:92229:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 46 lru_type: 1 00000080:00200000:0.0:1787827278.073204:0:92229:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 44 00000080:00200008:0.0:1787827278.073209:0:92229:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 100400 00000080:00200008:0.0:1787827278.073219:0:92229:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 100000 00000080:00200008:0.0:1787827278.073797:0:92229:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:0.0:1787827278.073803:0:92229:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:0.0:1787827278.073813:0:92229:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827278.073822:0:92229:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 46 lru_type: 1 00010000:00010000:0.0:1787827278.073832:0:92229:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 46 lru_type: 1 00000080:00200000:0.0:1787827278.073838:0:92229:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827278.073841:0:92229:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:0.0:1787827278.073843:0:92229:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee56d 00000080:00010000:0.0:1787827278.073848:0:92229:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee56d from it ffffab3ac1cc7cb8 00010000:00010000:0.0:1787827278.073852:0:92229:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 46 lru_type: 1 00010000:00010000:0.0:1787827278.073860:0:92229:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 47 lru_type: 1 00000080:00200008:0.0:1787827278.073886:0:92229:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:0.0:1787827278.073891:0:92229:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:0.0:1787827278.073899:0:92229:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827278.073904:0:92229:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 47 lru_type: 1 00010000:00010000:0.0:1787827278.073910:0:92229:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 47 lru_type: 1 00000080:00200000:0.0:1787827278.073919:0:92229:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827278.073921:0:92229:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:0.0:1787827278.073923:0:92229:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee56d 00000080:00010000:0.0:1787827278.073926:0:92229:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee56d from it ffffab3ac1cc7cb8 00010000:00010000:0.0:1787827278.073928:0:92229:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 47 lru_type: 1 00010000:00010000:0.0:1787827278.073933:0:92229:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 48 lru_type: 1 00000080:00200000:0.0:1787827278.073947:0:92229:0:(llite_lib.c:2903:ll_statfs()) VFS Op:sb=lustre (ffff985602bc9800) / proj=0 00000080:00000004:0.0:1787827278.073950:0:92229:0:(obd_class.h:1091:obd_statfs()) lustre-clilmv-ffff985602bc9800: age 3358, max_age 3427 00800000:00000004:0.0:1787827278.073959:0:92229:0:(obd_class.h:1091:obd_statfs()) lustre-MDT0000-mdc-ffff985602bc9800: age 3335, max_age 3427 00000100:00100000:0.0:1787827278.073983:0:92229:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607ed2c00 going to sleep for 0 seconds 00000100:00100000:0.0:1787827278.073990:0:92229:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5680 pname:cluuid:pid:xid:nid:opc:job df:7b4e8535-9db7-485d-abf6-bda966b63320:92229:1874669228224896:192.168.203.114@tcp:41:df.0 00800000:00020000:0.0:1787827278.074000:0:92229:0:(lmv_obd.c:1471:lmv_statfs()) lustre-MDT0000-mdc-ffff985602bc9800: can't stat MDS #0: rc = -110 00800000:00000004:0.0:1787827278.077313:0:92229:0:(obd_class.h:1091:obd_statfs()) lustre-MDT0001-mdc-ffff985602bc9800: age 3358, max_age 3427 00000100:00100000:0.0:1787827278.077334:0:92229:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607ed2c00 going to sleep for 0 seconds 00000100:00100000:0.0:1787827278.077340:0:92229:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5680 pname:cluuid:pid:xid:nid:opc:job df:7b4e8535-9db7-485d-abf6-bda966b63320:92229:1874669228225024:192.168.203.114@tcp:41:df.0 00800000:00020000:0.0:1787827278.077347:0:92229:0:(lmv_obd.c:1471:lmv_statfs()) lustre-MDT0001-mdc-ffff985602bc9800: can't stat MDS #1: rc = -110 00010000:00010000:0.0:1787827278.644845:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827279.668856:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827280.692814:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:1.0:1787827281.140921:0:2370:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1022 -> 192.168.203.114:988 00000400:00000100:1.0:1787827281.140958:0:2370:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.114@tcp at host 192.168.203.114 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1787827281.140972:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:1.0:1787827281.141008:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:1.0:1787827281.141018:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000100:00100000:2.0:1787827281.141057:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827278/real 1787827281] req@ffff9856085a0b40 x1874669228224640/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827303 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827281.141085:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0b40 x1874669228224640/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827303 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827281.141099:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827281.141105:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827281.141110:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228224640:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827281.141118:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827278/real 1787827281] req@ffff9856085a3c00 x1874669228224768/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827303 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827281.141132:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3c00 x1874669228224768/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827303 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827281.141140:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827281.141145:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827281.141149:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228224768:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827281.141156:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827278/real 1787827281] req@ffff9856085a0000 x1874669228224512/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827303 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827281.141176:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0000 x1874669228224512/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827303 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827281.141184:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827281.141189:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827281.141192:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228224512:192.168.203.114@tcp:250:kworker.0 00010000:00010000:3.0:1787827281.716814:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827282.740826:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827283.188809:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827283.188838:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827283.188843:0:38:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827283.188855:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3428 00000100:00080000:3.0:1787827283.188870:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827283.188879:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787827283.188925:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82940 x1874669228225152/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827283.188954:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827283.188957:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827283.188959:0:38:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827283.188964:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3428 00000100:00080000:3.0:1787827283.188969:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827283.188975:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827283.188999:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827283.189042:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228225152:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787827283.189049:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81e00 x1874669228225280/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827283.189064:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827283.189068:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827283.189071:0:38:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827283.189077:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3428 00000100:00080000:3.0:1787827283.189082:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827283.189088:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827283.189095:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000800:00000100:2.0:1787827283.189115:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787827283.189118:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81680 x1874669228225408/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827283.189165:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827283/real 1787827283] req@ffff985610f82940 x1874669228225152/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827308 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827283.189187:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82940 x1874669228225152/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827308 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827283.189199:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827283.189204:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827283.189209:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228225152:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827283.189243:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228225280:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827283.189270:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827283.189284:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827283/real 1787827283] req@ffff985610f81e00 x1874669228225280/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827308 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827283.189298:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81e00 x1874669228225280/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827308 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827283.189305:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827283.189308:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827283.189312:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228225280:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827283.189319:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228225408:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827283.189340:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827283.189352:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827283/real 1787827283] req@ffff985610f81680 x1874669228225408/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827308 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827283.189365:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81680 x1874669228225408/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827308 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827283.189373:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827283.189376:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827283.189379:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228225408:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787827283.764771:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000080:3.0:1787827284.221776:0:92248:0:(class_obd.c:336:class_handle_ioctl()) obdclass: cmd=40086685 len=0 uarg=00007fffbf214ab0 00000002:00000080:3.0:1787827284.221794:0:92248:0:(mdc_request.c:2317:mdc_iocontrol()) lustre-MDT0000-mdc-ffff985602bc9800: cmd=40086685 len=576 karg=ffff985608490c00 uarg=0000000000000000 00000100:00080000:3.0:1787827284.221801:0:92248:0:(import.c:214:ptlrpc_set_import_discon()) mdc: import ffff985611182000 already not connected (conn 321, was 0): DISCONN 00000100:00080000:3.0:1787827284.221808:0:92248:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827284.221816:0:92248:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3433 00000100:00080000:3.0:1787827284.221827:0:92248:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827284.221835:0:92248:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827284.221847:0:92248:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827284.221875:0:92248:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80f00 x1874669228225536/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827284.221899:0:92248:0:(recover.c:337:ptlrpc_recover_import()) lustre-MDT0000_UUID: recovery started, waiting 20 seconds 00000100:00100000:2.0:1787827284.221924:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228225536:192.168.203.114@tcp:38:lctl.0 00000800:00000100:2.0:1787827284.221975:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827284.221995:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827284/real 1787827284] req@ffff985610f80f00 x1874669228225536/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827309 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'lctl.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827284.222007:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80f00 x1874669228225536/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827309 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'lctl.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827284.222028:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827284.222031:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827284.222040:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228225536:192.168.203.114@tcp:38:lctl.0 00010000:00010000:2.0:1787827284.788808:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827285.812833:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827286.837044:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827287.860913:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827288.308763:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827288.308774:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827288.308779:0:38:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827288.308799:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3433 00000100:00080000:3.0:1787827288.308814:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827288.308823:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787827288.308867:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81a40 x1874669228225664/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827288.308893:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827288.308896:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827288.308897:0:38:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827288.308900:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3434 00000100:00080000:3.0:1787827288.308903:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827288.308906:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827288.308916:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827288.308925:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82d00 x1874669228225792/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827288.308934:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827288.308939:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:2.0:1787827288.308941:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228225664:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787827288.308942:0:38:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827288.308948:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3433 00000100:00080000:3.0:1787827288.308953:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827288.308956:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827288.308959:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827288.308970:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f812c0 x1874669228225920/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827288.309070:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228225792:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827288.309105:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228225920:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787827288.884791:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827289.908862:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827290.932785:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:3.0:1787827291.380889:0:2371:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1022 -> 192.168.203.114:988 00000400:00000100:3.0:1787827291.380921:0:2371:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.114@tcp at host 192.168.203.114 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:3.0:1787827291.380935:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:3.0:1787827291.380980:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:3.0:1787827291.380990:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000100:00100000:2.0:1787827291.381016:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827288/real 1787827291] req@ffff985610f82d00 x1874669228225792/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827313 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827291.381042:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82d00 x1874669228225792/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827313 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827291.381053:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827291.381059:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827291.381073:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228225792:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827291.381082:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827288/real 1787827291] req@ffff985610f812c0 x1874669228225920/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827313 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827291.381095:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f812c0 x1874669228225920/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827313 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827291.381106:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827291.381110:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827291.381114:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228225920:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827291.381121:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827288/real 1787827291] req@ffff985610f81a40 x1874669228225664/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827313 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827291.381135:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81a40 x1874669228225664/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827313 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827291.381143:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827291.381147:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827291.381150:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228225664:192.168.203.114@tcp:250:kworker.0 00010000:00010000:3.0:1787827291.956827:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827292.980823:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827293.428911:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827293.428927:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827293.428932:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827293.428960:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3439 00000100:00080000:0.0:1787827293.428992:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827293.429002:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787827293.429046:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df43c0 x1874669228226048/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827293.429075:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827293.429078:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827293.429080:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827293.429083:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3439 00000100:00080000:0.0:1787827293.429086:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827293.429090:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787827293.429102:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827293.429118:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5e00 x1874669228226176/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827293.429130:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228226048:192.168.203.114@tcp:250:kworker.0 00000100:00080000:0.0:1787827293.429134:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827293.429141:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827293.429143:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827293.429149:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3439 00000100:00080000:0.0:1787827293.429154:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827293.429159:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787827293.429165:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827293.429181:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4f00 x1874669228226304/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787827293.429211:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827293.429261:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827293/real 1787827293] req@ffff985607df43c0 x1874669228226048/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827318 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827293.429284:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df43c0 x1874669228226048/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827318 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827293.429297:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827293.429301:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827293.429307:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228226048:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827293.429341:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228226176:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827293.429368:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827293.429385:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827293/real 1787827293] req@ffff985607df5e00 x1874669228226176/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827318 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827293.429398:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5e00 x1874669228226176/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827318 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827293.429409:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827293.429413:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827293.429430:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228226176:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827293.429444:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228226304:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827293.429469:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827293.429487:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827293/real 1787827293] req@ffff985607df4f00 x1874669228226304/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827318 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827293.429501:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4f00 x1874669228226304/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827318 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827293.429510:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827293.429514:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827293.429517:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228226304:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787827294.004833:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827295.028887:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827296.052969:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827297.076894:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827298.100962:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827298.548743:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827298.548751:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827298.548754:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827298.548762:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3444 00000100:00080000:0.0:1787827298.548772:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827298.548780:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787827298.548814:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6d00 x1874669228226432/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827298.548838:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827298.548840:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827298.548842:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827298.548845:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3444 00000100:00080000:0.0:1787827298.548848:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827298.548851:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787827298.548858:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827298.548869:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df61c0 x1874669228226560/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827298.548877:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1787827298.548880:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228226432:192.168.203.114@tcp:250:kworker.0 00000100:00080000:0.0:1787827298.548881:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827298.548885:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827298.548890:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3444 00000100:00080000:0.0:1787827298.548893:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827298.548895:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787827298.548899:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827298.548907:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7480 x1874669228226688/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787827298.548953:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827298.548979:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827298/real 1787827298] req@ffff985607df6d00 x1874669228226432/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827323 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827298.548992:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6d00 x1874669228226432/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827323 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827298.549000:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827298.549012:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827298.549015:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228226432:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827298.549034:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228226560:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827298.549049:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827298.549057:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827298/real 1787827298] req@ffff985607df61c0 x1874669228226560/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827323 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827298.549065:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df61c0 x1874669228226560/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827323 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827298.549071:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827298.549073:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827298.549083:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228226560:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827298.549087:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228226688:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827298.549099:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827298.549108:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827298/real 1787827298] req@ffff985607df7480 x1874669228226688/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827323 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827298.549125:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7480 x1874669228226688/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827323 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827298.549131:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827298.549133:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827298.549135:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228226688:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787827299.124804:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827300.148806:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827301.172814:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827302.196861:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827303.220865:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827303.668808:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827303.668819:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827303.668823:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827303.668833:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3449 00000100:00080000:2.0:1787827303.668844:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827303.668851:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787827303.668900:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140da940 x1874669228226816/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827303.668929:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827303.668933:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827303.668935:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827303.668940:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3449 00000100:00080000:2.0:1787827303.668945:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827303.668950:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787827303.668959:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827303.668961:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228226816:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827303.668975:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db0c0 x1874669228226944/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827303.668988:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827303.668992:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827303.668994:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827303.668998:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3449 00000100:00080000:2.0:1787827303.669002:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827303.669006:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787827303.669011:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827303.669022:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140dad00 x1874669228227072/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787827303.669033:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787827303.669066:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827303/real 1787827303] req@ffff9856140da940 x1874669228226816/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827328 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827303.669080:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140da940 x1874669228226816/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827328 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827303.669088:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827303.669091:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827303.669094:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228226816:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787827303.669114:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228226944:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787827303.669129:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787827303.669138:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827303/real 1787827303] req@ffff9856140db0c0 x1874669228226944/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827328 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827303.669146:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db0c0 x1874669228226944/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827328 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827303.669156:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827303.669158:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827303.669169:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228226944:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787827303.669173:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228227072:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787827303.669185:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787827303.669192:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827303/real 1787827303] req@ffff9856140dad00 x1874669228227072/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827328 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827303.669200:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140dad00 x1874669228227072/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827328 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827303.669207:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827303.669209:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827303.669211:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228227072:192.168.203.114@tcp:38:kworker.0 00000100:00080000:3.0:1787827304.244783:0:92248:0:(recover.c:348:ptlrpc_recover_import()) lustre-MDT0000_UUID: recovery finished DISCONN, rc = -110 00010000:00010000:0.0:1787827304.244823:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200008:3.0:1787827304.248070:0:92252:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:3.0:1787827304.248078:0:92252:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:3.0:1787827304.248094:0:92252:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787827304.248109:0:92252:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 48 lru_type: 1 00010000:00010000:3.0:1787827304.248121:0:92252:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 48 lru_type: 1 00000080:00200000:3.0:1787827304.248128:0:92252:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787827304.248133:0:92252:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:3.0:1787827304.248136:0:92252:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee56d 00000080:00010000:3.0:1787827304.248141:0:92252:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee56d from it ffffab3ac1d07cb8 00010000:00010000:3.0:1787827304.248144:0:92252:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 48 lru_type: 1 00010000:00010000:3.0:1787827304.248151:0:92252:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 49 lru_type: 1 00000080:00200000:3.0:1787827304.248168:0:92252:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:3.0:1787827304.248172:0:92252:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787827304.248176:0:92252:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 49 lru_type: 1 00010000:00010000:3.0:1787827304.248181:0:92252:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 49 lru_type: 1 00000080:00200000:3.0:1787827304.248186:0:92252:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787827304.248188:0:92252:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:3.0:1787827304.248189:0:92252:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee56d 00000080:00010000:3.0:1787827304.248191:0:92252:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee56d from it ffffab3ac1d07bb0 00010000:00010000:3.0:1787827304.248193:0:92252:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 49 lru_type: 1 00010000:00010000:3.0:1787827304.248199:0:92252:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 50 lru_type: 1 00000080:00200000:3.0:1787827304.248204:0:92252:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 44 00000080:00200008:3.0:1787827304.248208:0:92252:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 100400 00000080:00200008:3.0:1787827304.248217:0:92252:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 100000 00000080:00200008:3.0:1787827304.248766:0:92252:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:3.0:1787827304.248771:0:92252:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:3.0:1787827304.248777:0:92252:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787827304.248783:0:92252:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 50 lru_type: 1 00010000:00010000:3.0:1787827304.248790:0:92252:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 50 lru_type: 1 00000080:00200000:3.0:1787827304.248795:0:92252:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787827304.248798:0:92252:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:3.0:1787827304.248800:0:92252:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee56d 00000080:00010000:3.0:1787827304.248803:0:92252:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee56d from it ffffab3ac1d07cb8 00010000:00010000:3.0:1787827304.248805:0:92252:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 50 lru_type: 1 00010000:00010000:3.0:1787827304.248810:0:92252:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 51 lru_type: 1 00000080:00200008:3.0:1787827304.248826:0:92252:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:3.0:1787827304.248830:0:92252:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:3.0:1787827304.248833:0:92252:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787827304.248837:0:92252:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 51 lru_type: 1 00010000:00010000:3.0:1787827304.248842:0:92252:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 51 lru_type: 1 00000080:00200000:3.0:1787827304.248846:0:92252:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787827304.248848:0:92252:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:3.0:1787827304.248849:0:92252:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee56d 00000080:00010000:3.0:1787827304.248851:0:92252:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee56d from it ffffab3ac1d07cb8 00010000:00010000:3.0:1787827304.248853:0:92252:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 51 lru_type: 1 00010000:00010000:3.0:1787827304.248859:0:92252:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 52 lru_type: 1 00000080:00200000:3.0:1787827304.248884:0:92252:0:(llite_lib.c:2903:ll_statfs()) VFS Op:sb=lustre (ffff985602bc9800) / proj=0 00000080:00000004:3.0:1787827304.248888:0:92252:0:(obd_class.h:1091:obd_statfs()) lustre-clilmv-ffff985602bc9800: age 3358, max_age 3454 00800000:00000004:3.0:1787827304.248896:0:92252:0:(obd_class.h:1091:obd_statfs()) lustre-MDT0000-mdc-ffff985602bc9800: age 3335, max_age 3454 00000100:00100000:3.0:1787827304.248924:0:92252:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5b3c0 going to sleep for 0 seconds 00000100:00100000:3.0:1787827304.248933:0:92252:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job df:7b4e8535-9db7-485d-abf6-bda966b63320:92252:1874669228227200:192.168.203.114@tcp:41:df.0 00800000:00020000:3.0:1787827304.248944:0:92252:0:(lmv_obd.c:1471:lmv_statfs()) lustre-MDT0000-mdc-ffff985602bc9800: can't stat MDS #0: rc = -110 00800000:00000004:3.0:1787827304.253843:0:92252:0:(obd_class.h:1091:obd_statfs()) lustre-MDT0001-mdc-ffff985602bc9800: age 3358, max_age 3454 00000100:00100000:3.0:1787827304.253937:0:92252:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5b3c0 going to sleep for 0 seconds 00000100:00100000:3.0:1787827304.253943:0:92252:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job df:7b4e8535-9db7-485d-abf6-bda966b63320:92252:1874669228227328:192.168.203.114@tcp:41:df.0 00800000:00020000:3.0:1787827304.253949:0:92252:0:(lmv_obd.c:1471:lmv_statfs()) lustre-MDT0001-mdc-ffff985602bc9800: can't stat MDS #1: rc = -110 00010000:00010000:2.0:1787827305.268924:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827306.292922:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827307.316908:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827308.340879:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827308.788823:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827308.788838:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827308.788856:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827308.788869:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3454 00000100:00080000:1.0:1787827308.788882:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827308.788890:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787827308.788947:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3840 x1874669228227456/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827308.788988:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827308.788991:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827308.788993:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827308.788997:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3454 00000100:00080000:1.0:1787827308.789001:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827308.789005:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787827308.789015:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827308.789029:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a1680 x1874669228227584/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827308.789041:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228227456:192.168.203.114@tcp:250:kworker.0 00000100:00080000:1.0:1787827308.789042:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827308.789059:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827308.789062:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827308.789067:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3454 00000100:00080000:1.0:1787827308.789069:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827308.789072:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787827308.789076:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827308.789088:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a03c0 x1874669228227712/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827308.789157:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228227584:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787827308.789181:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228227712:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787827309.364830:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827310.388827:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:3.0:1787827310.452987:0:2372:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1022 -> 192.168.203.114:988 00000400:00000100:3.0:1787827310.453030:0:2372:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.114@tcp at host 192.168.203.114 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:3.0:1787827310.453053:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:3.0:1787827310.453086:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:3.0:1787827310.453098:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000100:00100000:0.0:1787827310.453147:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827308/real 1787827310] req@ffff9856085a1680 x1874669228227584/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827333 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827310.453180:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a1680 x1874669228227584/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827333 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827310.453193:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827310.453198:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827310.453203:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228227584:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787827310.453212:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827308/real 1787827310] req@ffff9856085a03c0 x1874669228227712/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827333 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827310.453236:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a03c0 x1874669228227712/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827333 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827310.453245:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827310.453250:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827310.453254:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228227712:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787827310.453261:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827308/real 1787827310] req@ffff9856085a3840 x1874669228227456/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827333 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827310.453290:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3840 x1874669228227456/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827333 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827310.453300:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827310.453304:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827310.453307:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228227456:192.168.203.114@tcp:250:kworker.0 00000020:00000080:1.0:1787827310.461640:0:92271:0:(class_obd.c:336:class_handle_ioctl()) obdclass: cmd=40086685 len=0 uarg=00007ffeed0da410 00000002:00000080:1.0:1787827310.461654:0:92271:0:(mdc_request.c:2317:mdc_iocontrol()) lustre-MDT0000-mdc-ffff985602bc9800: cmd=40086685 len=576 karg=ffff98560865cc00 uarg=0000000000000000 00000100:00080000:1.0:1787827310.461660:0:92271:0:(import.c:214:ptlrpc_set_import_discon()) mdc: import ffff985611182000 already not connected (conn 327, was 0): DISCONN 00000100:00080000:1.0:1787827310.461666:0:92271:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827310.461673:0:92271:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3459 00000100:00080000:1.0:1787827310.461683:0:92271:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827310.461717:0:92271:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787827310.461728:0:92271:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827310.461755:0:92271:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a1e00 x1874669228227840/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827310.461774:0:92271:0:(recover.c:337:ptlrpc_recover_import()) lustre-MDT0000_UUID: recovery started, waiting 20 seconds 00000100:00100000:0.0:1787827310.461827:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a1e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228227840:192.168.203.114@tcp:38:lctl.0 00000800:00000100:0.0:1787827310.461904:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787827310.461932:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827310/real 1787827310] req@ffff9856085a1e00 x1874669228227840/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827335 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'lctl.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827310.461952:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a1e00 x1874669228227840/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827335 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'lctl.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827310.461962:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827310.461967:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827310.461979:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a1e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228227840:192.168.203.114@tcp:38:lctl.0 00010000:00010000:0.0:1787827311.412810:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827312.436805:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827313.460969:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827313.908870:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827313.908880:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827313.908884:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827313.908895:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3459 00000100:00080000:3.0:1787827313.908907:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827313.908914:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787827313.908955:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f830c0 x1874669228227968/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827313.908982:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827313.908986:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827313.908988:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827313.908992:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3461 00000100:00080000:3.0:1787827313.908996:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827313.909000:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827313.909012:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827313.909024:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80b40 x1874669228228096/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827313.909034:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228227968:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787827313.909035:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827313.909040:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827313.909041:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827313.909045:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3459 00000100:00080000:3.0:1787827313.909048:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827313.909052:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827313.909057:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827313.909066:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80780 x1874669228228224/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787827313.909115:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787827313.909156:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827313/real 1787827313] req@ffff985610f830c0 x1874669228227968/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827338 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827313.909176:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f830c0 x1874669228227968/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827338 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827313.909188:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827313.909193:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827313.909198:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228227968:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787827313.909225:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228228096:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787827313.909245:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787827313.909255:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827313/real 1787827313] req@ffff985610f80b40 x1874669228228096/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827338 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827313.909268:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80b40 x1874669228228096/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827338 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827313.909274:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827313.909276:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827313.909288:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228228096:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787827313.909292:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228228224:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787827313.909312:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787827313.909321:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827313/real 1787827313] req@ffff985610f80780 x1874669228228224/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827338 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827313.909330:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80780 x1874669228228224/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827338 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827313.909336:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827313.909339:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827313.909341:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228228224:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787827314.484766:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827315.508858:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827316.532867:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827317.556899:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827318.580907:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827319.028783:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827319.028791:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827319.028794:0:38:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827319.028802:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3464 00000100:00080000:0.0:1787827319.028810:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827319.028817:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787827319.028853:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7840 x1874669228228352/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827319.028882:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827319.028884:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827319.028886:0:38:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827319.028889:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3464 00000100:00080000:0.0:1787827319.028892:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827319.028895:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787827319.028903:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827319.028913:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4780 x1874669228228480/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827319.028921:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827319.028924:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827319.028927:0:38:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:2.0:1787827319.028928:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228228352:192.168.203.114@tcp:250:kworker.0 00000100:00080000:0.0:1787827319.028932:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3464 00000100:00080000:0.0:1787827319.028938:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827319.028940:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787827319.028943:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827319.028952:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4b40 x1874669228228608/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787827319.028987:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827319.029012:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827319/real 1787827319] req@ffff985607df7840 x1874669228228352/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827344 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827319.029024:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7840 x1874669228228352/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827344 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827319.029031:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827319.029034:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827319.029037:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228228352:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827319.029055:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228228480:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827319.029069:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827319.029077:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827319/real 1787827319] req@ffff985607df4780 x1874669228228480/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827344 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827319.029084:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4780 x1874669228228480/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827344 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827319.029090:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827319.029091:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827319.029102:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228228480:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827319.029106:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228228608:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827319.029118:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827319.029127:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827319/real 1787827319] req@ffff985607df4b40 x1874669228228608/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827344 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827319.029139:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4b40 x1874669228228608/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827344 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827319.029145:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827319.029148:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827319.029150:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228228608:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787827319.604855:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827320.628849:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827321.652826:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827322.676803:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827323.700871:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827324.148853:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827324.148863:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827324.148868:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827324.148878:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3469 00000100:00080000:1.0:1787827324.148889:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827324.148896:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787827324.148932:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a1a40 x1874669228228736/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827324.148960:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827324.148964:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827324.148966:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827324.148970:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3469 00000100:00080000:1.0:1787827324.148973:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827324.148978:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787827324.148989:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827324.149003:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228228736:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787827324.149005:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2580 x1874669228228864/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827324.149020:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827324.149024:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827324.149026:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827324.149030:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3469 00000100:00080000:1.0:1787827324.149035:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827324.149039:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787827324.149045:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827324.149057:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2940 x1874669228228992/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787827324.149068:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827324.149107:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827324/real 1787827324] req@ffff9856085a1a40 x1874669228228736/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827349 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827324.149121:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a1a40 x1874669228228736/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827349 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827324.149129:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827324.149132:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827324.149135:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228228736:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827324.149162:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228228864:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827324.149180:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827324.149189:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827324/real 1787827324] req@ffff9856085a2580 x1874669228228864/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827349 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827324.149202:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2580 x1874669228228864/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827349 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827324.149207:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827324.149209:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827324.149219:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228228864:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827324.149223:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228228992:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827324.149235:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827324.149244:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827324/real 1787827324] req@ffff9856085a2940 x1874669228228992/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827349 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827324.149252:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2940 x1874669228228992/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827349 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827324.149263:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827324.149267:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827324.149271:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228228992:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787827324.724797:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827325.748871:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827326.772849:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827327.796870:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827328.820877:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827329.268817:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827329.268828:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827329.268832:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827329.268842:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3474 00000100:00080000:3.0:1787827329.268854:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827329.268861:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787827329.268903:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83c00 x1874669228229120/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827329.268930:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827329.268933:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827329.268935:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827329.268938:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3474 00000100:00080000:3.0:1787827329.268940:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827329.268943:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827329.268954:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827329.268965:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f803c0 x1874669228229248/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827329.268973:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827329.268976:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827329.268977:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827329.268980:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3474 00000100:00080000:3.0:1787827329.268982:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827329.268985:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827329.268989:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827329.268999:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228229120:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787827329.269002:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80780 x1874669228229376/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787827329.269092:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827329.269130:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827329/real 1787827329] req@ffff985610f83c00 x1874669228229120/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827354 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827329.269162:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83c00 x1874669228229120/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827354 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827329.269179:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827329.269184:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827329.269189:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228229120:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827329.269197:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228229248:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827329.269222:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827329.269236:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827329/real 1787827329] req@ffff985610f803c0 x1874669228229248/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827354 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827329.269248:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f803c0 x1874669228229248/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827354 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827329.269257:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827329.269261:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827329.269275:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228229248:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827329.269305:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228229376:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827329.269331:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827329.269348:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827329/real 1787827329] req@ffff985610f80780 x1874669228229376/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827354 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827329.269360:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80780 x1874669228229376/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827354 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827329.269369:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827329.269372:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827329.269375:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228229376:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787827329.844855:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827330.484819:0:92271:0:(recover.c:348:ptlrpc_recover_import()) lustre-MDT0000_UUID: recovery finished DISCONN, rc = -110 00000080:00200008:1.0:1787827330.489477:0:92276:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:1.0:1787827330.489489:0:92276:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787827330.489512:0:92276:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827330.489531:0:92276:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 52 lru_type: 1 00010000:00010000:1.0:1787827330.489547:0:92276:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 52 lru_type: 1 00000080:00200000:1.0:1787827330.489556:0:92276:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827330.489564:0:92276:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787827330.489567:0:92276:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee56d 00000080:00010000:1.0:1787827330.489573:0:92276:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee56d from it ffffab3ac1d47cb8 00010000:00010000:1.0:1787827330.489576:0:92276:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 52 lru_type: 1 00010000:00010000:1.0:1787827330.489584:0:92276:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 53 lru_type: 1 00000080:00200000:1.0:1787827330.489609:0:92276:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787827330.489617:0:92276:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827330.489624:0:92276:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 53 lru_type: 1 00010000:00010000:1.0:1787827330.489634:0:92276:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 53 lru_type: 1 00000080:00200000:1.0:1787827330.489642:0:92276:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827330.489646:0:92276:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787827330.489649:0:92276:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee56d 00000080:00010000:1.0:1787827330.489653:0:92276:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee56d from it ffffab3ac1d47bb0 00010000:00010000:1.0:1787827330.489657:0:92276:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 53 lru_type: 1 00010000:00010000:1.0:1787827330.489672:0:92276:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 54 lru_type: 1 00000080:00200000:1.0:1787827330.489681:0:92276:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 44 00000080:00200008:1.0:1787827330.489720:0:92276:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 100400 00000080:00200008:1.0:1787827330.489734:0:92276:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 100000 00000080:00200008:1.0:1787827330.490544:0:92276:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:1.0:1787827330.490551:0:92276:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787827330.490565:0:92276:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827330.490577:0:92276:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 54 lru_type: 1 00010000:00010000:1.0:1787827330.490590:0:92276:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 54 lru_type: 1 00000080:00200000:1.0:1787827330.490597:0:92276:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827330.490604:0:92276:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787827330.490608:0:92276:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee56d 00000080:00010000:1.0:1787827330.490613:0:92276:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee56d from it ffffab3ac1d47cb8 00010000:00010000:1.0:1787827330.490617:0:92276:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 54 lru_type: 1 00010000:00010000:1.0:1787827330.490625:0:92276:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 55 lru_type: 1 00000080:00200008:1.0:1787827330.490654:0:92276:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:1.0:1787827330.490658:0:92276:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787827330.490667:0:92276:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827330.490674:0:92276:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 55 lru_type: 1 00010000:00010000:1.0:1787827330.490682:0:92276:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 55 lru_type: 1 00000080:00200000:1.0:1787827330.491059:0:92276:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827330.491068:0:92276:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787827330.491072:0:92276:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee56d 00000080:00010000:1.0:1787827330.491078:0:92276:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee56d from it ffffab3ac1d47cb8 00010000:00010000:1.0:1787827330.491082:0:92276:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 55 lru_type: 1 00010000:00010000:1.0:1787827330.491092:0:92276:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 56 lru_type: 1 00000080:00200000:1.0:1787827330.491116:0:92276:0:(llite_lib.c:2903:ll_statfs()) VFS Op:sb=lustre (ffff985602bc9800) / proj=0 00000080:00000004:1.0:1787827330.491120:0:92276:0:(obd_class.h:1091:obd_statfs()) lustre-clilmv-ffff985602bc9800: age 3358, max_age 3480 00800000:00000004:1.0:1787827330.491128:0:92276:0:(obd_class.h:1091:obd_statfs()) lustre-MDT0000-mdc-ffff985602bc9800: age 3335, max_age 3480 00000100:00100000:1.0:1787827330.491158:0:92276:0:(client.c:2715:ptlrpc_set_wait()) set ffff985610d3e0c0 going to sleep for 0 seconds 00000100:00100000:1.0:1787827330.491170:0:92276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a21c0 pname:cluuid:pid:xid:nid:opc:job df:7b4e8535-9db7-485d-abf6-bda966b63320:92276:1874669228229504:192.168.203.114@tcp:41:df.0 00800000:00020000:1.0:1787827330.491184:0:92276:0:(lmv_obd.c:1471:lmv_statfs()) lustre-MDT0000-mdc-ffff985602bc9800: can't stat MDS #0: rc = -110 00800000:00000004:1.0:1787827330.496284:0:92276:0:(obd_class.h:1091:obd_statfs()) lustre-MDT0001-mdc-ffff985602bc9800: age 3358, max_age 3480 00000100:00100000:1.0:1787827330.496313:0:92276:0:(client.c:2715:ptlrpc_set_wait()) set ffff985610d3e0c0 going to sleep for 0 seconds 00000100:00100000:1.0:1787827330.496321:0:92276:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a21c0 pname:cluuid:pid:xid:nid:opc:job df:7b4e8535-9db7-485d-abf6-bda966b63320:92276:1874669228229632:192.168.203.114@tcp:41:df.0 00800000:00020000:1.0:1787827330.496333:0:92276:0:(lmv_obd.c:1471:lmv_statfs()) lustre-MDT0001-mdc-ffff985602bc9800: can't stat MDS #1: rc = -110 00010000:00010000:2.0:1787827330.868842:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827331.892791:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827332.916926:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827333.940826:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827334.388790:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827334.388801:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827334.388807:0:38:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827334.388818:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3480 00000100:00080000:0.0:1787827334.388829:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827334.388837:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787827334.388880:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4000 x1874669228229760/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827334.388910:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827334.388913:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827334.388914:0:38:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827334.388917:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3480 00000100:00080000:0.0:1787827334.388919:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827334.388922:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787827334.388932:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827334.388943:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7c00 x1874669228229888/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827334.388951:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827334.388954:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827334.388955:0:38:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827334.388958:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3480 00000100:00080000:0.0:1787827334.388960:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827334.388963:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787827334.388967:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827334.388972:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228229760:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787827334.388978:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4b40 x1874669228230016/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787827334.389044:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827334.389079:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827334/real 1787827334] req@ffff985607df4000 x1874669228229760/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827359 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827334.389094:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4000 x1874669228229760/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827359 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827334.389102:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827334.389105:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827334.389109:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228229760:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827334.389119:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228229888:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827334.389136:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827334.389144:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827334/real 1787827334] req@ffff985607df7c00 x1874669228229888/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827359 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827334.389152:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7c00 x1874669228229888/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827359 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827334.389157:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827334.389159:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827334.389161:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228229888:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827334.389185:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228230016:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827334.389199:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827334.389206:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827334/real 1787827334] req@ffff985607df4b40 x1874669228230016/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827359 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827334.389213:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4b40 x1874669228230016/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827359 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827334.389218:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827334.389220:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827334.389222:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228230016:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787827334.964836:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827335.988816:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000080:0.0:1787827336.701354:0:92294:0:(class_obd.c:336:class_handle_ioctl()) obdclass: cmd=40086685 len=0 uarg=00007ffce5d8c840 00000002:00000080:0.0:1787827336.701375:0:92294:0:(mdc_request.c:2317:mdc_iocontrol()) lustre-MDT0000-mdc-ffff985602bc9800: cmd=40086685 len=576 karg=ffff985608614400 uarg=0000000000000000 00000100:00080000:0.0:1787827336.701381:0:92294:0:(import.c:214:ptlrpc_set_import_discon()) mdc: import ffff985611182000 already not connected (conn 333, was 0): DISCONN 00000100:00080000:0.0:1787827336.701387:0:92294:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827336.701393:0:92294:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3485 00000100:00080000:0.0:1787827336.701402:0:92294:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827336.701407:0:92294:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787827336.701416:0:92294:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827336.701439:0:92294:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4780 x1874669228230144/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827336.701460:0:92294:0:(recover.c:337:ptlrpc_recover_import()) lustre-MDT0000_UUID: recovery started, waiting 20 seconds 00000100:00100000:2.0:1787827336.701489:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228230144:192.168.203.114@tcp:38:lctl.0 00000800:00000100:2.0:1787827336.701540:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827336.701561:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827336/real 1787827336] req@ffff985607df4780 x1874669228230144/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827361 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'lctl.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827336.701573:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4780 x1874669228230144/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827361 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'lctl.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827336.701580:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827336.701583:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827336.701594:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228230144:192.168.203.114@tcp:38:lctl.0 00010000:00010000:0.0:1787827337.012866:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827338.036845:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827339.060858:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827339.508871:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827339.508880:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827339.508884:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827339.508892:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3485 00000100:00080000:3.0:1787827339.508901:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827339.508907:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787827339.508942:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f830c0 x1874669228230272/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827339.508966:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827339.508969:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827339.508971:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827339.508974:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3487 00000100:00080000:3.0:1787827339.508976:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827339.508979:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827339.508987:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827339.508995:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81a40 x1874669228230400/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827339.509004:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827339.509007:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827339.509008:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827339.509012:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3485 00000100:00080000:3.0:1787827339.509014:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827339.509016:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827339.509019:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827339.509027:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f812c0 x1874669228230528/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827339.509035:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228230272:192.168.203.114@tcp:250:kworker.0 00000800:00000100:2.0:1787827339.509101:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827339.509136:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827339/real 1787827339] req@ffff985610f830c0 x1874669228230272/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827364 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827339.509157:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f830c0 x1874669228230272/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827364 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827339.509169:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827339.509174:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827339.509178:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228230272:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827339.509188:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228230400:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827339.509212:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827339.509226:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827339/real 1787827339] req@ffff985610f81a40 x1874669228230400/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827364 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827339.509241:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81a40 x1874669228230400/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827364 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827339.509250:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827339.509254:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827339.509271:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228230400:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827339.509301:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228230528:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827339.509325:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827339.509337:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827339/real 1787827339] req@ffff985610f812c0 x1874669228230528/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827364 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827339.509352:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f812c0 x1874669228230528/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827364 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827339.509367:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827339.509372:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827339.509375:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228230528:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787827340.084788:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827341.108839:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827342.132866:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827343.156839:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827344.180858:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827344.628788:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827344.628797:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827344.628800:0:38:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827344.628809:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3490 00000100:00080000:3.0:1787827344.628818:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827344.628824:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787827344.628855:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82d00 x1874669228230656/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827344.628878:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827344.628881:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827344.628882:0:38:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827344.628885:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3490 00000100:00080000:3.0:1787827344.628887:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827344.628890:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827344.628903:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827344.628913:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80f00 x1874669228230784/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827344.628921:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1787827344.628924:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228230656:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787827344.628925:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827344.628929:0:38:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827344.628934:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3490 00000100:00080000:3.0:1787827344.628941:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827344.628943:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827344.628947:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827344.628956:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81680 x1874669228230912/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827344.629005:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228230784:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827344.629027:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228230912:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787827345.204788:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827346.228925:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827347.252793:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:3.0:1787827347.700901:0:2373:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.114:988 00000400:00000100:3.0:1787827347.700936:0:2373:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.114@tcp at host 192.168.203.114 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:3.0:1787827347.700949:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:3.0:1787827347.700989:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:3.0:1787827347.701000:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000100:00100000:2.0:1787827347.701092:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827344/real 1787827347] req@ffff985610f80f00 x1874669228230784/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827369 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827347.701134:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80f00 x1874669228230784/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827369 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827347.701150:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827347.701157:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827347.701186:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228230784:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827347.701196:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827344/real 1787827347] req@ffff985610f81680 x1874669228230912/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827369 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827347.701212:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81680 x1874669228230912/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827369 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827347.701230:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827347.701235:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827347.701240:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228230912:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827347.701251:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827344/real 1787827347] req@ffff985610f82d00 x1874669228230656/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827369 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827347.701266:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82d00 x1874669228230656/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827369 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827347.701276:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827347.701281:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827347.701285:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228230656:192.168.203.114@tcp:250:kworker.0 00010000:00010000:3.0:1787827348.276855:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827349.300831:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827349.748852:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827349.748865:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827349.748870:0:38:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827349.748884:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3495 00000100:00080000:3.0:1787827349.748899:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827349.748908:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787827349.748954:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81e00 x1874669228231040/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827349.748992:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827349.748997:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827349.748999:0:38:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827349.749004:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3495 00000100:00080000:3.0:1787827349.749008:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827349.749013:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827349.749025:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827349.749041:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82940 x1874669228231168/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827349.749054:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827349.749059:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827349.749062:0:38:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827349.749067:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3495 00000100:00080000:3.0:1787827349.749071:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00100000:2.0:1787827349.749076:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228231040:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787827349.749079:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827349.749088:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827349.749099:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82580 x1874669228231296/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787827349.749177:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827349.749238:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827349/real 1787827349] req@ffff985610f81e00 x1874669228231040/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827374 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827349.749260:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81e00 x1874669228231040/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827374 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827349.749273:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827349.749277:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827349.749281:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228231040:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827349.749313:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228231168:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827349.749341:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827349.749356:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827349/real 1787827349] req@ffff985610f82940 x1874669228231168/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827374 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827349.749369:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82940 x1874669228231168/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827374 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827349.749378:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827349.749382:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827349.749411:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228231168:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827349.749419:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228231296:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827349.749443:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827349.749453:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827349/real 1787827349] req@ffff985610f82580 x1874669228231296/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827374 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827349.749462:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82580 x1874669228231296/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827374 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827349.749468:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827349.749471:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827349.749473:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228231296:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787827350.324855:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827351.348868:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827352.372868:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827353.396915:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827354.420874:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827354.868799:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827354.868807:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827354.868811:0:38:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827354.868819:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3500 00000100:00080000:3.0:1787827354.868828:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827354.868834:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787827354.868865:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f821c0 x1874669228231424/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827354.868897:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827354.868900:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827354.868901:0:38:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827354.868904:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3500 00000100:00080000:3.0:1787827354.868906:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827354.868909:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827354.868917:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827354.868930:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83480 x1874669228231552/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827354.868937:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827354.868940:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827354.868942:0:38:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827354.868945:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3500 00000100:00080000:3.0:1787827354.868947:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827354.868952:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787827354.868954:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f821c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228231424:192.168.203.114@tcp:250:kworker.0 00000008:00100000:3.0:1787827354.868957:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827354.868968:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82580 x1874669228231680/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787827354.869013:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827354.869041:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827354/real 1787827354] req@ffff985610f821c0 x1874669228231424/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827379 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827354.869054:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f821c0 x1874669228231424/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827379 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827354.869061:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827354.869064:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827354.869067:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f821c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228231424:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827354.869072:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228231552:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827354.869087:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827354.869095:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827354/real 1787827354] req@ffff985610f83480 x1874669228231552/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827379 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827354.869102:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83480 x1874669228231552/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827379 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827354.869107:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827354.869109:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827354.869119:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228231552:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827354.869139:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228231680:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827354.869155:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827354.869162:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827354/real 1787827354] req@ffff985610f82580 x1874669228231680/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827379 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827354.869172:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82580 x1874669228231680/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827379 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827354.869183:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827354.869185:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827354.869188:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228231680:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787827355.444788:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827356.468885:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827356.724746:0:92294:0:(recover.c:348:ptlrpc_recover_import()) lustre-MDT0000_UUID: recovery finished DISCONN, rc = -110 00000080:00200008:0.0:1787827356.728053:0:92299:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:0.0:1787827356.728062:0:92299:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:0.0:1787827356.728080:0:92299:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827356.728096:0:92299:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 56 lru_type: 1 00010000:00010000:0.0:1787827356.728107:0:92299:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 56 lru_type: 1 00000080:00200000:0.0:1787827356.728114:0:92299:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827356.728119:0:92299:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:0.0:1787827356.728121:0:92299:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee56d 00000080:00010000:0.0:1787827356.728128:0:92299:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee56d from it ffffab3ac1d5fcb8 00010000:00010000:0.0:1787827356.728131:0:92299:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 56 lru_type: 1 00010000:00010000:0.0:1787827356.728137:0:92299:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 57 lru_type: 1 00000080:00200000:0.0:1787827356.728157:0:92299:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:0.0:1787827356.728161:0:92299:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827356.728165:0:92299:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 57 lru_type: 1 00010000:00010000:0.0:1787827356.728171:0:92299:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 57 lru_type: 1 00000080:00200000:0.0:1787827356.728175:0:92299:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827356.728178:0:92299:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:0.0:1787827356.728179:0:92299:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee56d 00000080:00010000:0.0:1787827356.728181:0:92299:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee56d from it ffffab3ac1d5fbb0 00010000:00010000:0.0:1787827356.728184:0:92299:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 57 lru_type: 1 00010000:00010000:0.0:1787827356.728189:0:92299:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 58 lru_type: 1 00000080:00200000:0.0:1787827356.728194:0:92299:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 44 00000080:00200008:0.0:1787827356.728199:0:92299:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 100400 00000080:00200008:0.0:1787827356.728208:0:92299:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 100000 00000080:00200008:0.0:1787827356.728779:0:92299:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:0.0:1787827356.728784:0:92299:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:0.0:1787827356.728790:0:92299:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827356.728797:0:92299:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 58 lru_type: 1 00010000:00010000:0.0:1787827356.728804:0:92299:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 58 lru_type: 1 00000080:00200000:0.0:1787827356.728809:0:92299:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827356.728812:0:92299:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:0.0:1787827356.728814:0:92299:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee56d 00000080:00010000:0.0:1787827356.728817:0:92299:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee56d from it ffffab3ac1d5fcb8 00010000:00010000:0.0:1787827356.728819:0:92299:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 58 lru_type: 1 00010000:00010000:0.0:1787827356.728825:0:92299:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 59 lru_type: 1 00000080:00200008:0.0:1787827356.728845:0:92299:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:0.0:1787827356.728848:0:92299:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:0.0:1787827356.728852:0:92299:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827356.728856:0:92299:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 59 lru_type: 1 00010000:00010000:0.0:1787827356.728861:0:92299:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 59 lru_type: 1 00000080:00200000:0.0:1787827356.728866:0:92299:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827356.728868:0:92299:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:0.0:1787827356.728870:0:92299:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee56d 00000080:00010000:0.0:1787827356.728872:0:92299:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee56d from it ffffab3ac1d5fcb8 00010000:00010000:0.0:1787827356.728874:0:92299:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 59 lru_type: 1 00010000:00010000:0.0:1787827356.728878:0:92299:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 60 lru_type: 1 00000080:00200000:0.0:1787827356.728891:0:92299:0:(llite_lib.c:2903:ll_statfs()) VFS Op:sb=lustre (ffff985602bc9800) / proj=0 00000080:00000004:0.0:1787827356.728894:0:92299:0:(obd_class.h:1091:obd_statfs()) lustre-clilmv-ffff985602bc9800: age 3358, max_age 3506 00800000:00000004:0.0:1787827356.728902:0:92299:0:(obd_class.h:1091:obd_statfs()) lustre-MDT0000-mdc-ffff985602bc9800: age 3335, max_age 3506 00000100:00100000:0.0:1787827356.728930:0:92299:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607ed2c00 going to sleep for 0 seconds 00000100:00100000:0.0:1787827356.728939:0:92299:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7840 pname:cluuid:pid:xid:nid:opc:job df:7b4e8535-9db7-485d-abf6-bda966b63320:92299:1874669228231808:192.168.203.114@tcp:41:df.0 00800000:00020000:0.0:1787827356.728950:0:92299:0:(lmv_obd.c:1471:lmv_statfs()) lustre-MDT0000-mdc-ffff985602bc9800: can't stat MDS #0: rc = -110 00800000:00000004:0.0:1787827356.734110:0:92299:0:(obd_class.h:1091:obd_statfs()) lustre-MDT0001-mdc-ffff985602bc9800: age 3358, max_age 3506 00000100:00100000:0.0:1787827356.734198:0:92299:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607ed2c00 going to sleep for 0 seconds 00000100:00100000:0.0:1787827356.734204:0:92299:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7840 pname:cluuid:pid:xid:nid:opc:job df:7b4e8535-9db7-485d-abf6-bda966b63320:92299:1874669228231936:192.168.203.114@tcp:41:df.0 00800000:00020000:0.0:1787827356.734215:0:92299:0:(lmv_obd.c:1471:lmv_statfs()) lustre-MDT0001-mdc-ffff985602bc9800: can't stat MDS #1: rc = -110 00010000:00010000:0.0:1787827357.492795:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827358.516915:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827359.540854:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827359.988859:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827359.988891:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827359.988896:0:38:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827359.988917:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3505 00000100:00080000:0.0:1787827359.988949:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827359.988959:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787827359.989003:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6940 x1874669228232064/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827359.989031:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827359.989034:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827359.989035:0:38:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827359.989038:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3505 00000100:00080000:0.0:1787827359.989057:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827359.989060:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787827359.989070:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827359.989082:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df52c0 x1874669228232192/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827359.989090:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827359.989094:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:2.0:1787827359.989100:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228232064:192.168.203.114@tcp:250:kworker.0 00000100:00080000:0.0:1787827359.989102:0:38:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827359.989111:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3505 00000100:00080000:0.0:1787827359.989114:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827359.989116:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787827359.989120:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827359.989134:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7480 x1874669228232320/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787827359.989179:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827359.989221:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827359/real 1787827359] req@ffff985607df6940 x1874669228232064/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827384 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827359.989236:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6940 x1874669228232064/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827384 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827359.989246:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827359.989249:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827359.989252:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228232064:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827359.989275:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228232192:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827359.989296:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827359.989310:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827359/real 1787827359] req@ffff985607df52c0 x1874669228232192/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827384 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827359.989323:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df52c0 x1874669228232192/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827384 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827359.989333:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827359.989337:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827359.989341:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228232192:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827359.989347:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228232320:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827359.989364:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827359.989372:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827359/real 1787827359] req@ffff985607df7480 x1874669228232320/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827384 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827359.989380:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7480 x1874669228232320/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827384 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827359.989386:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827359.989388:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827359.989391:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228232320:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787827360.564862:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827361.588827:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827362.612856:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000080:2.0:1787827362.942844:0:92318:0:(class_obd.c:336:class_handle_ioctl()) obdclass: cmd=40086685 len=0 uarg=00007ffc01dab000 00000002:00000080:2.0:1787827362.942862:0:92318:0:(mdc_request.c:2317:mdc_iocontrol()) lustre-MDT0000-mdc-ffff985602bc9800: cmd=40086685 len=576 karg=ffff9856110f3c00 uarg=0000000000000000 00000100:00080000:2.0:1787827362.942869:0:92318:0:(import.c:214:ptlrpc_set_import_discon()) mdc: import ffff985611182000 already not connected (conn 339, was 0): DISCONN 00000100:00080000:2.0:1787827362.942876:0:92318:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827362.942886:0:92318:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3510 00000100:00080000:2.0:1787827362.942897:0:92318:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827362.942904:0:92318:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787827362.942915:0:92318:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827362.942944:0:92318:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8f00 x1874669228232448/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827362.942975:0:92318:0:(recover.c:337:ptlrpc_recover_import()) lustre-MDT0000_UUID: recovery started, waiting 20 seconds 00000100:00100000:0.0:1787827362.943009:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228232448:192.168.203.114@tcp:38:lctl.0 00000800:00000100:0.0:1787827362.943081:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787827362.943108:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827362/real 1787827362] req@ffff9856140d8f00 x1874669228232448/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827387 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'lctl.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827362.943123:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8f00 x1874669228232448/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827387 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'lctl.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827362.943132:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827362.943136:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827362.943147:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228232448:192.168.203.114@tcp:38:lctl.0 00010000:00010000:3.0:1787827363.636838:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827364.660895:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827365.108787:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827365.108803:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827365.108808:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827365.108817:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3510 00000100:00080000:1.0:1787827365.108827:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827365.108835:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787827365.108875:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a30c0 x1874669228232576/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827365.108897:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827365.108902:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827365.108904:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827365.108919:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3513 00000100:00080000:1.0:1787827365.108924:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827365.108928:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787827365.108938:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827365.108944:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228232576:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787827365.108953:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0f00 x1874669228232704/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827365.108965:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827365.108968:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827365.108969:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827365.108974:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3510 00000100:00080000:1.0:1787827365.108978:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827365.108991:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787827365.108995:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827365.109007:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3480 x1874669228232832/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787827365.109023:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787827365.109056:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827365/real 1787827365] req@ffff9856085a30c0 x1874669228232576/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827390 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827365.109070:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a30c0 x1874669228232576/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827390 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827365.109079:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827365.109081:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827365.109084:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228232576:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787827365.109102:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228232704:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787827365.109119:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787827365.109137:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827365/real 1787827365] req@ffff9856085a0f00 x1874669228232704/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827390 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827365.109145:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0f00 x1874669228232704/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827390 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827365.109151:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827365.109153:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827365.109163:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228232704:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787827365.109167:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228232832:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787827365.109180:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787827365.109187:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827365/real 1787827365] req@ffff9856085a3480 x1874669228232832/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827390 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827365.109195:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3480 x1874669228232832/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827390 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827365.109201:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827365.109204:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827365.109206:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228232832:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787827365.684812:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827366.708829:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827367.732830:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827368.756818:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827369.780874:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827370.228880:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827370.228893:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827370.228898:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827370.228913:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3515 00000100:00080000:2.0:1787827370.228928:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827370.228937:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787827370.228989:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140dbc00 x1874669228232960/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827370.229023:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827370.229027:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827370.229029:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827370.229032:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3515 00000100:00080000:2.0:1787827370.229035:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827370.229038:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787827370.229049:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827370.229060:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db840 x1874669228233088/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827370.229070:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827370.229073:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827370.229076:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827370.229080:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3515 00000100:00080000:2.0:1787827370.229082:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827370.229087:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787827370.229093:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228232960:192.168.203.114@tcp:250:kworker.0 00000008:00100000:2.0:1787827370.229093:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827370.229104:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140da1c0 x1874669228233216/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787827370.229188:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787827370.229240:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827370/real 1787827370] req@ffff9856140dbc00 x1874669228232960/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827395 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827370.229267:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140dbc00 x1874669228232960/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827395 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827370.229281:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827370.229285:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827370.229289:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228232960:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787827370.229297:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228233088:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787827370.229346:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787827370.229365:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827370/real 1787827370] req@ffff9856140db840 x1874669228233088/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827395 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827370.229381:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db840 x1874669228233088/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827395 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827370.229392:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827370.229396:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827370.229445:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228233088:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787827370.229502:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228233216:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787827370.229527:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787827370.229537:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827370/real 1787827370] req@ffff9856140da1c0 x1874669228233216/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827395 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827370.229548:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140da1c0 x1874669228233216/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827395 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827370.229555:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827370.229558:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827370.229561:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228233216:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787827370.804806:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827371.828992:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827372.852814:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827373.876930:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827374.900867:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827375.348760:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827375.348768:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827375.348771:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827375.348779:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3520 00000100:00080000:2.0:1787827375.348787:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827375.348793:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787827375.348825:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140da1c0 x1874669228233344/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827375.348847:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827375.348849:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827375.348851:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827375.348853:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3520 00000100:00080000:2.0:1787827375.348856:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827375.348858:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787827375.348866:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827375.348875:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db840 x1874669228233472/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827375.348883:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827375.348885:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827375.348889:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1787827375.348890:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228233344:192.168.203.114@tcp:250:kworker.0 00000100:00080000:2.0:1787827375.348893:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3520 00000100:00080000:2.0:1787827375.348907:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827375.348910:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787827375.348925:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827375.348935:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140dbc00 x1874669228233600/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787827375.348964:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787827375.348992:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827375/real 1787827375] req@ffff9856140da1c0 x1874669228233344/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827400 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827375.349005:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140da1c0 x1874669228233344/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827400 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827375.349013:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827375.349016:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827375.349019:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228233344:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787827375.349037:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228233472:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787827375.349062:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787827375.349070:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827375/real 1787827375] req@ffff9856140db840 x1874669228233472/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827400 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827375.349080:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db840 x1874669228233472/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827400 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827375.349085:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827375.349088:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827375.349097:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228233472:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787827375.349101:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228233600:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787827375.349115:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787827375.349127:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827375/real 1787827375] req@ffff9856140dbc00 x1874669228233600/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827400 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827375.349135:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140dbc00 x1874669228233600/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827400 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827375.349140:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827375.349143:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827375.349145:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228233600:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787827375.924836:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827376.948828:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827377.972881:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827378.996812:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827380.020875:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827380.468807:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827380.468818:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827380.468824:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827380.468834:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3526 00000100:00080000:2.0:1787827380.468846:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827380.468854:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787827380.468894:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8f00 x1874669228233728/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827380.468922:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827380.468927:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827380.468930:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827380.468936:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3526 00000100:00080000:2.0:1787827380.468940:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827380.468956:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787827380.468959:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228233728:192.168.203.114@tcp:250:kworker.0 00000008:00100000:2.0:1787827380.468969:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827380.468986:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140dad00 x1874669228233856/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827380.468999:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827380.469002:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827380.469005:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827380.469009:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3526 00000100:00080000:2.0:1787827380.469013:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000800:00000100:0.0:1787827380.469019:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00080000:2.0:1787827380.469020:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787827380.469029:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827380.469043:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db0c0 x1874669228233984/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827380.469051:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827380/real 1787827380] req@ffff9856140d8f00 x1874669228233728/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827405 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827380.469069:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8f00 x1874669228233728/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827405 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827380.469079:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827380.469082:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827380.469086:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228233728:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787827380.469115:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228233856:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787827380.469129:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787827380.469137:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827380/real 1787827380] req@ffff9856140dad00 x1874669228233856/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827405 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827380.469144:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140dad00 x1874669228233856/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827405 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827380.469150:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827380.469152:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827380.469163:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228233856:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787827380.469167:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228233984:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787827380.469178:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787827380.469204:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827380/real 1787827380] req@ffff9856140db0c0 x1874669228233984/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827405 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827380.469214:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db0c0 x1874669228233984/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827405 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827380.469220:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827380.469223:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827380.469225:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228233984:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787827381.044865:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827382.068845:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827382.964878:0:92318:0:(recover.c:348:ptlrpc_recover_import()) lustre-MDT0000_UUID: recovery finished DISCONN, rc = -110 00000080:00200008:0.0:1787827382.970398:0:92322:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:0.0:1787827382.970411:0:92322:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:0.0:1787827382.970437:0:92322:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827382.970460:0:92322:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 60 lru_type: 1 00010000:00010000:0.0:1787827382.970481:0:92322:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 60 lru_type: 1 00000080:00200000:0.0:1787827382.970494:0:92322:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827382.970504:0:92322:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:0.0:1787827382.970509:0:92322:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee56d 00000080:00010000:0.0:1787827382.970519:0:92322:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee56d from it ffffab3ac0d9bcb8 00010000:00010000:0.0:1787827382.970525:0:92322:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 60 lru_type: 1 00010000:00010000:0.0:1787827382.970534:0:92322:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 61 lru_type: 1 00000080:00200000:0.0:1787827382.970584:0:92322:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:0.0:1787827382.970595:0:92322:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827382.970605:0:92322:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 61 lru_type: 1 00010000:00010000:0.0:1787827382.970618:0:92322:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 61 lru_type: 1 00000080:00200000:0.0:1787827382.970638:0:92322:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827382.970644:0:92322:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:0.0:1787827382.970648:0:92322:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee56d 00000080:00010000:0.0:1787827382.970654:0:92322:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee56d from it ffffab3ac0d9bbb0 00010000:00010000:0.0:1787827382.970659:0:92322:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 61 lru_type: 1 00010000:00010000:0.0:1787827382.970670:0:92322:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 62 lru_type: 1 00000080:00200000:0.0:1787827382.970683:0:92322:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 44 00000080:00200008:0.0:1787827382.970692:0:92322:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 100400 00000080:00200008:0.0:1787827382.970712:0:92322:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 100000 00000080:00200008:0.0:1787827382.971737:0:92322:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:0.0:1787827382.971744:0:92322:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:0.0:1787827382.971792:0:92322:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827382.971800:0:92322:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 62 lru_type: 1 00010000:00010000:0.0:1787827382.971808:0:92322:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 62 lru_type: 1 00000080:00200000:0.0:1787827382.971814:0:92322:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827382.971818:0:92322:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:0.0:1787827382.971820:0:92322:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee56d 00000080:00010000:0.0:1787827382.971824:0:92322:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee56d from it ffffab3ac0d9bcb8 00010000:00010000:0.0:1787827382.971827:0:92322:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 62 lru_type: 1 00010000:00010000:0.0:1787827382.971834:0:92322:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 63 lru_type: 1 00000080:00200008:0.0:1787827382.971874:0:92322:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:0.0:1787827382.971878:0:92322:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:0.0:1787827382.971884:0:92322:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827382.971890:0:92322:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 63 lru_type: 1 00010000:00010000:0.0:1787827382.971898:0:92322:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 63 lru_type: 1 00000080:00200000:0.0:1787827382.971904:0:92322:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827382.971907:0:92322:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:0.0:1787827382.971909:0:92322:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee56d 00000080:00010000:0.0:1787827382.971912:0:92322:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee56d from it ffffab3ac0d9bcb8 00010000:00010000:0.0:1787827382.971915:0:92322:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 63 lru_type: 1 00010000:00010000:0.0:1787827382.971922:0:92322:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 64 lru_type: 1 00000080:00200000:0.0:1787827382.971955:0:92322:0:(llite_lib.c:2903:ll_statfs()) VFS Op:sb=lustre (ffff985602bc9800) / proj=0 00000080:00000004:0.0:1787827382.971959:0:92322:0:(obd_class.h:1091:obd_statfs()) lustre-clilmv-ffff985602bc9800: age 3358, max_age 3532 00800000:00000004:0.0:1787827382.971967:0:92322:0:(obd_class.h:1091:obd_statfs()) lustre-MDT0000-mdc-ffff985602bc9800: age 3335, max_age 3532 00000100:00100000:0.0:1787827382.972021:0:92322:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607ed2180 going to sleep for 0 seconds 00000100:00100000:0.0:1787827382.972034:0:92322:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job df:7b4e8535-9db7-485d-abf6-bda966b63320:92322:1874669228234112:192.168.203.114@tcp:41:df.0 00800000:00020000:0.0:1787827382.972052:0:92322:0:(lmv_obd.c:1471:lmv_statfs()) lustre-MDT0000-mdc-ffff985602bc9800: can't stat MDS #0: rc = -110 00800000:00000004:0.0:1787827382.972058:0:92322:0:(obd_class.h:1091:obd_statfs()) lustre-MDT0001-mdc-ffff985602bc9800: age 3358, max_age 3532 00000100:00100000:0.0:1787827382.972073:0:92322:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607ed2180 going to sleep for 0 seconds 00000100:00100000:0.0:1787827382.972078:0:92322:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job df:7b4e8535-9db7-485d-abf6-bda966b63320:92322:1874669228234240:192.168.203.114@tcp:41:df.0 00800000:00020000:0.0:1787827382.972084:0:92322:0:(lmv_obd.c:1471:lmv_statfs()) lustre-MDT0001-mdc-ffff985602bc9800: can't stat MDS #1: rc = -110 00010000:00010000:0.0:1787827383.092816:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827384.116844:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827385.140948:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827385.588809:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827385.588822:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827385.588829:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827385.588844:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3531 00000100:00080000:2.0:1787827385.588859:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827385.588871:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787827385.588919:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140da940 x1874669228234368/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827385.588956:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827385.588960:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827385.588963:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827385.588967:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3531 00000100:00080000:2.0:1787827385.588972:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827385.588977:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787827385.588999:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827385.589015:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d83c0 x1874669228234496/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827385.589039:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1787827385.589047:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228234368:192.168.203.114@tcp:250:kworker.0 00000100:00080000:2.0:1787827385.589048:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827385.589053:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827385.589066:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3531 00000100:00080000:2.0:1787827385.589070:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827385.589074:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787827385.589080:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827385.589094:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8b40 x1874669228234624/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787827385.589156:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787827385.589219:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827385/real 1787827385] req@ffff9856140da940 x1874669228234368/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827410 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827385.589242:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140da940 x1874669228234368/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827410 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827385.589256:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827385.589262:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827385.589267:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228234368:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787827385.589295:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228234496:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787827385.589379:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787827385.589397:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827385/real 1787827385] req@ffff9856140d83c0 x1874669228234496/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827410 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827385.589412:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d83c0 x1874669228234496/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827410 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827385.589422:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827385.589426:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827385.589430:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228234496:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787827385.589437:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228234624:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787827385.589473:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787827385.589485:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827385/real 1787827385] req@ffff9856140d8b40 x1874669228234624/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827410 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827385.589499:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8b40 x1874669228234624/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827410 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827385.589508:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827385.589511:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827385.589514:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228234624:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787827386.164842:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827387.188826:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827388.212786:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000080:1.0:1787827389.181835:0:92341:0:(class_obd.c:336:class_handle_ioctl()) obdclass: cmd=40086685 len=0 uarg=00007ffe54f67230 00000002:00000080:1.0:1787827389.181851:0:92341:0:(mdc_request.c:2317:mdc_iocontrol()) lustre-MDT0000-mdc-ffff985602bc9800: cmd=40086685 len=576 karg=ffff98560865d800 uarg=0000000000000000 00000100:00080000:1.0:1787827389.181858:0:92341:0:(import.c:214:ptlrpc_set_import_discon()) mdc: import ffff985611182000 already not connected (conn 345, was 0): DISCONN 00000100:00080000:1.0:1787827389.181865:0:92341:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827389.181873:0:92341:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3536 00000100:00080000:1.0:1787827389.181883:0:92341:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827389.181890:0:92341:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787827389.181900:0:92341:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827389.181925:0:92341:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a12c0 x1874669228234752/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827389.181950:0:92341:0:(recover.c:337:ptlrpc_recover_import()) lustre-MDT0000_UUID: recovery started, waiting 20 seconds 00000100:00100000:0.0:1787827389.181977:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228234752:192.168.203.114@tcp:38:lctl.0 00000800:00000100:0.0:1787827389.182029:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787827389.182050:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827389/real 1787827389] req@ffff9856085a12c0 x1874669228234752/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827414 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'lctl.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827389.182062:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a12c0 x1874669228234752/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827414 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'lctl.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827389.182069:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827389.182080:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827389.182092:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228234752:192.168.203.114@tcp:38:lctl.0 00010000:00010000:3.0:1787827389.236867:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827390.260804:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827390.708912:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827390.708937:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827390.708943:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827390.708959:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3536 00000100:00080000:3.0:1787827390.708977:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827390.708991:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787827390.709065:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82940 x1874669228234880/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827390.709118:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827390.709123:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827390.709125:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827390.709139:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3539 00000100:00080000:3.0:1787827390.709145:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827390.709150:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827390.709162:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827390.709179:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81e00 x1874669228235008/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827390.709190:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827390.709195:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827390.709197:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827390.709202:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3536 00000100:00080000:3.0:1787827390.709206:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827390.709210:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827390.709214:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827390.709225:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82d00 x1874669228235136/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827390.709244:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228234880:192.168.203.114@tcp:250:kworker.0 00000800:00000100:0.0:1787827390.709359:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787827390.709396:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827390/real 1787827390] req@ffff985610f82940 x1874669228234880/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827415 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827390.709412:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82940 x1874669228234880/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827415 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827390.709423:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827390.709427:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827390.709431:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228234880:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787827390.709438:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228235008:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787827390.709457:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787827390.709466:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827390/real 1787827390] req@ffff985610f81e00 x1874669228235008/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827415 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827390.709486:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81e00 x1874669228235008/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827415 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827390.709493:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827390.709496:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827390.709509:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228235008:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787827390.709533:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228235136:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787827390.709552:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787827390.709569:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827390/real 1787827390] req@ffff985610f82d00 x1874669228235136/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827415 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827390.709584:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82d00 x1874669228235136/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827415 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827390.709592:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827390.709595:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827390.709597:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228235136:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787827391.284873:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827392.308978:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827393.332870:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827394.357005:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827395.381047:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827395.828835:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827395.828857:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827395.828862:0:38:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827395.828872:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3541 00000100:00080000:2.0:1787827395.828884:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827395.828890:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787827395.828935:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d9680 x1874669228235264/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827395.828962:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827395.828964:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827395.828966:0:38:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827395.828968:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3541 00000100:00080000:2.0:1787827395.828981:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827395.828983:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787827395.828991:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827395.828999:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db480 x1874669228235392/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827395.829006:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827395.829009:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827395.829011:0:38:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827395.829013:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3541 00000100:00080000:2.0:1787827395.829015:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827395.829018:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787827395.829020:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827395.829031:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8780 x1874669228235520/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827395.829032:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228235264:192.168.203.114@tcp:250:kworker.0 00000800:00000100:0.0:1787827395.829120:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787827395.829184:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827395/real 1787827395] req@ffff9856140d9680 x1874669228235264/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827420 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827395.829205:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d9680 x1874669228235264/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827420 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827395.829216:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827395.829220:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827395.829225:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228235264:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787827395.829233:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228235392:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787827395.829274:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787827395.829295:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827395/real 1787827395] req@ffff9856140db480 x1874669228235392/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827420 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827395.829309:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db480 x1874669228235392/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827420 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827395.829319:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827395.829323:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827395.829336:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228235392:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787827395.829364:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228235520:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787827395.829390:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787827395.829408:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827395/real 1787827395] req@ffff9856140d8780 x1874669228235520/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827420 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827395.829422:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8780 x1874669228235520/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827420 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827395.829430:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827395.829433:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827395.829436:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228235520:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787827396.404786:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827397.428842:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827398.452817:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827399.476825:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827400.501008:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827400.948886:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827400.948897:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827400.948902:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827400.948914:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3546 00000100:00080000:3.0:1787827400.948926:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827400.948934:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787827400.948977:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81680 x1874669228235648/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827400.949013:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827400.949017:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827400.949019:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827400.949023:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3546 00000100:00080000:3.0:1787827400.949027:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827400.949031:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827400.949042:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827400.949055:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83840 x1874669228235776/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827400.949094:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827400.949100:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827400.949102:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827400.949107:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3546 00000100:00080000:3.0:1787827400.949112:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827400.949117:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827400.949125:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827400.949135:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228235648:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787827400.949143:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f812c0 x1874669228235904/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787827400.949294:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787827400.949338:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827400/real 1787827400] req@ffff985610f81680 x1874669228235648/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827425 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827400.949359:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81680 x1874669228235648/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827425 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827400.949372:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827400.949377:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827400.949382:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228235648:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787827400.949391:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228235776:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787827400.949418:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787827400.949431:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827400/real 1787827400] req@ffff985610f83840 x1874669228235776/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827425 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827400.949444:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83840 x1874669228235776/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827425 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827400.949452:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827400.949456:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827400.949478:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228235776:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787827400.949505:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228235904:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787827400.949529:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787827400.949542:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827400/real 1787827400] req@ffff985610f812c0 x1874669228235904/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827425 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827400.949561:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f812c0 x1874669228235904/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827425 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827400.949571:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827400.949575:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827400.949579:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228235904:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787827401.524841:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827402.548909:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827403.572814:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827404.596887:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827405.621008:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827406.068804:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827406.068832:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827406.068837:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827406.068850:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3551 00000100:00080000:3.0:1787827406.068863:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827406.068872:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787827406.068928:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81a40 x1874669228236032/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827406.068974:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827406.068977:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827406.068979:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827406.068982:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3551 00000100:00080000:3.0:1787827406.068985:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827406.068988:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827406.068997:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827406.069011:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f830c0 x1874669228236160/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827406.069019:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827406.069023:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827406.069025:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827406.069031:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3551 00000100:00080000:3.0:1787827406.069034:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827406.069039:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787827406.069044:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228236032:192.168.203.114@tcp:250:kworker.0 00000008:00100000:3.0:1787827406.069047:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827406.069064:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80b40 x1874669228236288/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787827406.069125:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787827406.069163:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827406/real 1787827406] req@ffff985610f81a40 x1874669228236032/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827431 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827406.069193:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81a40 x1874669228236032/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827431 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827406.069202:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827406.069205:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827406.069209:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228236032:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787827406.069215:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228236160:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787827406.069234:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787827406.069244:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827406/real 1787827406] req@ffff985610f830c0 x1874669228236160/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827431 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827406.069252:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f830c0 x1874669228236160/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827431 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827406.069257:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827406.069259:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827406.069274:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228236160:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787827406.069299:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228236288:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787827406.069315:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787827406.069327:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827406/real 1787827406] req@ffff985610f80b40 x1874669228236288/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827431 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827406.069337:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80b40 x1874669228236288/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827431 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827406.069344:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827406.069346:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827406.069349:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228236288:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787827406.644803:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827407.668837:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827408.692805:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827409.204877:0:92341:0:(recover.c:348:ptlrpc_recover_import()) lustre-MDT0000_UUID: recovery finished DISCONN, rc = -110 00000080:00200008:0.0:1787827409.210292:0:92345:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:0.0:1787827409.210305:0:92345:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:0.0:1787827409.210331:0:92345:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827409.210353:0:92345:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 64 lru_type: 1 00010000:00010000:0.0:1787827409.210372:0:92345:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 64 lru_type: 1 00000080:00200000:0.0:1787827409.210384:0:92345:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827409.210394:0:92345:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:0.0:1787827409.210399:0:92345:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee56d 00000080:00010000:0.0:1787827409.210410:0:92345:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee56d from it ffffab3ac1ec7cb8 00010000:00010000:0.0:1787827409.210416:0:92345:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 64 lru_type: 1 00010000:00010000:0.0:1787827409.210429:0:92345:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 65 lru_type: 1 00000080:00200000:0.0:1787827409.210466:0:92345:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:0.0:1787827409.210473:0:92345:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827409.210479:0:92345:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 65 lru_type: 1 00010000:00010000:0.0:1787827409.210487:0:92345:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 65 lru_type: 1 00000080:00200000:0.0:1787827409.210493:0:92345:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827409.210496:0:92345:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:0.0:1787827409.210498:0:92345:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee56d 00000080:00010000:0.0:1787827409.210501:0:92345:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee56d from it ffffab3ac1ec7bb0 00010000:00010000:0.0:1787827409.210503:0:92345:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 65 lru_type: 1 00010000:00010000:0.0:1787827409.210510:0:92345:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 66 lru_type: 1 00000080:00200000:0.0:1787827409.210516:0:92345:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 44 00000080:00200008:0.0:1787827409.210523:0:92345:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 100400 00000080:00200008:0.0:1787827409.210536:0:92345:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 100000 00000080:00200008:0.0:1787827409.211334:0:92345:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:0.0:1787827409.211339:0:92345:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:0.0:1787827409.211348:0:92345:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827409.211355:0:92345:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 66 lru_type: 1 00010000:00010000:0.0:1787827409.211374:0:92345:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 66 lru_type: 1 00000080:00200000:0.0:1787827409.211383:0:92345:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827409.211389:0:92345:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:0.0:1787827409.211392:0:92345:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee56d 00000080:00010000:0.0:1787827409.211398:0:92345:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee56d from it ffffab3ac1ec7cb8 00010000:00010000:0.0:1787827409.211402:0:92345:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 66 lru_type: 1 00010000:00010000:0.0:1787827409.211414:0:92345:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 67 lru_type: 1 00000080:00200008:0.0:1787827409.211444:0:92345:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:0.0:1787827409.211464:0:92345:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:0.0:1787827409.211471:0:92345:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827409.211479:0:92345:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 67 lru_type: 1 00010000:00010000:0.0:1787827409.211487:0:92345:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 67 lru_type: 1 00000080:00200000:0.0:1787827409.211493:0:92345:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827409.211497:0:92345:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:0.0:1787827409.211499:0:92345:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee56d 00000080:00010000:0.0:1787827409.211503:0:92345:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee56d from it ffffab3ac1ec7cb8 00010000:00010000:0.0:1787827409.211507:0:92345:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 67 lru_type: 1 00010000:00010000:0.0:1787827409.211514:0:92345:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 68 lru_type: 1 00000080:00200000:0.0:1787827409.211535:0:92345:0:(llite_lib.c:2903:ll_statfs()) VFS Op:sb=lustre (ffff985602bc9800) / proj=0 00000080:00000004:0.0:1787827409.211539:0:92345:0:(obd_class.h:1091:obd_statfs()) lustre-clilmv-ffff985602bc9800: age 3358, max_age 3558 00800000:00000004:0.0:1787827409.211553:0:92345:0:(obd_class.h:1091:obd_statfs()) lustre-MDT0000-mdc-ffff985602bc9800: age 3335, max_age 3558 00000100:00100000:0.0:1787827409.211607:0:92345:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607ed2e40 going to sleep for 0 seconds 00000100:00100000:0.0:1787827409.211621:0:92345:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4f00 pname:cluuid:pid:xid:nid:opc:job df:7b4e8535-9db7-485d-abf6-bda966b63320:92345:1874669228236416:192.168.203.114@tcp:41:df.0 00800000:00020000:0.0:1787827409.211638:0:92345:0:(lmv_obd.c:1471:lmv_statfs()) lustre-MDT0000-mdc-ffff985602bc9800: can't stat MDS #0: rc = -110 00800000:00000004:0.0:1787827409.219079:0:92345:0:(obd_class.h:1091:obd_statfs()) lustre-MDT0001-mdc-ffff985602bc9800: age 3358, max_age 3558 00000100:00100000:0.0:1787827409.219104:0:92345:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607ed2e40 going to sleep for 0 seconds 00000100:00100000:0.0:1787827409.219111:0:92345:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4f00 pname:cluuid:pid:xid:nid:opc:job df:7b4e8535-9db7-485d-abf6-bda966b63320:92345:1874669228236544:192.168.203.114@tcp:41:df.0 00800000:00020000:0.0:1787827409.219123:0:92345:0:(lmv_obd.c:1471:lmv_statfs()) lustre-MDT0001-mdc-ffff985602bc9800: can't stat MDS #1: rc = -110 00010000:00010000:3.0:1787827409.716937:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827410.740838:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827411.188820:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827411.188831:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827411.188836:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827411.188846:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3556 00000100:00080000:3.0:1787827411.188858:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827411.188866:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787827411.188906:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80000 x1874669228236672/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827411.188939:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827411.188944:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827411.188962:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827411.188967:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3556 00000100:00080000:3.0:1787827411.188971:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827411.188976:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827411.188987:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827411.189002:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80780 x1874669228236800/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827411.189014:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827411.189019:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827411.189021:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827411.189025:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3556 00000100:00080000:3.0:1787827411.189028:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827411.189033:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827411.189039:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827411.189057:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228236672:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787827411.189083:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f803c0 x1874669228236928/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827411.189219:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228236800:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787827411.189260:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228236928:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787827411.764827:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:1.0:1787827412.341389:0:2370:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.114:988 00000400:00000100:1.0:1787827412.341508:0:2370:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.114@tcp at host 192.168.203.114 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1787827412.341535:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:1.0:1787827412.341594:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:1.0:1787827412.341609:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000100:00100000:0.0:1787827412.341667:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827411/real 1787827412] req@ffff985610f803c0 x1874669228236928/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827436 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827412.341801:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f803c0 x1874669228236928/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827436 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827412.341838:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827412.341851:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827412.341861:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228236928:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787827412.341883:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827411/real 1787827412] req@ffff985610f80000 x1874669228236672/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827436 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827412.341909:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80000 x1874669228236672/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827436 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827412.341926:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827412.341933:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827412.341940:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228236672:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787827412.341995:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827411/real 1787827412] req@ffff985610f80780 x1874669228236800/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827436 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827412.342022:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80780 x1874669228236800/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827436 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827412.342037:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827412.342044:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827412.342050:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228236800:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787827412.788948:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827413.812779:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827414.836909:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000080:3.0:1787827415.423497:0:92364:0:(class_obd.c:336:class_handle_ioctl()) obdclass: cmd=40086685 len=0 uarg=00007fffe7160b80 00000002:00000080:3.0:1787827415.423511:0:92364:0:(mdc_request.c:2317:mdc_iocontrol()) lustre-MDT0000-mdc-ffff985602bc9800: cmd=40086685 len=576 karg=ffff985608492c00 uarg=0000000000000000 00000100:00080000:3.0:1787827415.423517:0:92364:0:(import.c:214:ptlrpc_set_import_discon()) mdc: import ffff985611182000 already not connected (conn 351, was 0): DISCONN 00000100:00080000:3.0:1787827415.423523:0:92364:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827415.423529:0:92364:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3561 00000100:00080000:3.0:1787827415.423538:0:92364:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827415.423543:0:92364:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827415.423552:0:92364:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827415.423574:0:92364:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80780 x1874669228237056/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827415.423595:0:92364:0:(recover.c:337:ptlrpc_recover_import()) lustre-MDT0000_UUID: recovery started, waiting 20 seconds 00000100:00100000:0.0:1787827415.423623:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228237056:192.168.203.114@tcp:38:lctl.0 00000800:00000100:0.0:1787827415.423677:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787827415.423779:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827415/real 1787827415] req@ffff985610f80780 x1874669228237056/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827440 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'lctl.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827415.423795:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80780 x1874669228237056/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827440 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'lctl.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827415.423803:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827415.423820:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827415.423830:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228237056:192.168.203.114@tcp:38:lctl.0 00000100:00080000:3.0:1787827415.423838:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827415.423843:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827415.423845:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827415.423851:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3561 00000100:00080000:3.0:1787827415.423855:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827415.423859:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787827415.423871:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80000 x1874669228237184/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827415.423907:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827415.423910:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827415.423911:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827415.423914:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3561 00000100:00080000:3.0:1787827415.423917:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827415.423920:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827415.423925:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827415.423930:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228237184:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787827415.423936:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f803c0 x1874669228237312/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787827415.423974:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787827415.423994:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827415/real 1787827415] req@ffff985610f80000 x1874669228237184/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827440 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827415.424006:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80000 x1874669228237184/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827440 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827415.424014:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827415.424017:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827415.424020:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228237184:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787827415.424032:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228237312:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787827415.424050:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787827415.424059:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827415/real 1787827415] req@ffff985610f803c0 x1874669228237312/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827440 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827415.424068:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f803c0 x1874669228237312/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827440 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827415.424081:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827415.424084:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827415.424087:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228237312:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787827415.860868:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827416.884811:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827417.908936:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827418.932802:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827419.956956:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827420.916848:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827420.916858:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827420.916863:0:38:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827420.916874:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3566 00000100:00080000:0.0:1787827420.916885:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827420.916893:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787827420.916935:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5e00 x1874669228237440/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827420.916973:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827420.916980:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827420.916983:0:38:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827420.916989:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3566 00000100:00080000:0.0:1787827420.916995:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827420.917001:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787827420.917014:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827420.917034:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df43c0 x1874669228237568/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827420.917050:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1787827420.917051:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228237440:192.168.203.114@tcp:250:kworker.0 00000100:00080000:0.0:1787827420.917056:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827420.917059:0:38:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827420.917093:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3566 00000100:00080000:0.0:1787827420.917099:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827420.917105:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787827420.917111:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827420.917128:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5680 x1874669228237696/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787827420.917160:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827420.917206:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827420/real 1787827420] req@ffff985607df5e00 x1874669228237440/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827445 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827420.917232:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5e00 x1874669228237440/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827445 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827420.917248:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827420.917254:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827420.917261:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228237440:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827420.917294:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228237568:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827420.917326:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827420.917340:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827420/real 1787827420] req@ffff985607df43c0 x1874669228237568/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827445 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827420.917352:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df43c0 x1874669228237568/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827445 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827420.917361:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827420.917365:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827420.917383:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228237568:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827420.917389:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228237696:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827420.917408:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827420.917423:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827420/real 1787827420] req@ffff985607df5680 x1874669228237696/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827445 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827420.917441:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5680 x1874669228237696/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827445 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827420.917449:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827420.917454:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827420.917460:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228237696:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787827420.980910:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827422.004817:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827423.028925:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827424.052923:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827425.076958:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827426.036924:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827426.036936:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827426.036942:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827426.036956:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3571 00000100:00080000:3.0:1787827426.036987:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827426.036997:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787827426.037091:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80b40 x1874669228237824/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827426.037126:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827426.037131:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827426.037134:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827426.037138:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3571 00000100:00080000:3.0:1787827426.037145:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827426.037151:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827426.037163:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827426.037179:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f830c0 x1874669228237952/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827426.037193:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827426.037198:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827426.037200:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827426.037205:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3571 00000100:00080000:3.0:1787827426.037209:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827426.037218:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787827426.037226:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228237824:192.168.203.114@tcp:250:kworker.0 00000008:00100000:3.0:1787827426.037231:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827426.037260:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81a40 x1874669228238080/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787827426.037331:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827426.037401:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827426/real 1787827426] req@ffff985610f80b40 x1874669228237824/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827451 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827426.037433:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80b40 x1874669228237824/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827451 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827426.037464:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827426.037471:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827426.037478:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228237824:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827426.037492:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228237952:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827426.037539:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827426.037563:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827426/real 1787827426] req@ffff985610f830c0 x1874669228237952/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827451 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827426.037586:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f830c0 x1874669228237952/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827451 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827426.037602:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827426.037608:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827426.037634:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228237952:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827426.037674:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228238080:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827426.037783:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827426.037845:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827426/real 1787827426] req@ffff985610f81a40 x1874669228238080/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827451 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827426.037865:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81a40 x1874669228238080/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827451 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827426.037877:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827426.037882:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827426.037887:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228238080:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787827426.100891:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827427.124870:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827428.148980:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827429.172892:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827430.196927:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827431.156843:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827431.156852:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827431.156856:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827431.156866:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3576 00000100:00080000:3.0:1787827431.156875:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827431.156881:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787827431.156915:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f812c0 x1874669228238208/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827431.156942:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827431.156946:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827431.156947:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827431.156950:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3576 00000100:00080000:3.0:1787827431.156953:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827431.156955:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827431.156963:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827431.156972:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83840 x1874669228238336/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827431.156981:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827431.156983:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827431.156985:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827431.156987:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3576 00000100:00080000:3.0:1787827431.157003:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827431.157006:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827431.157010:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827431.157018:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81680 x1874669228238464/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827431.157024:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228238208:192.168.203.114@tcp:250:kworker.0 00000800:00000100:2.0:1787827431.157132:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827431.157174:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827431/real 1787827431] req@ffff985610f812c0 x1874669228238208/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827456 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827431.157193:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f812c0 x1874669228238208/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827456 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827431.157205:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827431.157209:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827431.157214:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228238208:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827431.157222:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228238336:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827431.157263:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827431.157272:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827431/real 1787827431] req@ffff985610f83840 x1874669228238336/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827456 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827431.157280:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83840 x1874669228238336/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827456 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827431.157286:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827431.157288:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827431.157301:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228238336:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827431.157345:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228238464:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827431.157368:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827431.157381:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827431/real 1787827431] req@ffff985610f81680 x1874669228238464/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827456 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827431.157401:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81680 x1874669228238464/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827456 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827431.157411:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827431.157415:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827431.157418:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228238464:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787827431.220818:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827432.244886:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827433.268824:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827434.292808:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827435.316817:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827435.764810:0:92364:0:(recover.c:348:ptlrpc_recover_import()) lustre-MDT0000_UUID: recovery finished DISCONN, rc = -110 00000080:00200008:3.0:1787827435.768359:0:92368:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:3.0:1787827435.768369:0:92368:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:3.0:1787827435.768388:0:92368:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787827435.768404:0:92368:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 68 lru_type: 1 00010000:00010000:3.0:1787827435.768420:0:92368:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 68 lru_type: 1 00000080:00200000:3.0:1787827435.768427:0:92368:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787827435.768433:0:92368:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:3.0:1787827435.768436:0:92368:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee56d 00000080:00010000:3.0:1787827435.768442:0:92368:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee56d from it ffffab3ac1c57cb8 00010000:00010000:3.0:1787827435.768445:0:92368:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 68 lru_type: 1 00010000:00010000:3.0:1787827435.768451:0:92368:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 69 lru_type: 1 00000080:00200000:3.0:1787827435.768472:0:92368:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:3.0:1787827435.768476:0:92368:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787827435.768481:0:92368:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 69 lru_type: 1 00010000:00010000:3.0:1787827435.768487:0:92368:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 69 lru_type: 1 00000080:00200000:3.0:1787827435.768495:0:92368:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787827435.768499:0:92368:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:3.0:1787827435.768502:0:92368:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee56d 00000080:00010000:3.0:1787827435.768506:0:92368:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee56d from it ffffab3ac1c57bb0 00010000:00010000:3.0:1787827435.768509:0:92368:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 69 lru_type: 1 00010000:00010000:3.0:1787827435.768518:0:92368:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 70 lru_type: 1 00000080:00200000:3.0:1787827435.768528:0:92368:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 44 00000080:00200008:3.0:1787827435.768546:0:92368:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 100400 00000080:00200008:3.0:1787827435.768559:0:92368:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 100000 00000080:00200008:3.0:1787827435.769230:0:92368:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:3.0:1787827435.769728:0:92368:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:3.0:1787827435.769741:0:92368:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787827435.769752:0:92368:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 70 lru_type: 1 00010000:00010000:3.0:1787827435.769765:0:92368:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 70 lru_type: 1 00000080:00200000:3.0:1787827435.769785:0:92368:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787827435.769791:0:92368:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:3.0:1787827435.769794:0:92368:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee56d 00000080:00010000:3.0:1787827435.769800:0:92368:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee56d from it ffffab3ac1c57cb8 00010000:00010000:3.0:1787827435.769804:0:92368:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 70 lru_type: 1 00010000:00010000:3.0:1787827435.769813:0:92368:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 71 lru_type: 1 00000080:00200008:3.0:1787827435.769851:0:92368:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:3.0:1787827435.769857:0:92368:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:3.0:1787827435.769864:0:92368:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787827435.769871:0:92368:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 71 lru_type: 1 00010000:00010000:3.0:1787827435.769880:0:92368:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 71 lru_type: 1 00000080:00200000:3.0:1787827435.769886:0:92368:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787827435.769890:0:92368:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:3.0:1787827435.769893:0:92368:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee56d 00000080:00010000:3.0:1787827435.769897:0:92368:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee56d from it ffffab3ac1c57cb8 00010000:00010000:3.0:1787827435.769901:0:92368:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 71 lru_type: 1 00010000:00010000:3.0:1787827435.769909:0:92368:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 72 lru_type: 1 00000080:00200000:3.0:1787827435.769929:0:92368:0:(llite_lib.c:2903:ll_statfs()) VFS Op:sb=lustre (ffff985602bc9800) / proj=0 00000080:00000004:3.0:1787827435.769933:0:92368:0:(obd_class.h:1091:obd_statfs()) lustre-clilmv-ffff985602bc9800: age 3358, max_age 3585 00800000:00000004:3.0:1787827435.769942:0:92368:0:(obd_class.h:1091:obd_statfs()) lustre-MDT0000-mdc-ffff985602bc9800: age 3335, max_age 3585 00000100:00100000:3.0:1787827435.769990:0:92368:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bc00 going to sleep for 0 seconds 00000100:00100000:3.0:1787827435.770003:0:92368:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job df:7b4e8535-9db7-485d-abf6-bda966b63320:92368:1874669228238592:192.168.203.114@tcp:41:df.0 00800000:00020000:3.0:1787827435.770020:0:92368:0:(lmv_obd.c:1471:lmv_statfs()) lustre-MDT0000-mdc-ffff985602bc9800: can't stat MDS #0: rc = -110 00800000:00000004:3.0:1787827435.770025:0:92368:0:(obd_class.h:1091:obd_statfs()) lustre-MDT0001-mdc-ffff985602bc9800: age 3358, max_age 3585 00000100:00100000:3.0:1787827435.770038:0:92368:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bc00 going to sleep for 0 seconds 00000100:00100000:3.0:1787827435.770045:0:92368:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job df:7b4e8535-9db7-485d-abf6-bda966b63320:92368:1874669228238720:192.168.203.114@tcp:41:df.0 00800000:00020000:3.0:1787827435.770054:0:92368:0:(lmv_obd.c:1471:lmv_statfs()) lustre-MDT0001-mdc-ffff985602bc9800: can't stat MDS #1: rc = -110 00000100:00080000:2.0:1787827436.276845:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827436.276854:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827436.276858:0:38:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827436.276867:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3581 00000100:00080000:2.0:1787827436.276879:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827436.276888:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787827436.276953:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8780 x1874669228238848/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827436.277037:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827436.277040:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827436.277042:0:38:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827436.277046:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3581 00000100:00080000:2.0:1787827436.277050:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827436.277054:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787827436.277064:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827436.277078:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db480 x1874669228238976/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827436.277087:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228238848:192.168.203.114@tcp:250:kworker.0 00000100:00080000:2.0:1787827436.277092:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827436.277097:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827436.277099:0:38:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827436.277103:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3581 00000100:00080000:2.0:1787827436.277106:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827436.277109:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787827436.277124:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827436.277152:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d9680 x1874669228239104/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787827436.277169:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787827436.277231:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827436/real 1787827436] req@ffff9856140d8780 x1874669228238848/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827461 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827436.277251:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8780 x1874669228238848/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827461 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827436.277263:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827436.277268:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827436.277273:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228238848:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787827436.277299:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228238976:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787827436.277324:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787827436.277352:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827436/real 1787827436] req@ffff9856140db480 x1874669228238976/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827461 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827436.277380:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db480 x1874669228238976/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827461 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827436.277390:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827436.277394:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827436.277398:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228238976:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787827436.277407:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228239104:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787827436.277452:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787827436.277483:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827436/real 1787827436] req@ffff9856140d9680 x1874669228239104/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827461 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827436.277506:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d9680 x1874669228239104/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827461 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827436.277522:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827436.277528:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827436.277535:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228239104:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787827436.340934:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827437.364825:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827438.388878:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827439.412837:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827440.436886:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827441.396842:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827441.396852:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827441.396856:0:38:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827441.396866:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3587 00000100:00080000:2.0:1787827441.396876:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827441.396882:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787827441.396917:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d83c0 x1874669228239232/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827441.396958:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827441.396988:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827441.396990:0:38:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827441.396994:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3587 00000100:00080000:2.0:1787827441.396997:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827441.397000:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787827441.397009:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827441.397019:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8b40 x1874669228239360/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827441.397028:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827441.397032:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:0.0:1787827441.397034:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228239232:192.168.203.114@tcp:250:kworker.0 00000100:00080000:2.0:1787827441.397034:0:38:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827441.397040:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3587 00000100:00080000:2.0:1787827441.397056:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827441.397059:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787827441.397062:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827441.397070:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140da940 x1874669228239488/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787827441.397121:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787827441.397150:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827441/real 1787827441] req@ffff9856140d83c0 x1874669228239232/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827466 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827441.397163:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d83c0 x1874669228239232/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827466 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827441.397171:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827441.397174:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827441.397188:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228239232:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787827441.397206:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228239360:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787827441.397220:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787827441.397228:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827441/real 1787827441] req@ffff9856140d8b40 x1874669228239360/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827466 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827441.397235:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8b40 x1874669228239360/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827466 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827441.397240:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827441.397242:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827441.397245:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228239360:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787827441.397248:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228239488:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787827441.397258:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787827441.397263:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827441/real 1787827441] req@ffff9856140da940 x1874669228239488/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827466 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827441.397270:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140da940 x1874669228239488/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827466 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827441.397280:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827441.397282:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827441.397284:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228239488:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787827441.460938:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000080:3.0:1787827441.982513:0:92387:0:(class_obd.c:336:class_handle_ioctl()) obdclass: cmd=40086685 len=0 uarg=00007ffcffdeccc0 00000002:00000080:3.0:1787827441.982532:0:92387:0:(mdc_request.c:2317:mdc_iocontrol()) lustre-MDT0000-mdc-ffff985602bc9800: cmd=40086685 len=576 karg=ffff985608491400 uarg=0000000000000000 00000100:00080000:3.0:1787827441.982538:0:92387:0:(import.c:214:ptlrpc_set_import_discon()) mdc: import ffff985611182000 already not connected (conn 357, was 0): DISCONN 00000100:00080000:3.0:1787827441.982545:0:92387:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827441.982553:0:92387:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3592 00000100:00080000:3.0:1787827441.982565:0:92387:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827441.982572:0:92387:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827441.982582:0:92387:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827441.982617:0:92387:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81e00 x1874669228239616/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827441.982647:0:92387:0:(recover.c:337:ptlrpc_recover_import()) lustre-MDT0000_UUID: recovery started, waiting 20 seconds 00000100:00100000:0.0:1787827441.982789:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228239616:192.168.203.114@tcp:38:lctl.0 00000800:00000100:0.0:1787827441.982857:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787827441.982883:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827441/real 1787827441] req@ffff985610f81e00 x1874669228239616/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827466 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'lctl.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827441.982908:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81e00 x1874669228239616/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827466 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'lctl.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827441.982934:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827441.982940:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827441.982952:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228239616:192.168.203.114@tcp:38:lctl.0 00010000:00010000:0.0:1787827442.484815:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827443.508967:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827444.532863:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827445.556808:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827446.516790:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827446.516798:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827446.516802:0:38:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827446.516811:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3592 00000100:00080000:2.0:1787827446.516821:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827446.516827:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787827446.516859:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db0c0 x1874669228239744/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827446.516894:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827446.516898:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827446.516900:0:38:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827446.516905:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3592 00000100:00080000:2.0:1787827446.516908:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827446.516913:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787827446.516924:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827446.516941:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140dad00 x1874669228239872/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827446.516951:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827446.516954:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827446.516955:0:38:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827446.516958:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3592 00000100:00080000:2.0:1787827446.516961:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827446.516964:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787827446.516968:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827446.516978:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228239744:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827446.516979:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8f00 x1874669228240000/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787827446.517073:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787827446.517116:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827446/real 1787827446] req@ffff9856140db0c0 x1874669228239744/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827471 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827446.517132:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db0c0 x1874669228239744/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827471 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827446.517141:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827446.517146:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827446.517149:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228239744:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787827446.517155:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228239872:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787827446.517173:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787827446.517181:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827446/real 1787827446] req@ffff9856140dad00 x1874669228239872/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827471 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827446.517190:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140dad00 x1874669228239872/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827471 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827446.517195:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827446.517197:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827446.517212:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228239872:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787827446.517241:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228240000:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787827446.517258:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787827446.517266:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827446/real 1787827446] req@ffff9856140d8f00 x1874669228240000/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827471 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827446.517275:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8f00 x1874669228240000/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827471 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827446.517281:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827446.517283:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827446.517285:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228240000:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787827446.580874:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827447.604893:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827448.628828:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827449.652810:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827450.676915:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827451.636814:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827451.636825:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827451.636829:0:38:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827451.636840:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3597 00000100:00080000:2.0:1787827451.636863:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827451.636872:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787827451.636919:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140dbc00 x1874669228240128/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827451.636949:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827451.636954:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827451.636956:0:38:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827451.636960:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3597 00000100:00080000:2.0:1787827451.636964:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827451.636968:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787827451.636978:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827451.636994:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228240128:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827451.636995:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db840 x1874669228240256/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827451.637012:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827451.637016:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827451.637018:0:38:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827451.637023:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3597 00000100:00080000:2.0:1787827451.637028:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827451.637032:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787827451.637038:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827451.637051:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140da1c0 x1874669228240384/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787827451.637060:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787827451.637103:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827451/real 1787827451] req@ffff9856140dbc00 x1874669228240128/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827476 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827451.637121:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140dbc00 x1874669228240128/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827476 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827451.637129:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827451.637132:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827451.637134:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228240128:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787827451.637153:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228240256:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787827451.637170:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787827451.637179:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827451/real 1787827451] req@ffff9856140db840 x1874669228240256/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827476 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827451.637187:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db840 x1874669228240256/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827476 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827451.637192:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827451.637194:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827451.637208:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228240256:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787827451.637212:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228240384:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787827451.637225:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787827451.637232:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827451/real 1787827451] req@ffff9856140da1c0 x1874669228240384/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827476 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827451.637241:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140da1c0 x1874669228240384/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827476 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827451.637248:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827451.637251:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827451.637253:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228240384:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787827451.700857:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827452.724793:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827453.748886:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827454.772786:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827455.796814:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827456.756780:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827456.756788:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827456.756791:0:38:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827456.756800:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3602 00000100:00080000:0.0:1787827456.756810:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827456.756832:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787827456.756865:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5a40 x1874669228240512/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827456.756891:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827456.756895:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827456.756897:0:38:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827456.756901:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3602 00000100:00080000:0.0:1787827456.756905:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827456.756910:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787827456.756921:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827456.756939:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228240512:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787827456.756940:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6580 x1874669228240640/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827456.756958:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827456.756962:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827456.756964:0:38:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827456.756968:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3602 00000100:00080000:0.0:1787827456.756973:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827456.756978:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787827456.756984:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827456.756998:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df70c0 x1874669228240768/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787827456.757008:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827456.757068:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827456/real 1787827456] req@ffff985607df5a40 x1874669228240512/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827481 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827456.757087:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5a40 x1874669228240512/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827481 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827456.757096:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827456.757100:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827456.757105:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228240512:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827456.757153:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228240640:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827456.757175:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827456.757188:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827456/real 1787827456] req@ffff985607df6580 x1874669228240640/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827481 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827456.757197:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6580 x1874669228240640/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827481 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827456.757203:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827456.757205:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827456.757237:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228240640:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827456.757245:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228240768:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827456.757261:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827456.757269:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827456/real 1787827456] req@ffff985607df70c0 x1874669228240768/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827481 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827456.757280:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df70c0 x1874669228240768/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827481 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827456.757288:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827456.757291:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827456.757294:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228240768:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787827456.820871:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827457.844832:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827458.868879:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827459.892876:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827460.916902:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827461.876827:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827461.876836:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827461.876840:0:38:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827461.876847:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3607 00000100:00080000:2.0:1787827461.876857:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827461.876864:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787827461.876902:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140da1c0 x1874669228240896/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827461.876939:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827461.876942:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827461.876943:0:38:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827461.876946:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3607 00000100:00080000:2.0:1787827461.876949:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827461.876954:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787827461.876964:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827461.876980:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db840 x1874669228241024/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827461.876994:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827461.877002:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:0.0:1787827461.877003:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228240896:192.168.203.114@tcp:250:kworker.0 00000100:00080000:2.0:1787827461.877006:0:38:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827461.877020:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3607 00000100:00080000:2.0:1787827461.877024:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827461.877027:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787827461.877032:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827461.877046:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140dbc00 x1874669228241152/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787827461.877098:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787827461.877138:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827461/real 1787827461] req@ffff9856140da1c0 x1874669228240896/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827486 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827461.877152:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140da1c0 x1874669228240896/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827486 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827461.877162:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827461.877165:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827461.877169:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228240896:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787827461.877194:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228241024:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787827461.877211:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787827461.877219:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827461/real 1787827461] req@ffff9856140db840 x1874669228241024/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827486 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827461.877228:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db840 x1874669228241024/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827486 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827461.877233:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827461.877235:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827461.877251:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228241024:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787827461.877255:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228241152:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787827461.877269:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787827461.877277:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827461/real 1787827461] req@ffff9856140dbc00 x1874669228241152/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827486 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827461.877285:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140dbc00 x1874669228241152/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827486 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827461.877292:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827461.877295:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827461.877298:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228241152:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787827461.940832:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827461.988788:0:92387:0:(recover.c:348:ptlrpc_recover_import()) lustre-MDT0000_UUID: recovery finished DISCONN, rc = -110 00000080:00200008:3.0:1787827461.992439:0:92391:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:3.0:1787827461.992448:0:92391:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:3.0:1787827461.992466:0:92391:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787827461.992483:0:92391:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 72 lru_type: 1 00010000:00010000:3.0:1787827461.992495:0:92391:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 72 lru_type: 1 00000080:00200000:3.0:1787827461.992502:0:92391:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787827461.992508:0:92391:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:3.0:1787827461.992511:0:92391:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee56d 00000080:00010000:3.0:1787827461.992517:0:92391:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee56d from it ffffab3ac1f47cb8 00010000:00010000:3.0:1787827461.992520:0:92391:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 72 lru_type: 1 00010000:00010000:3.0:1787827461.992526:0:92391:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 73 lru_type: 1 00000080:00200000:3.0:1787827461.992546:0:92391:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:3.0:1787827461.992551:0:92391:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787827461.992554:0:92391:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 73 lru_type: 1 00010000:00010000:3.0:1787827461.992560:0:92391:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 73 lru_type: 1 00000080:00200000:3.0:1787827461.992565:0:92391:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787827461.992567:0:92391:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:3.0:1787827461.992569:0:92391:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee56d 00000080:00010000:3.0:1787827461.992571:0:92391:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee56d from it ffffab3ac1f47bb0 00010000:00010000:3.0:1787827461.992573:0:92391:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 73 lru_type: 1 00010000:00010000:3.0:1787827461.992578:0:92391:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 74 lru_type: 1 00000080:00200000:3.0:1787827461.992583:0:92391:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 44 00000080:00200008:3.0:1787827461.992589:0:92391:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 100400 00000080:00200008:3.0:1787827461.992598:0:92391:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 100000 00000080:00200008:3.0:1787827461.993211:0:92391:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:3.0:1787827461.993216:0:92391:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:3.0:1787827461.993222:0:92391:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787827461.993229:0:92391:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 74 lru_type: 1 00010000:00010000:3.0:1787827461.993235:0:92391:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 74 lru_type: 1 00000080:00200000:3.0:1787827461.993240:0:92391:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787827461.993243:0:92391:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:3.0:1787827461.993245:0:92391:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee56d 00000080:00010000:3.0:1787827461.993248:0:92391:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee56d from it ffffab3ac1f47cb8 00010000:00010000:3.0:1787827461.993251:0:92391:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 74 lru_type: 1 00010000:00010000:3.0:1787827461.993256:0:92391:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 75 lru_type: 1 00000080:00200008:3.0:1787827461.993272:0:92391:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:3.0:1787827461.993275:0:92391:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:3.0:1787827461.993278:0:92391:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787827461.993282:0:92391:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 75 lru_type: 1 00010000:00010000:3.0:1787827461.993288:0:92391:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 75 lru_type: 1 00000080:00200000:3.0:1787827461.993293:0:92391:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787827461.993295:0:92391:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:3.0:1787827461.993297:0:92391:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee56d 00000080:00010000:3.0:1787827461.993299:0:92391:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee56d from it ffffab3ac1f47cb8 00010000:00010000:3.0:1787827461.993301:0:92391:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 75 lru_type: 1 00010000:00010000:3.0:1787827461.993305:0:92391:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 76 lru_type: 1 00000080:00200000:3.0:1787827461.993316:0:92391:0:(llite_lib.c:2903:ll_statfs()) VFS Op:sb=lustre (ffff985602bc9800) / proj=0 00000080:00000004:3.0:1787827461.993320:0:92391:0:(obd_class.h:1091:obd_statfs()) lustre-clilmv-ffff985602bc9800: age 3358, max_age 3611 00800000:00000004:3.0:1787827461.993328:0:92391:0:(obd_class.h:1091:obd_statfs()) lustre-MDT0000-mdc-ffff985602bc9800: age 3335, max_age 3611 00000100:00100000:3.0:1787827461.993357:0:92391:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5b180 going to sleep for 0 seconds 00000100:00100000:3.0:1787827461.993366:0:92391:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job df:7b4e8535-9db7-485d-abf6-bda966b63320:92391:1874669228241280:192.168.203.114@tcp:41:df.0 00800000:00020000:3.0:1787827461.993376:0:92391:0:(lmv_obd.c:1471:lmv_statfs()) lustre-MDT0000-mdc-ffff985602bc9800: can't stat MDS #0: rc = -110 00800000:00000004:3.0:1787827461.993379:0:92391:0:(obd_class.h:1091:obd_statfs()) lustre-MDT0001-mdc-ffff985602bc9800: age 3358, max_age 3611 00000100:00100000:3.0:1787827461.993385:0:92391:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5b180 going to sleep for 0 seconds 00000100:00100000:3.0:1787827461.993387:0:92391:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job df:7b4e8535-9db7-485d-abf6-bda966b63320:92391:1874669228241408:192.168.203.114@tcp:41:df.0 00800000:00020000:3.0:1787827461.993390:0:92391:0:(lmv_obd.c:1471:lmv_statfs()) lustre-MDT0001-mdc-ffff985602bc9800: can't stat MDS #1: rc = -110 00010000:00010000:0.0:1787827462.964856:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827463.988817:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827465.012843:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827466.036848:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827466.996869:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827466.996878:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827466.996883:0:38:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827466.996890:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3612 00000100:00080000:0.0:1787827466.996900:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827466.996908:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787827466.996944:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5e00 x1874669228241536/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827466.996974:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827466.996977:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827466.996978:0:38:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827466.996981:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3612 00000100:00080000:0.0:1787827466.996985:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827466.996989:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787827466.996998:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827466.997013:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4f00 x1874669228241664/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827466.997022:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228241536:192.168.203.114@tcp:250:kworker.0 00000100:00080000:0.0:1787827466.997039:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827466.997044:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827466.997046:0:38:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827466.997052:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3612 00000100:00080000:0.0:1787827466.997057:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827466.997061:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787827466.997065:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827466.997078:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df61c0 x1874669228241792/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787827466.997109:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827466.997148:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827466/real 1787827466] req@ffff985607df5e00 x1874669228241536/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827491 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827466.997163:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5e00 x1874669228241536/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827491 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827466.997172:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827466.997176:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827466.997179:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228241536:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827466.997204:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228241664:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827466.997219:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827466.997228:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827466/real 1787827466] req@ffff985607df4f00 x1874669228241664/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827491 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827466.997236:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4f00 x1874669228241664/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827491 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827466.997241:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827466.997243:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827466.997245:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228241664:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827466.997249:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228241792:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827466.997260:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827466.997265:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827466/real 1787827466] req@ffff985607df61c0 x1874669228241792/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827491 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827466.997273:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df61c0 x1874669228241792/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827491 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827466.997278:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827466.997280:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827466.997281:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228241792:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787827467.060875:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827468.084833:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827469.108829:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827470.132786:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827471.156899:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827472.116835:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827472.116846:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827472.116851:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827472.116862:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3617 00000100:00080000:2.0:1787827472.116875:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827472.116883:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787827472.116931:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db0c0 x1874669228241920/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827472.116960:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827472.116963:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827472.116964:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827472.116967:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3617 00000100:00080000:2.0:1787827472.116975:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827472.116979:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787827472.116989:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827472.117001:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228241920:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827472.117002:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140da940 x1874669228242048/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827472.117016:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827472.117019:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827472.117020:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827472.117023:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3617 00000100:00080000:2.0:1787827472.117027:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827472.117030:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787827472.117032:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827472.117040:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8b40 x1874669228242176/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787827472.117088:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787827472.117128:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827472/real 1787827472] req@ffff9856140db0c0 x1874669228241920/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827497 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827472.117148:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db0c0 x1874669228241920/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827497 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827472.117160:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827472.117165:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827472.117170:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228241920:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787827472.117200:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228242048:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787827472.117221:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787827472.117231:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827472/real 1787827472] req@ffff9856140da940 x1874669228242048/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827497 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827472.117240:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140da940 x1874669228242048/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827497 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827472.117246:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827472.117248:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827472.117251:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228242048:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787827472.117257:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228242176:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787827472.117268:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787827472.117274:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827472/real 1787827472] req@ffff9856140d8b40 x1874669228242176/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827497 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827472.117282:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8b40 x1874669228242176/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827497 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827472.117288:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827472.117289:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827472.117291:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228242176:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787827472.180842:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827473.204891:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827474.228823:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827475.252801:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827476.276847:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827477.236748:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827477.236757:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827477.236760:0:38:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827477.236768:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3622 00000100:00080000:2.0:1787827477.236776:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827477.236782:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787827477.236812:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d83c0 x1874669228242304/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827477.236834:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827477.236837:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827477.236838:0:38:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827477.236841:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3622 00000100:00080000:2.0:1787827477.236843:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827477.236845:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787827477.236853:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827477.236863:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d9680 x1874669228242432/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827477.236872:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827477.236876:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:0.0:1787827477.236878:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228242304:192.168.203.114@tcp:250:kworker.0 00000100:00080000:2.0:1787827477.236879:0:38:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827477.236886:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3622 00000100:00080000:2.0:1787827477.236889:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827477.236893:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787827477.236896:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827477.236906:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db480 x1874669228242560/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827477.236968:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228242432:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787827477.236998:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228242560:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787827477.300825:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827478.324819:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827479.348855:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:3.0:1787827479.477010:0:2371:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.114:988 00000400:00000100:3.0:1787827479.477047:0:2371:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.114@tcp at host 192.168.203.114 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:3.0:1787827479.477070:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:3.0:1787827479.477117:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:3.0:1787827479.477127:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000100:00100000:0.0:1787827479.477161:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827477/real 1787827479] req@ffff9856140d9680 x1874669228242432/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827502 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827479.477186:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d9680 x1874669228242432/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827502 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827479.477197:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827479.477203:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827479.477208:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228242432:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787827479.477216:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827477/real 1787827479] req@ffff9856140db480 x1874669228242560/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827502 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827479.477227:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db480 x1874669228242560/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827502 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827479.477234:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827479.477238:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827479.477242:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228242560:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787827479.477249:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827477/real 1787827479] req@ffff9856140d83c0 x1874669228242304/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827502 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827479.477260:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d83c0 x1874669228242304/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827502 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827479.477268:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827479.477272:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827479.477275:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228242304:192.168.203.114@tcp:250:kworker.0 00010000:00010000:0.0:1787827480.372810:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827481.396814:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827482.356806:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827482.356817:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827482.356822:0:38:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827482.356833:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3627 00000100:00080000:2.0:1787827482.356844:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827482.356852:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787827482.356889:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8780 x1874669228242688/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827482.356921:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827482.356925:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827482.356927:0:38:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827482.356931:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3627 00000100:00080000:2.0:1787827482.356935:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827482.356941:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787827482.356952:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827482.356961:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228242688:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827482.356969:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d9a40 x1874669228242816/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827482.356982:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827482.356986:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827482.356989:0:38:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827482.356994:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3627 00000100:00080000:2.0:1787827482.356999:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827482.357002:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787827482.357006:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000800:00000100:0.0:1787827482.357017:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827482.357019:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d83c0 x1874669228242944/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827482.357055:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827482/real 1787827482] req@ffff9856140d8780 x1874669228242688/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827507 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827482.357069:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8780 x1874669228242688/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827507 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827482.357082:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827482.357085:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827482.357088:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228242688:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787827482.357106:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228242816:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787827482.357122:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787827482.357131:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827482/real 1787827482] req@ffff9856140d9a40 x1874669228242816/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827507 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827482.357139:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d9a40 x1874669228242816/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827507 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827482.357144:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827482.357147:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827482.357149:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228242816:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787827482.357152:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228242944:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787827482.357162:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787827482.357168:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827482/real 1787827482] req@ffff9856140d83c0 x1874669228242944/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827507 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827482.357175:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d83c0 x1874669228242944/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827507 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827482.357181:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827482.357183:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827482.357185:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228242944:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787827482.420841:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827483.444908:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827484.468916:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827485.492811:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827486.516837:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827487.476840:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827487.476851:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827487.476855:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827487.476866:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3633 00000100:00080000:0.0:1787827487.476878:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827487.476887:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787827487.476930:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7840 x1874669228243072/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827487.476969:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827487.476975:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827487.476978:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827487.476986:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3633 00000100:00080000:0.0:1787827487.476993:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827487.477000:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787827487.477012:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228243072:192.168.203.114@tcp:250:kworker.0 00000008:00100000:0.0:1787827487.477013:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827487.477036:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df52c0 x1874669228243200/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827487.477050:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827487.477056:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827487.477059:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827487.477065:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3633 00000100:00080000:0.0:1787827487.477071:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827487.477078:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000800:00000100:2.0:1787827487.477083:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000008:00100000:0.0:1787827487.477090:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827487.477134:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6940 x1874669228243328/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827487.477147:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827487/real 1787827487] req@ffff985607df7840 x1874669228243072/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827512 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827487.477177:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7840 x1874669228243072/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827512 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827487.477190:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827487.477195:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827487.477200:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228243072:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827487.477225:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228243200:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827487.477248:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827487.477260:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827487/real 1787827487] req@ffff985607df52c0 x1874669228243200/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827512 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827487.477272:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df52c0 x1874669228243200/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827512 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827487.477287:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827487.477291:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827487.477294:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228243200:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827487.477300:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228243328:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827487.477315:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827487.477324:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827487/real 1787827487] req@ffff985607df6940 x1874669228243328/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827512 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827487.477335:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6940 x1874669228243328/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827512 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827487.477343:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827487.477345:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827487.477348:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228243328:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787827487.540919:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827488.564862:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827489.588843:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827490.612841:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827491.636878:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827492.596877:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827492.596891:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827492.596897:0:38:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827492.596912:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3638 00000100:00080000:2.0:1787827492.596929:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827492.596940:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787827492.597001:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8b40 x1874669228243456/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827492.597046:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827492.597052:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827492.597054:0:38:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827492.597060:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3638 00000100:00080000:2.0:1787827492.597066:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827492.597072:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787827492.597086:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827492.597103:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140da940 x1874669228243584/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827492.597120:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1787827492.597144:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228243456:192.168.203.114@tcp:250:kworker.0 00000100:00080000:2.0:1787827492.597161:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827492.597164:0:38:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827492.597171:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3638 00000100:00080000:2.0:1787827492.597175:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827492.597180:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787827492.597187:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827492.597215:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db0c0 x1874669228243712/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787827492.597260:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787827492.597322:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827492/real 1787827492] req@ffff9856140d8b40 x1874669228243456/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827517 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827492.597357:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8b40 x1874669228243456/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827517 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827492.597379:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827492.597397:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827492.597403:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228243456:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787827492.597438:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228243584:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787827492.597472:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787827492.597488:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827492/real 1787827492] req@ffff9856140da940 x1874669228243584/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827517 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827492.597503:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140da940 x1874669228243584/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827517 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827492.597513:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827492.597518:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827492.597522:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228243584:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787827492.597529:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228243712:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787827492.597548:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787827492.597559:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827492/real 1787827492] req@ffff9856140db0c0 x1874669228243712/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827517 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827492.597607:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db0c0 x1874669228243712/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827517 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827492.597612:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827492.597615:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827492.597617:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228243712:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787827492.660929:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827493.684902:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827494.708872:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827495.732859:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827496.756845:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827497.716839:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827497.716850:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827497.716855:0:38:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827497.716866:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3643 00000100:00080000:0.0:1787827497.716878:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827497.716886:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787827497.716930:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4780 x1874669228243840/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827497.716959:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827497.716963:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827497.716966:0:38:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827497.716969:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3643 00000100:00080000:0.0:1787827497.716973:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827497.716977:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787827497.716988:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827497.717007:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4b40 x1874669228243968/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827497.717010:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228243840:192.168.203.114@tcp:250:kworker.0 00000100:00080000:0.0:1787827497.717032:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827497.717037:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827497.717040:0:38:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827497.717046:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3643 00000100:00080000:0.0:1787827497.717050:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827497.717054:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787827497.717059:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827497.717074:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7c00 x1874669228244096/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787827497.717112:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827497.717151:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827497/real 1787827497] req@ffff985607df4780 x1874669228243840/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827522 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827497.717169:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4780 x1874669228243840/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827522 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827497.717180:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827497.717184:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827497.717188:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228243840:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827497.717213:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228243968:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827497.717233:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827497.717242:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827497/real 1787827497] req@ffff985607df4b40 x1874669228243968/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827522 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827497.717252:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4b40 x1874669228243968/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827522 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827497.717258:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827497.717261:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827497.717264:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228243968:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827497.717267:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228244096:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827497.717279:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827497.717286:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827497/real 1787827497] req@ffff985607df7c00 x1874669228244096/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827522 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827497.717295:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7c00 x1874669228244096/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827522 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827497.717301:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827497.717303:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827497.717305:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228244096:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787827497.780997:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827498.804846:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827499.828819:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827500.852895:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827501.876981:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827502.836837:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827502.836848:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827502.836853:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827502.836864:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3648 00000100:00080000:1.0:1787827502.836875:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827502.836884:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787827502.836946:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0780 x1874669228244224/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827502.836971:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827502.836974:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827502.836975:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827502.836978:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3648 00000100:00080000:1.0:1787827502.836980:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827502.836984:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787827502.836992:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827502.837020:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228244224:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787827502.837021:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0000 x1874669228244352/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827502.837039:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827502.837043:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827502.837046:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827502.837051:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3648 00000100:00080000:1.0:1787827502.837056:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827502.837062:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787827502.837068:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000800:00000100:2.0:1787827502.837081:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827502.837086:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3c00 x1874669228244480/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827502.837120:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827502/real 1787827502] req@ffff9856085a0780 x1874669228244224/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827527 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827502.837135:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0780 x1874669228244224/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827527 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827502.837159:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827502.837163:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827502.837165:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228244224:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827502.837184:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228244352:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827502.837199:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827502.837207:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827502/real 1787827502] req@ffff9856085a0000 x1874669228244352/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827527 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827502.837215:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0000 x1874669228244352/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827527 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827502.837220:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827502.837222:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827502.837224:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228244352:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827502.837228:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228244480:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827502.837264:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827502.837275:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827502/real 1787827502] req@ffff9856085a3c00 x1874669228244480/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827527 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827502.837304:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3c00 x1874669228244480/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827527 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827502.837313:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827502.837317:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827502.837320:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228244480:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787827502.900905:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827503.924893:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827504.948874:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827505.972932:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827506.996862:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827507.956790:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827507.956799:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827507.956804:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827507.956814:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3653 00000100:00080000:1.0:1787827507.956825:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827507.956832:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787827507.956866:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2d00 x1874669228244608/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827507.956897:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827507.956901:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827507.956903:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827507.956907:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3653 00000100:00080000:1.0:1787827507.956910:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827507.956914:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787827507.956922:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827507.956934:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0b40 x1874669228244736/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827507.956946:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827507.956950:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827507.956952:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827507.956957:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3653 00000100:00080000:1.0:1787827507.956972:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00100000:2.0:1787827507.956978:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228244608:192.168.203.114@tcp:250:kworker.0 00000100:00080000:1.0:1787827507.956979:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787827507.956988:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827507.957000:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3c00 x1874669228244864/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787827507.957041:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827507.957070:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827507/real 1787827507] req@ffff9856085a2d00 x1874669228244608/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827532 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827507.957083:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2d00 x1874669228244608/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827532 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827507.957091:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827507.957094:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827507.957097:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228244608:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827507.957101:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228244736:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827507.957115:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827507.957122:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827507/real 1787827507] req@ffff9856085a0b40 x1874669228244736/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827532 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827507.957130:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0b40 x1874669228244736/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827532 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827507.957135:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827507.957138:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827507.957140:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228244736:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827507.957158:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228244864:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827507.957168:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827507.957174:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827507/real 1787827507] req@ffff9856085a3c00 x1874669228244864/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827532 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827507.957181:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3c00 x1874669228244864/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827532 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827507.957186:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827507.957188:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827507.957190:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228244864:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787827508.020858:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827509.044828:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827510.068901:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827511.092788:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827512.116855:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827513.076833:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827513.076844:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827513.076849:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827513.076859:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3658 00000100:00080000:3.0:1787827513.076871:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827513.076880:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787827513.076921:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82580 x1874669228244992/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827513.076948:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827513.076952:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827513.076955:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827513.076959:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3658 00000100:00080000:3.0:1787827513.076963:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827513.076968:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827513.076981:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827513.076997:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83480 x1874669228245120/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827513.077015:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228244992:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787827513.077022:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827513.077031:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827513.077033:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827513.077038:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3658 00000100:00080000:3.0:1787827513.077043:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827513.077047:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827513.077052:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827513.077063:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f821c0 x1874669228245248/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787827513.077099:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827513.077136:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827513/real 1787827513] req@ffff985610f82580 x1874669228244992/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827538 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827513.077166:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82580 x1874669228244992/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827538 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827513.077178:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827513.077183:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827513.077188:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228244992:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827513.077220:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228245120:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827513.077245:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827513.077258:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827513/real 1787827513] req@ffff985610f83480 x1874669228245120/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827538 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827513.077271:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83480 x1874669228245120/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827538 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827513.077280:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827513.077284:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827513.077287:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228245120:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827513.077295:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f821c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228245248:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827513.077314:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827513.077326:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827513/real 1787827513] req@ffff985610f821c0 x1874669228245248/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827538 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827513.077339:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f821c0 x1874669228245248/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827538 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827513.077347:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827513.077351:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827513.077355:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f821c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228245248:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787827513.140862:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827514.164873:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827515.188958:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827516.212909:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827517.237002:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827518.196804:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827518.196813:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827518.196818:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827518.196827:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3663 00000100:00080000:2.0:1787827518.196843:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827518.196850:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787827518.196884:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140dad00 x1874669228245376/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827518.196910:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827518.196913:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827518.196915:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827518.196918:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3663 00000100:00080000:2.0:1787827518.196921:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827518.196924:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787827518.196933:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827518.196961:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8f00 x1874669228245504/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827518.196970:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827518.196972:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827518.196974:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827518.196979:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3663 00000100:00100000:0.0:1787827518.196980:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228245376:192.168.203.114@tcp:250:kworker.0 00000100:00080000:2.0:1787827518.196982:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827518.196988:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787827518.196994:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827518.197002:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140dbc00 x1874669228245632/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787827518.197068:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787827518.197112:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827518/real 1787827518] req@ffff9856140dad00 x1874669228245376/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827543 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827518.197132:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140dad00 x1874669228245376/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827543 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827518.197144:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827518.197150:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827518.197154:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228245376:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787827518.197179:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228245504:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787827518.197203:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787827518.197223:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827518/real 1787827518] req@ffff9856140d8f00 x1874669228245504/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827543 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827518.197237:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8f00 x1874669228245504/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827543 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827518.197245:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827518.197249:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827518.197252:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228245504:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787827518.197259:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228245632:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787827518.197280:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787827518.197291:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827518/real 1787827518] req@ffff9856140dbc00 x1874669228245632/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827543 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827518.197332:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140dbc00 x1874669228245632/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827543 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827518.197341:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827518.197345:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827518.197348:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228245632:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787827518.260840:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827519.284807:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827520.308898:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827521.332913:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827522.356943:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827523.316815:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827523.316825:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827523.316830:0:38:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827523.316841:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3668 00000100:00080000:2.0:1787827523.316853:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827523.316860:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787827523.316904:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db840 x1874669228245760/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827523.316933:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827523.316936:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827523.316938:0:38:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827523.316941:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3668 00000100:00080000:2.0:1787827523.316943:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827523.316946:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787827523.316955:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827523.316963:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140da1c0 x1874669228245888/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827523.316971:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827523.316974:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827523.316975:0:38:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827523.316978:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3668 00000100:00080000:2.0:1787827523.316980:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827523.316983:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787827523.316986:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827523.316992:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228245760:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827523.316998:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140dbc00 x1874669228246016/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787827523.317066:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787827523.317112:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827523/real 1787827523] req@ffff9856140db840 x1874669228245760/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827548 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827523.317126:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db840 x1874669228245760/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827548 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827523.317136:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827523.317139:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827523.317143:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228245760:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787827523.317148:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228245888:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787827523.317166:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787827523.317182:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827523/real 1787827523] req@ffff9856140da1c0 x1874669228245888/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827548 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827523.317190:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140da1c0 x1874669228245888/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827548 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827523.317197:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827523.317199:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827523.317201:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228245888:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787827523.317228:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228246016:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787827523.317240:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787827523.317247:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827523/real 1787827523] req@ffff9856140dbc00 x1874669228246016/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827548 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827523.317254:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140dbc00 x1874669228246016/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827548 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827523.317260:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827523.317262:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827523.317264:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228246016:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787827523.380953:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827524.404795:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827525.428848:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827526.452863:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827527.476880:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827528.436779:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827528.436787:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827528.436791:0:38:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827528.436799:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3674 00000100:00080000:2.0:1787827528.436811:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827528.436819:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787827528.436857:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8f00 x1874669228246144/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827528.436880:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827528.436883:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827528.436884:0:38:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827528.436887:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3674 00000100:00080000:2.0:1787827528.436890:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827528.436893:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787827528.436902:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827528.436912:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140dad00 x1874669228246272/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827528.436917:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228246144:192.168.203.114@tcp:250:kworker.0 00000100:00080000:2.0:1787827528.436925:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827528.436928:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827528.436930:0:38:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827528.436934:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3674 00000100:00080000:2.0:1787827528.436937:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827528.436940:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787827528.436943:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827528.436952:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db0c0 x1874669228246400/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787827528.436996:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787827528.437025:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827528/real 1787827528] req@ffff9856140d8f00 x1874669228246144/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827553 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827528.437039:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8f00 x1874669228246144/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827553 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827528.437047:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827528.437050:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827528.437053:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228246144:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787827528.437073:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228246272:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787827528.437090:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787827528.437099:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827528/real 1787827528] req@ffff9856140dad00 x1874669228246272/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827553 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827528.437107:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140dad00 x1874669228246272/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827553 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827528.437112:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827528.437114:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827528.437117:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228246272:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787827528.437120:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228246400:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787827528.437131:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787827528.437137:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827528/real 1787827528] req@ffff9856140db0c0 x1874669228246400/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827553 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827528.437144:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db0c0 x1874669228246400/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827553 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827528.437149:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827528.437151:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827528.437153:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228246400:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787827528.500832:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827529.524965:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827530.548855:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827531.572812:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827532.596847:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827533.556796:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827533.556806:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827533.556810:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827533.556820:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3679 00000100:00080000:2.0:1787827533.556831:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827533.556839:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787827533.556875:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140da940 x1874669228246528/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827533.556903:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827533.556907:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827533.556909:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827533.556914:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3679 00000100:00080000:2.0:1787827533.556917:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827533.556922:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787827533.556931:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827533.556944:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228246528:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827533.556946:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8b40 x1874669228246656/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827533.556962:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827533.556966:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827533.556967:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827533.556972:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3679 00000100:00080000:2.0:1787827533.556976:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827533.556980:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787827533.556985:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827533.556998:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d9680 x1874669228246784/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787827533.557004:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787827533.557038:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827533/real 1787827533] req@ffff9856140da940 x1874669228246528/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827558 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827533.557052:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140da940 x1874669228246528/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827558 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827533.557060:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827533.557064:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827533.557069:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228246528:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787827533.557098:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228246656:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787827533.557127:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787827533.557155:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827533/real 1787827533] req@ffff9856140d8b40 x1874669228246656/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827558 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827533.557167:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8b40 x1874669228246656/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827558 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827533.557175:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827533.557178:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827533.557181:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228246656:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787827533.557187:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228246784:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787827533.557210:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787827533.557220:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827533/real 1787827533] req@ffff9856140d9680 x1874669228246784/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827558 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827533.557234:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d9680 x1874669228246784/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827558 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827533.557241:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827533.557244:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827533.557247:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228246784:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787827533.620846:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827534.644816:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827535.668851:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827536.692932:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827537.716958:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827538.676898:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827538.676910:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827538.676914:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827538.676923:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3684 00000100:00080000:3.0:1787827538.676933:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827538.676957:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787827538.676997:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82940 x1874669228246912/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827538.677032:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827538.677036:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827538.677039:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827538.677044:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3684 00000100:00080000:3.0:1787827538.677049:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827538.677054:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827538.677074:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827538.677092:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82d00 x1874669228247040/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827538.677116:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228246912:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787827538.677128:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827538.677133:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827538.677136:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827538.677143:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3684 00000100:00080000:3.0:1787827538.677148:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827538.677155:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827538.677161:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827538.677174:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81e00 x1874669228247168/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787827538.677223:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827538.677317:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827538/real 1787827538] req@ffff985610f82940 x1874669228246912/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827563 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827538.677338:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82940 x1874669228246912/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827563 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827538.677350:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827538.677354:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827538.677358:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228246912:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827538.677387:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228247040:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827538.677419:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827538.677430:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827538/real 1787827538] req@ffff985610f82d00 x1874669228247040/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827563 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827538.677440:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82d00 x1874669228247040/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827563 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827538.677446:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827538.677448:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827538.677451:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228247040:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827538.677456:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228247168:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827538.677469:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827538.677476:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827538/real 1787827538] req@ffff985610f81e00 x1874669228247168/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827563 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827538.677485:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81e00 x1874669228247168/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827563 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827538.677491:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827538.677494:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827538.677496:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228247168:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787827538.741021:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827539.764787:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827540.788834:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827541.812789:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827542.836873:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827543.796760:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827543.796768:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827543.796773:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827543.796781:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3689 00000100:00080000:2.0:1787827543.796790:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827543.796795:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787827543.796830:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db480 x1874669228247296/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827543.796854:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827543.796857:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827543.796858:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827543.796861:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3689 00000100:00080000:2.0:1787827543.796863:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827543.796866:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787827543.796874:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827543.796883:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d83c0 x1874669228247424/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827543.796891:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827543.796894:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827543.796896:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1787827543.796899:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228247296:192.168.203.114@tcp:250:kworker.0 00000100:00080000:2.0:1787827543.796901:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3689 00000100:00080000:2.0:1787827543.796907:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827543.796910:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787827543.796913:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827543.796923:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d9a40 x1874669228247552/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827543.796988:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228247424:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787827543.797047:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228247552:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787827543.860809:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827544.884805:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827545.908885:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:3.0:1787827546.868974:0:2372:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.114:988 00000400:00000100:3.0:1787827546.869038:0:2372:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.114@tcp at host 192.168.203.114 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:3.0:1787827546.869072:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:3.0:1787827546.869121:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:3.0:1787827546.869133:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000100:00100000:0.0:1787827546.869175:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827543/real 1787827546] req@ffff9856140d83c0 x1874669228247424/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827568 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827546.869228:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d83c0 x1874669228247424/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827568 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827546.869244:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827546.869250:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827546.869256:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228247424:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787827546.869267:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827543/real 1787827546] req@ffff9856140d9a40 x1874669228247552/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827568 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827546.869285:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d9a40 x1874669228247552/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827568 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827546.869296:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827546.869301:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827546.869306:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228247552:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787827546.869316:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827543/real 1787827546] req@ffff9856140db480 x1874669228247296/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827568 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827546.869332:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db480 x1874669228247296/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827568 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827546.869341:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827546.869345:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827546.869350:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228247296:192.168.203.114@tcp:250:kworker.0 00010000:00010000:0.0:1787827546.932814:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827547.956897:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827548.916820:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827548.916830:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827548.916834:0:38:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827548.916843:0:38:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3694 00000100:00080000:0.0:1787827548.916853:0:38:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827548.916859:0:38:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787827548.916893:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4b40 x1874669228247680/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827548.916928:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827548.916932:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827548.916934:0:38:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827548.916937:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3694 00000100:00080000:0.0:1787827548.916940:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827548.916944:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787827548.916953:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827548.916966:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4780 x1874669228247808/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827548.916980:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228247680:192.168.203.114@tcp:250:kworker.0 00000100:00080000:0.0:1787827548.916987:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827548.917009:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827548.917010:0:38:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827548.917014:0:38:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3694 00000100:00080000:0.0:1787827548.917017:0:38:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827548.917020:0:38:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787827548.917023:0:38:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827548.917033:0:38:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6940 x1874669228247936/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787827548.917086:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827548.917122:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827548/real 1787827548] req@ffff985607df4b40 x1874669228247680/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827573 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827548.917143:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4b40 x1874669228247680/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827573 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827548.917155:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827548.917159:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827548.917164:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228247680:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827548.917213:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228247808:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827548.917241:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827548.917265:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827548/real 1787827548] req@ffff985607df4780 x1874669228247808/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827573 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827548.917279:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4780 x1874669228247808/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827573 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827548.917288:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827548.917291:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827548.917295:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228247808:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827548.917302:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228247936:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827548.917328:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827548.917342:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827548/real 1787827548] req@ffff985607df6940 x1874669228247936/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827573 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827548.917356:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6940 x1874669228247936/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827573 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827548.917372:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827548.917376:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827548.917379:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228247936:192.168.203.114@tcp:38:kworker.0 00010000:00010000:3.0:1787827548.980893:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827550.004814:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827551.028863:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827552.052830:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827553.076830:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827554.036795:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827554.036803:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827554.036807:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827554.036814:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3699 00000100:00080000:0.0:1787827554.036824:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827554.036829:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787827554.036861:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df52c0 x1874669228248064/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827554.036884:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827554.036887:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827554.036888:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827554.036890:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3699 00000100:00080000:0.0:1787827554.036893:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827554.036895:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787827554.036902:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827554.036911:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7840 x1874669228248192/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827554.036919:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827554.036921:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827554.036923:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827554.036929:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3699 00000100:00080000:0.0:1787827554.036931:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00100000:2.0:1787827554.036931:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228248064:192.168.203.114@tcp:250:kworker.0 00000100:00080000:0.0:1787827554.036938:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787827554.036943:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827554.036951:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7480 x1874669228248320/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787827554.036990:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827554.037027:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827554/real 1787827554] req@ffff985607df52c0 x1874669228248064/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827579 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827554.037041:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df52c0 x1874669228248064/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827579 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827554.037050:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827554.037053:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827554.037056:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228248064:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827554.037076:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228248192:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827554.037092:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827554.037100:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827554/real 1787827554] req@ffff985607df7840 x1874669228248192/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827579 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827554.037108:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7840 x1874669228248192/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827579 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827554.037114:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827554.037116:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827554.037118:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228248192:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827554.037123:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228248320:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827554.037133:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827554.037156:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827554/real 1787827554] req@ffff985607df7480 x1874669228248320/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827579 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827554.037165:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7480 x1874669228248320/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827579 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827554.037170:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827554.037173:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827554.037175:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228248320:192.168.203.114@tcp:38:kworker.0 00010000:00010000:3.0:1787827554.100849:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827555.124881:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827556.148949:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827557.172817:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827558.196879:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827559.156875:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827559.156883:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827559.156887:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827559.156896:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3704 00000100:00080000:3.0:1787827559.156905:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827559.156911:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787827559.156958:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81680 x1874669228248448/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827559.156981:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827559.156983:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827559.156985:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827559.156988:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3704 00000100:00080000:3.0:1787827559.156991:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827559.156994:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827559.157003:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827559.157012:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228248448:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787827559.157015:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83840 x1874669228248576/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827559.157027:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827559.157029:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827559.157030:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827559.157033:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3704 00000100:00080000:3.0:1787827559.157036:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827559.157039:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827559.157043:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827559.157053:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f812c0 x1874669228248704/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787827559.157111:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827559.157159:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827559/real 1787827559] req@ffff985610f81680 x1874669228248448/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827584 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827559.157174:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81680 x1874669228248448/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827584 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827559.157182:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827559.157185:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827559.157188:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228248448:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827559.157207:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228248576:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827559.157224:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827559.157233:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827559/real 1787827559] req@ffff985610f83840 x1874669228248576/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827584 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827559.157241:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83840 x1874669228248576/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827584 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827559.157247:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827559.157249:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827559.157251:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228248576:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827559.157255:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228248704:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827559.157265:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827559.157271:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827559/real 1787827559] req@ffff985610f812c0 x1874669228248704/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827584 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827559.157278:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f812c0 x1874669228248704/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827584 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827559.157283:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827559.157285:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827559.157287:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228248704:192.168.203.114@tcp:38:kworker.0 00010000:00010000:3.0:1787827559.220836:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827560.244823:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827561.268843:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827562.292818:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827563.316853:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827564.276823:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827564.276833:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827564.276838:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827564.276858:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3709 00000100:00080000:3.0:1787827564.276869:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827564.276876:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787827564.276912:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80b40 x1874669228248832/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827564.276949:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827564.276953:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827564.276955:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827564.276959:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3709 00000100:00080000:3.0:1787827564.276962:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827564.276966:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827564.276977:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827564.277000:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228248832:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787827564.277002:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f803c0 x1874669228248960/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827564.277016:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827564.277021:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827564.277023:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827564.277027:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3709 00000100:00080000:3.0:1787827564.277031:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827564.277036:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827564.277041:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000800:00000100:2.0:1787827564.277059:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787827564.277061:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80000 x1874669228249088/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827564.277119:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827564/real 1787827564] req@ffff985610f80b40 x1874669228248832/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827589 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827564.277133:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80b40 x1874669228248832/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827589 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827564.277146:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827564.277151:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827564.277155:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228248832:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827564.277189:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228248960:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827564.277214:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827564.277227:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827564/real 1787827564] req@ffff985610f803c0 x1874669228248960/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827589 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827564.277240:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f803c0 x1874669228248960/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827589 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827564.277249:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827564.277253:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827564.277256:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228248960:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827564.277264:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228249088:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827564.277280:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827564.277291:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827564/real 1787827564] req@ffff985610f80000 x1874669228249088/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827589 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827564.277303:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80000 x1874669228249088/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827589 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827564.277312:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827564.277316:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827564.277320:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228249088:192.168.203.114@tcp:38:kworker.0 00010000:00010000:3.0:1787827564.340916:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827565.364805:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827566.388778:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:3.0:1787827566.474066:0:92859:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == replay-single test 44c: race in target handle connect ========================================================== 06:46:06 (1787827566) 00010000:00010000:0.0:1787827567.412782:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827568.436836:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827569.396854:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827569.396878:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827569.396882:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827569.396892:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3715 00000100:00080000:2.0:1787827569.396903:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827569.396910:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787827569.396947:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d9680 x1874669228249216/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827569.396978:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827569.396982:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827569.396984:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827569.396987:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3715 00000100:00080000:2.0:1787827569.396990:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827569.396994:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787827569.397004:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827569.397018:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8b40 x1874669228249344/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827569.397060:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228249216:192.168.203.114@tcp:250:kworker.0 00000100:00080000:2.0:1787827569.397060:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827569.397074:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827569.397092:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827569.397098:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3715 00000100:00080000:2.0:1787827569.397104:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827569.397109:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787827569.397125:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827569.397153:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140da940 x1874669228249472/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787827569.397168:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787827569.397209:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827569/real 1787827569] req@ffff9856140d9680 x1874669228249216/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827594 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827569.397227:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d9680 x1874669228249216/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827594 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827569.397247:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827569.397252:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827569.397257:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228249216:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787827569.397284:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228249344:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787827569.397308:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787827569.397320:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827569/real 1787827569] req@ffff9856140d8b40 x1874669228249344/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827594 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827569.397329:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8b40 x1874669228249344/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827594 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827569.397335:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827569.397338:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827569.397342:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228249344:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787827569.397350:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228249472:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787827569.397384:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787827569.397400:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827569/real 1787827569] req@ffff9856140da940 x1874669228249472/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827594 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827569.397412:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140da940 x1874669228249472/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827594 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827569.397418:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827569.397420:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787827569.397430:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228249472:192.168.203.114@tcp:38:kworker.0 00010000:00010000:3.0:1787827569.460859:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827570.484813:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827571.508979:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200008:1.0:1787827572.416277:0:92880:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:1.0:1787827572.416293:0:92880:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787827572.416321:0:92880:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827572.416346:0:92880:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 76 lru_type: 1 00010000:00010000:1.0:1787827572.416365:0:92880:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 76 lru_type: 1 00000080:00200000:1.0:1787827572.416375:0:92880:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827572.416384:0:92880:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787827572.416389:0:92880:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee56d 00000080:00010000:1.0:1787827572.416412:0:92880:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee56d from it ffffab3ac17d7cb8 00010000:00010000:1.0:1787827572.416421:0:92880:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 76 lru_type: 1 00010000:00010000:1.0:1787827572.416432:0:92880:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 77 lru_type: 1 00000080:00200008:1.0:1787827572.417099:0:92880:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), request_mask 0, flags 0, foreign 0 00000080:00200000:1.0:1787827572.417105:0:92880:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787827572.417116:0:92880:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827572.417125:0:92880:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 77 lru_type: 1 00010000:00010000:1.0:1787827572.417136:0:92880:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 77 lru_type: 1 00000080:00200000:1.0:1787827572.417143:0:92880:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827572.417148:0:92880:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787827572.417151:0:92880:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee56d 00000080:00010000:1.0:1787827572.417155:0:92880:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee56d from it ffffab3ac17d7cb8 00010000:00010000:1.0:1787827572.417159:0:92880:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 77 lru_type: 1 00010000:00010000:1.0:1787827572.417167:0:92880:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 78 lru_type: 1 00000080:00200000:1.0:1787827572.417269:0:92880:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8),name=/ 00000002:00010000:1.0:1787827572.417284:0:92880:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827572.417295:0:92880:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 78 lru_type: 1 00010000:00010000:1.0:1787827572.417325:0:92880:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 78 lru_type: 1 00000080:00200000:1.0:1787827572.417340:0:92880:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827572.417345:0:92880:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) 00000080:00010000:1.0:1787827572.417348:0:92880:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff98561158e3c8) for lock 0xb3f83dad1b7ee56d 00000080:00010000:1.0:1787827572.417353:0:92880:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xb3f83dad1b7ee56d from it ffffab3ac17d7bb0 00010000:00010000:1.0:1787827572.417357:0:92880:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 78 lru_type: 1 00010000:00010000:1.0:1787827572.417383:0:92880:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff985602bc9800 lock: ffff985610d86c00/0xb3f83dad1b7ee56d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x5aa6d504d91e4a41 expref: -99 pid: 90092 timeout: 0 lvb_type: 0 lru_score: 79 lru_type: 1 00000080:00200000:1.0:1787827572.417392:0:92880:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8), inode mode 41ed mask 44 00000080:00200008:1.0:1787827572.417400:0:92880:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff98561158e3c8), flags 100000 00000080:00200080:1.0:1787827572.417495:0:92880:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff98561158e3c8) cmd=c00866a4 arg=7ffd9aa64590 00800000:00000080:1.0:1787827572.417515:0:92880:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff985602bc9800: cmd=c00866a4 len=592 karg=ffff98560865f000 uarg=0000000000000000 00800000:00000004:1.0:1787827572.417523:0:92880:0:(obd_class.h:1091:obd_statfs()) lustre-MDT0000-mdc-ffff985602bc9800: age 3335, max_age 3722 00000100:00080000:1.0:1787827572.417580:0:92880:0:(client.c:1845:ptlrpc_send_new_req()) @@@ req waiting for recovery: (FULL != DISCONN) req@ffff9856085a1e00 x1874669228249600/t0(0) o41->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 224/368 e 0 to 0 dl 0 ref 2 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827572.417592:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827572.532867:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827573.428802:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827573.556975:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827574.452896:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787827574.516821:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827574.516829:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827574.516833:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827574.516841:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3720 00000100:00080000:0.0:1787827574.516851:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827574.516856:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787827574.516888:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5e00 x1874669228249728/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827574.516914:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827574.516917:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827574.516918:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827574.516921:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3720 00000100:00080000:0.0:1787827574.516924:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827574.516928:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787827574.516936:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827574.516946:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228249728:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787827574.516947:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df43c0 x1874669228249856/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827574.516961:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827574.516963:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827574.516965:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827574.516967:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3720 00000100:00080000:0.0:1787827574.516970:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827574.516972:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787827574.516975:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827574.516984:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5680 x1874669228249984/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787827574.517024:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827574.517055:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827574/real 1787827574] req@ffff985607df5e00 x1874669228249728/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827599 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827574.517069:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5e00 x1874669228249728/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827599 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827574.517076:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827574.517080:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827574.517083:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228249728:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827574.517100:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228249856:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827574.517116:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827574.517123:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827574/real 1787827574] req@ffff985607df43c0 x1874669228249856/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827599 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827574.517131:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df43c0 x1874669228249856/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827599 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827574.517137:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827574.517139:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827574.517141:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228249856:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827574.517145:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228249984:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827574.517155:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827574.517161:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827574/real 1787827574] req@ffff985607df5680 x1874669228249984/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827599 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827574.517168:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5680 x1874669228249984/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827599 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827574.517173:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827574.517175:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827574.517176:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228249984:192.168.203.114@tcp:38:kworker.0 00010000:00010000:3.0:1787827574.580831:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827575.476823:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827575.604843:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827576.500874:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787827576.629015:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827577.524832:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827577.652840:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827578.548862:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827578.676892:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827579.572827:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787827579.636840:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827579.636853:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827579.636859:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827579.636872:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3725 00000100:00080000:0.0:1787827579.636886:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827579.636896:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787827579.636941:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6580 x1874669228250112/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827579.637037:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827579.637045:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827579.637059:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827579.637068:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3725 00000100:00080000:0.0:1787827579.637077:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827579.637087:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787827579.637101:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827579.637111:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228250112:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787827579.637131:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5a40 x1874669228250240/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827579.637171:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827579.637188:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827579.637191:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827579.637198:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3725 00000100:00080000:0.0:1787827579.637205:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000800:00000100:1.0:1787827579.637212:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00080000:0.0:1787827579.637215:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787827579.637241:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827579.637274:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df70c0 x1874669228250368/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827579.637285:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827579/real 1787827579] req@ffff985607df6580 x1874669228250112/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827604 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827579.637310:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6580 x1874669228250112/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827604 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827579.637322:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827579.637327:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827579.637331:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228250112:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787827579.637354:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228250240:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787827579.637376:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827579.637388:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827579/real 1787827579] req@ffff985607df5a40 x1874669228250240/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827604 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827579.637399:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5a40 x1874669228250240/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827604 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827579.637407:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827579.637410:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827579.637413:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228250240:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787827579.637418:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228250368:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787827579.637433:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827579.637441:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827579/real 1787827579] req@ffff985607df70c0 x1874669228250368/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827604 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827579.637476:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df70c0 x1874669228250368/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827604 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827579.637483:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827579.637486:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827579.637488:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228250368:192.168.203.114@tcp:38:kworker.0 00010000:00010000:3.0:1787827579.700842:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827580.596869:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827580.724869:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827581.620806:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787827581.748869:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827582.644776:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827582.772798:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827583.668775:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827583.796905:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827584.692810:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787827584.756810:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827584.756821:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827584.756825:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827584.756837:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3730 00000100:00080000:0.0:1787827584.756849:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827584.756858:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787827584.756917:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5680 x1874669228250496/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827584.756984:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827584.756987:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827584.756989:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827584.756992:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3730 00000100:00080000:0.0:1787827584.756996:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827584.757000:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787827584.757010:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827584.757031:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228250496:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787827584.757037:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df43c0 x1874669228250624/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827584.757048:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827584.757051:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827584.757054:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827584.757063:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3730 00000100:00080000:0.0:1787827584.757068:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827584.757076:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787827584.757100:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827584.757118:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5e00 x1874669228250752/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827584.757139:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827584.757205:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827584/real 1787827584] req@ffff985607df5680 x1874669228250496/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827609 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827584.757238:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5680 x1874669228250496/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827609 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827584.757248:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827584.757251:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827584.757254:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228250496:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787827584.757277:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228250624:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787827584.757309:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827584.757319:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827584/real 1787827584] req@ffff985607df43c0 x1874669228250624/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827609 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827584.757327:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df43c0 x1874669228250624/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827609 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827584.757334:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827584.757337:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827584.757339:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228250624:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787827584.757342:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228250752:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787827584.757353:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827584.757359:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827584/real 1787827584] req@ffff985607df5e00 x1874669228250752/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827609 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827584.757366:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5e00 x1874669228250752/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827609 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827584.757371:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827584.757380:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827584.757382:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228250752:192.168.203.114@tcp:38:kworker.0 00010000:00010000:3.0:1787827584.820855:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827585.716847:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827585.844899:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827586.740822:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787827586.868844:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827587.764844:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827587.892874:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827588.788790:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827588.916851:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827589.812758:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787827589.876775:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827589.876786:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827589.876790:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827589.876801:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3735 00000100:00080000:0.0:1787827589.876812:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827589.876820:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787827589.876857:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4f00 x1874669228250880/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827589.876882:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827589.876885:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827589.876897:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827589.876900:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3735 00000100:00080000:0.0:1787827589.876903:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827589.876906:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787827589.876914:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827589.876925:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6d00 x1874669228251008/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827589.876934:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228250880:192.168.203.114@tcp:250:kworker.0 00000100:00080000:0.0:1787827589.876935:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827589.876942:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827589.876945:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827589.876951:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3735 00000100:00080000:0.0:1787827589.876953:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827589.876955:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787827589.876958:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827589.876968:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df61c0 x1874669228251136/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827589.877001:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827589.877040:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827589/real 1787827589] req@ffff985607df4f00 x1874669228250880/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827614 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827589.877064:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4f00 x1874669228250880/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827614 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827589.877075:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827589.877078:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827589.877081:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228250880:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787827589.877101:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228251008:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787827589.877119:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827589.877128:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827589/real 1787827589] req@ffff985607df6d00 x1874669228251008/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827614 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827589.877136:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6d00 x1874669228251008/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827614 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827589.877144:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827589.877148:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827589.877151:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228251008:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787827589.877157:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228251136:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787827589.877178:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827589.877190:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827589/real 1787827589] req@ffff985607df61c0 x1874669228251136/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827614 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827589.877210:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df61c0 x1874669228251136/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827614 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827589.877219:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827589.877222:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827589.877226:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228251136:192.168.203.114@tcp:38:kworker.0 00010000:00010000:3.0:1787827589.940825:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827590.836815:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827590.964820:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827591.860773:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787827591.988869:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827592.884784:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827593.012882:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827593.908809:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827594.036929:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827594.932771:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787827594.996821:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827594.996834:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827594.996838:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827594.996864:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3740 00000100:00080000:2.0:1787827594.996878:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827594.996886:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787827594.996926:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db0c0 x1874669228251264/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827594.996952:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827594.996956:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827594.996958:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827594.996961:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3740 00000100:00080000:2.0:1787827594.996964:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827594.996967:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787827594.996975:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827594.996987:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140dad00 x1874669228251392/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827594.996996:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827594.997000:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827594.997001:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827594.997005:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3740 00000100:00080000:2.0:1787827594.997007:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827594.997010:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787827594.997014:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827594.997021:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228251264:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827594.997026:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8f00 x1874669228251520/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827594.997108:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827594.997153:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827594/real 1787827594] req@ffff9856140db0c0 x1874669228251264/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827619 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827594.997177:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db0c0 x1874669228251264/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827619 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827594.997192:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827594.997197:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827594.997203:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228251264:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787827594.997214:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228251392:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787827594.997248:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827594.997282:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827594/real 1787827594] req@ffff9856140dad00 x1874669228251392/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827619 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827594.997300:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140dad00 x1874669228251392/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827619 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827594.997312:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827594.997316:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827594.997320:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228251392:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787827594.997352:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228251520:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787827594.997384:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827594.997400:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827594/real 1787827594] req@ffff9856140d8f00 x1874669228251520/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827619 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827594.997417:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8f00 x1874669228251520/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827619 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827594.997429:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827594.997433:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827594.997438:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228251520:192.168.203.114@tcp:38:kworker.0 00010000:00010000:3.0:1787827595.060983:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827595.956816:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827596.084824:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827596.980811:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787827597.108992:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827598.004886:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827598.132905:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827599.028771:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827599.156949:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827600.052801:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787827600.116830:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827600.116841:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827600.116845:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827600.116873:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3745 00000100:00080000:1.0:1787827600.116886:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827600.116897:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787827600.116965:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2580 x1874669228251648/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827600.117022:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827600.117026:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827600.117028:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827600.117032:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3745 00000100:00080000:1.0:1787827600.117036:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827600.117040:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787827600.117051:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827600.117064:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a12c0 x1874669228251776/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827600.117070:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228251648:192.168.203.114@tcp:250:kworker.0 00000100:00080000:1.0:1787827600.117086:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827600.117090:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827600.117092:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827600.117097:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3745 00000100:00080000:1.0:1787827600.117101:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827600.117104:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787827600.117108:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827600.117120:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a1a40 x1874669228251904/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787827600.117148:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827600.117185:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827600/real 1787827600] req@ffff9856085a2580 x1874669228251648/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827625 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827600.117203:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2580 x1874669228251648/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827625 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827600.117234:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827600.117238:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827600.117242:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228251648:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827600.117280:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228251776:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827600.117297:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827600.117306:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827600/real 1787827600] req@ffff9856085a12c0 x1874669228251776/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827625 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827600.117314:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a12c0 x1874669228251776/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827625 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827600.117320:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827600.117322:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827600.117324:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228251776:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827600.117328:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228251904:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827600.117344:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827600.117350:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827600/real 1787827600] req@ffff9856085a1a40 x1874669228251904/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827625 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827600.117358:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a1a40 x1874669228251904/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827625 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827600.117363:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827600.117365:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827600.117367:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228251904:192.168.203.114@tcp:38:kworker.0 00010000:00010000:3.0:1787827600.180875:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827601.076801:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827601.204936:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827602.100792:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787827602.228992:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827603.124820:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827603.252883:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827604.148779:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827604.276825:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827605.172756:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787827605.236850:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827605.236865:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827605.236871:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827605.236884:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3750 00000100:00080000:3.0:1787827605.236898:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827605.236908:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787827605.236959:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80780 x1874669228252032/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827605.237002:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827605.237007:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827605.237009:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827605.237013:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3750 00000100:00080000:3.0:1787827605.237017:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827605.237022:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827605.237034:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827605.237049:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80000 x1874669228252160/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827605.237060:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827605.237064:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827605.237066:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827605.237071:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3750 00000100:00080000:3.0:1787827605.237075:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00100000:2.0:1787827605.237088:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228252032:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787827605.237092:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827605.237103:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827605.237124:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f803c0 x1874669228252288/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787827605.237160:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827605.237193:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827605/real 1787827605] req@ffff985610f80780 x1874669228252032/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827630 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827605.237216:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80780 x1874669228252032/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827630 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827605.237225:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827605.237229:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827605.237232:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228252032:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827605.237238:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228252160:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827605.237254:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827605.237262:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827605/real 1787827605] req@ffff985610f80000 x1874669228252160/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827630 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827605.237270:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80000 x1874669228252160/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827630 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827605.237276:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827605.237278:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827605.237280:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228252160:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827605.237302:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228252288:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827605.237321:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827605.237333:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827605/real 1787827605] req@ffff985610f803c0 x1874669228252288/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827630 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827605.237346:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f803c0 x1874669228252288/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827630 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827605.237364:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827605.237367:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827605.237369:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228252288:192.168.203.114@tcp:38:kworker.0 00010000:00010000:3.0:1787827605.300836:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827606.196838:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827606.324832:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827607.220920:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787827607.348839:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827608.244833:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827608.372931:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827609.268885:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827609.396954:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827610.292806:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787827610.356845:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827610.356858:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827610.356863:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827610.356877:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3755 00000100:00080000:3.0:1787827610.356892:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827610.356902:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787827610.356952:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80b40 x1874669228252416/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827610.356992:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827610.356996:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827610.357008:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827610.357012:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3755 00000100:00080000:3.0:1787827610.357016:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827610.357019:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827610.357031:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827610.357044:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f830c0 x1874669228252544/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827610.357054:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827610.357058:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827610.357059:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827610.357063:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3755 00000100:00080000:3.0:1787827610.357067:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827610.357071:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827610.357076:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827610.357087:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81a40 x1874669228252672/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827610.357118:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228252416:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827610.357253:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228252544:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827610.357281:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228252672:192.168.203.114@tcp:38:kworker.0 00010000:00010000:3.0:1787827610.421064:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827611.316861:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827611.444857:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827612.340835:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787827612.468940:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827613.364773:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000400:00000100:1.0:1787827613.429176:0:2373:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.114:988 00000400:00000100:1.0:1787827613.429232:0:2373:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.114@tcp at host 192.168.203.114 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1787827613.429254:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:1.0:1787827613.429294:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:1.0:1787827613.429310:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000100:00100000:2.0:1787827613.429323:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827610/real 1787827613] req@ffff985610f80b40 x1874669228252416/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827635 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827613.429359:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80b40 x1874669228252416/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827635 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827613.429375:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827613.429381:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827613.429387:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228252416:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827613.429399:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827610/real 1787827613] req@ffff985610f830c0 x1874669228252544/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827635 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827613.429423:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f830c0 x1874669228252544/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827635 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827613.429432:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827613.429436:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827613.429440:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228252544:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827613.429447:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827610/real 1787827613] req@ffff985610f81a40 x1874669228252672/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827635 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827613.429459:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81a40 x1874669228252672/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827635 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827613.429468:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827613.429472:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827613.429475:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228252672:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787827613.492873:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827614.388824:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827614.516912:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827615.412830:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787827615.476910:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827615.476920:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827615.476926:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827615.476937:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3761 00000100:00080000:3.0:1787827615.476948:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827615.476957:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787827615.476994:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f812c0 x1874669228252800/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827615.477037:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827615.477041:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827615.477044:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827615.477048:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3761 00000100:00080000:3.0:1787827615.477053:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827615.477067:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827615.477079:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827615.477093:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228252800:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787827615.477097:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83840 x1874669228252928/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827615.477135:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827615.477139:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827615.477142:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827615.477147:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3761 00000100:00080000:3.0:1787827615.477152:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827615.477158:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827615.477164:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000800:00000100:2.0:1787827615.477182:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787827615.477184:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81680 x1874669228253056/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827615.477241:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827615/real 1787827615] req@ffff985610f812c0 x1874669228252800/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827640 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827615.477265:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f812c0 x1874669228252800/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827640 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827615.477278:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827615.477284:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827615.477301:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228252800:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827615.477331:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228252928:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827615.477364:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827615.477381:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827615/real 1787827615] req@ffff985610f83840 x1874669228252928/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827640 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827615.477399:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83840 x1874669228252928/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827640 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827615.477410:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827615.477414:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827615.477419:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228252928:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827615.477427:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228253056:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827615.477457:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827615.477474:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827615/real 1787827615] req@ffff985610f81680 x1874669228253056/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827640 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827615.477492:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81680 x1874669228253056/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827640 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827615.477513:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827615.477518:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827615.477522:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228253056:192.168.203.114@tcp:38:kworker.0 00010000:00010000:3.0:1787827615.540917:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827616.436833:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827616.564860:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827617.460797:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787827617.588905:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827618.484806:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827618.612808:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827619.508789:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827619.636877:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827620.532756:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787827620.596792:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827620.596802:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827620.596807:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827620.596817:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3766 00000100:00080000:3.0:1787827620.596826:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827620.596834:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787827620.596869:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80f00 x1874669228253184/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827620.596899:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827620.596903:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827620.596905:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827620.596909:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3766 00000100:00080000:3.0:1787827620.596913:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827620.596917:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827620.596927:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827620.596940:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81e00 x1874669228253312/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827620.596946:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228253184:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787827620.596959:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827620.596963:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827620.596966:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827620.596971:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3766 00000100:00080000:3.0:1787827620.596975:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827620.596980:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827620.596985:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827620.596997:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82d00 x1874669228253440/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787827620.597005:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827620.597037:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827620/real 1787827620] req@ffff985610f80f00 x1874669228253184/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827645 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827620.597051:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80f00 x1874669228253184/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827645 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827620.597059:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827620.597062:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827620.597065:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228253184:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827620.597084:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228253312:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827620.597100:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827620.597109:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827620/real 1787827620] req@ffff985610f81e00 x1874669228253312/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827645 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827620.597118:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81e00 x1874669228253312/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827645 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827620.597125:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827620.597128:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827620.597131:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228253312:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827620.597134:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228253440:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827620.597147:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827620.597153:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827620/real 1787827620] req@ffff985610f82d00 x1874669228253440/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827645 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827620.597161:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82d00 x1874669228253440/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827645 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827620.597166:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827620.597168:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827620.597171:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228253440:192.168.203.114@tcp:38:kworker.0 00010000:00010000:3.0:1787827620.660843:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827621.556775:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827621.684777:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827622.580776:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787827622.708882:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827623.604785:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827623.732848:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827624.628815:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827624.756869:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827625.652778:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787827625.716814:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827625.716824:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827625.716830:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827625.716840:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3771 00000100:00080000:3.0:1787827625.716851:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827625.716858:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787827625.716897:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82940 x1874669228253568/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827625.716943:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827625.716947:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827625.716949:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827625.716953:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3771 00000100:00080000:3.0:1787827625.716956:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827625.716960:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827625.716970:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827625.716983:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82580 x1874669228253696/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827625.716994:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827625.716997:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827625.717000:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827625.717003:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3771 00000100:00080000:3.0:1787827625.717007:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00100000:2.0:1787827625.717013:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228253568:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787827625.717014:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827625.717027:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827625.717041:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83480 x1874669228253824/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787827625.717100:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827625.717137:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827625/real 1787827625] req@ffff985610f82940 x1874669228253568/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827650 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827625.717154:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82940 x1874669228253568/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827650 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827625.717176:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827625.717183:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827625.717189:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228253568:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827625.717200:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228253696:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827625.717233:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827625.717251:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827625/real 1787827625] req@ffff985610f82580 x1874669228253696/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827650 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827625.717266:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82580 x1874669228253696/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827650 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827625.717280:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827625.717285:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827625.717290:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228253696:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827625.717321:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228253824:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827625.717352:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827625.717366:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827625/real 1787827625] req@ffff985610f83480 x1874669228253824/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827650 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827625.717378:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83480 x1874669228253824/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827650 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827625.717386:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827625.717389:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827625.717392:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228253824:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787827625.780844:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827626.676839:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827626.804873:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827627.700815:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787827627.828862:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827628.724833:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827628.852825:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827629.748756:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827629.876969:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827630.772861:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787827630.836797:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827630.836811:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827630.836817:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827630.836832:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3776 00000100:00080000:3.0:1787827630.836852:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827630.836865:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787827630.836921:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f821c0 x1874669228253952/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827630.836963:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827630.836969:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827630.836971:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827630.836976:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3776 00000100:00080000:3.0:1787827630.836982:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827630.836988:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827630.837002:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827630.837021:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83480 x1874669228254080/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827630.837040:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827630.837051:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:2.0:1787827630.837055:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f821c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228253952:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787827630.837055:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827630.837077:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3776 00000100:00080000:3.0:1787827630.837082:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827630.837087:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827630.837093:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827630.837109:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82580 x1874669228254208/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787827630.837187:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827630.837244:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827630/real 1787827630] req@ffff985610f821c0 x1874669228253952/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827655 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827630.837266:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f821c0 x1874669228253952/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827655 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827630.837281:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827630.837287:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827630.837291:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f821c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228253952:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827630.837326:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228254080:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827630.837351:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827630.837364:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827630/real 1787827630] req@ffff985610f83480 x1874669228254080/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827655 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827630.837376:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83480 x1874669228254080/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827655 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827630.837385:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827630.837389:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827630.837392:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228254080:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827630.837398:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228254208:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827630.837416:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827630.837427:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827630/real 1787827630] req@ffff985610f82580 x1874669228254208/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827655 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827630.837438:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82580 x1874669228254208/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827655 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827630.837446:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827630.837449:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827630.837452:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228254208:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787827630.900962:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827631.796854:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827631.924881:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827632.820804:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787827632.948969:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827633.844861:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827633.972857:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827634.868834:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827634.996929:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827635.892817:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787827635.956816:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827635.956824:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827635.956827:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827635.956836:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3781 00000100:00080000:3.0:1787827635.956845:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827635.956851:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787827635.956884:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82940 x1874669228254336/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827635.956907:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827635.956910:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827635.956911:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827635.956914:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3781 00000100:00080000:3.0:1787827635.956916:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827635.956918:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827635.956926:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827635.956934:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82d00 x1874669228254464/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827635.956941:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827635.956943:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827635.956945:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827635.956948:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3781 00000100:00080000:3.0:1787827635.956953:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827635.956955:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827635.956958:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827635.956965:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81e00 x1874669228254592/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827635.956998:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228254336:192.168.203.114@tcp:250:kworker.0 00000800:00000100:2.0:1787827635.957079:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827635.957120:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827635/real 1787827635] req@ffff985610f82940 x1874669228254336/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827660 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827635.957136:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82940 x1874669228254336/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827660 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827635.957146:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827635.957150:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827635.957153:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228254336:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827635.957160:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228254464:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827635.957177:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827635.957186:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827635/real 1787827635] req@ffff985610f82d00 x1874669228254464/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827660 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827635.957195:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82d00 x1874669228254464/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827660 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827635.957202:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827635.957204:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827635.957207:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228254464:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827635.957211:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228254592:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827635.957223:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827635.957229:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827635/real 1787827635] req@ffff985610f81e00 x1874669228254592/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827660 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827635.957237:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81e00 x1874669228254592/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827660 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827635.957243:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827635.957245:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827635.957247:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228254592:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787827636.020866:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827636.916833:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827637.044830:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827637.940810:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787827638.068924:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827638.964823:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827639.092823:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827639.988800:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827640.116873:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827641.012888:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787827641.076835:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827641.076845:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827641.076850:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827641.076860:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3786 00000100:00080000:3.0:1787827641.076870:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827641.076877:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787827641.076916:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80f00 x1874669228254720/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827641.076945:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827641.076949:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827641.076951:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827641.076954:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3786 00000100:00080000:3.0:1787827641.076958:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827641.076961:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827641.076978:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827641.076992:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81680 x1874669228254848/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827641.077003:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1787827641.077007:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228254720:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787827641.077008:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827641.077013:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827641.077024:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3786 00000100:00080000:3.0:1787827641.077028:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827641.077031:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827641.077037:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827641.077066:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83840 x1874669228254976/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787827641.077106:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827641.077159:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827641/real 1787827641] req@ffff985610f80f00 x1874669228254720/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827666 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827641.077177:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80f00 x1874669228254720/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827666 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827641.077188:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827641.077191:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827641.077196:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228254720:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827641.077221:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228254848:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827641.077241:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827641.077252:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827641/real 1787827641] req@ffff985610f81680 x1874669228254848/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827666 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827641.077263:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81680 x1874669228254848/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827666 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827641.077271:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827641.077274:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827641.077277:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228254848:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827641.077281:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228254976:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827641.077294:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827641.077302:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827641/real 1787827641] req@ffff985610f83840 x1874669228254976/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827666 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827641.077312:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83840 x1874669228254976/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827666 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827641.077319:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827641.077321:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827641.077324:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228254976:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787827641.140905:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827642.036861:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827642.164824:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827643.060765:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827643.188836:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827644.084770:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827644.212866:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827645.108936:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827645.236799:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827646.132796:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787827646.196762:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827646.196769:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827646.196773:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827646.196781:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3791 00000100:00080000:3.0:1787827646.196791:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827646.196797:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787827646.196830:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f812c0 x1874669228255104/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827646.196852:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827646.196854:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827646.196856:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827646.196858:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3791 00000100:00080000:3.0:1787827646.196861:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827646.196863:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827646.196871:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827646.196885:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81a40 x1874669228255232/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827646.196892:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228255104:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787827646.196903:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827646.196907:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827646.196910:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827646.196915:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3791 00000100:00080000:3.0:1787827646.196920:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827646.196924:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827646.196930:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827646.196943:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f830c0 x1874669228255360/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787827646.196952:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827646.196983:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827646/real 1787827646] req@ffff985610f812c0 x1874669228255104/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827671 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827646.196997:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f812c0 x1874669228255104/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827671 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827646.197004:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827646.197007:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827646.197010:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228255104:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827646.197028:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228255232:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827646.197042:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827646.197049:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827646/real 1787827646] req@ffff985610f81a40 x1874669228255232/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827671 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827646.197057:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81a40 x1874669228255232/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827671 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827646.197063:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827646.197065:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827646.197067:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228255232:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827646.197080:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228255360:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827646.197090:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827646.197096:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827646/real 1787827646] req@ffff985610f830c0 x1874669228255360/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827671 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827646.197107:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f830c0 x1874669228255360/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827671 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827646.197112:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827646.197114:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827646.197116:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228255360:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787827646.260809:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827647.156861:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827647.284875:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827648.180875:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827648.308797:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827649.204801:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827649.332820:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827650.228832:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827650.356848:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827651.252761:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787827651.316792:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827651.316800:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827651.316804:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827651.316811:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3796 00000100:00080000:3.0:1787827651.316820:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827651.316825:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787827651.316857:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80b40 x1874669228255488/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827651.316888:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827651.316890:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827651.316892:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827651.316894:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3796 00000100:00080000:3.0:1787827651.316897:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827651.316899:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827651.316912:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827651.316922:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f803c0 x1874669228255616/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827651.316930:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827651.316933:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827651.316935:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827651.316938:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3796 00000100:00080000:3.0:1787827651.316940:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827651.316942:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827651.316945:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827651.316953:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80000 x1874669228255744/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827651.316971:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228255488:192.168.203.114@tcp:250:kworker.0 00000800:00000100:2.0:1787827651.317050:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827651.317091:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827651/real 1787827651] req@ffff985610f80b40 x1874669228255488/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827676 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827651.317107:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80b40 x1874669228255488/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827676 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827651.317116:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827651.317120:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827651.317131:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228255488:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827651.317138:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228255616:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827651.317156:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827651.317168:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827651/real 1787827651] req@ffff985610f803c0 x1874669228255616/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827676 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827651.317177:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f803c0 x1874669228255616/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827676 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827651.317188:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827651.317191:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827651.317193:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228255616:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827651.317218:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228255744:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827651.317231:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827651.317237:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827651/real 1787827651] req@ffff985610f80000 x1874669228255744/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827676 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827651.317245:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80000 x1874669228255744/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827676 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827651.317250:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827651.317252:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827651.317254:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228255744:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787827651.380827:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827652.276770:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827652.404819:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827653.300872:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827653.428895:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827654.324834:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827654.452822:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827655.348774:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827655.476850:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827656.372811:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787827656.436808:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827656.436816:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827656.436820:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827656.436836:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3802 00000100:00080000:3.0:1787827656.436845:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827656.436851:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787827656.436883:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80780 x1874669228255872/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827656.436905:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827656.436908:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827656.436909:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827656.436912:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3802 00000100:00080000:3.0:1787827656.436914:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827656.436916:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827656.436924:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827656.436933:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83c00 x1874669228256000/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827656.436940:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827656.436943:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827656.436944:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827656.436947:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3802 00000100:00080000:3.0:1787827656.436949:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827656.436954:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787827656.436956:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228255872:192.168.203.114@tcp:250:kworker.0 00000008:00100000:3.0:1787827656.436957:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827656.436969:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80000 x1874669228256128/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787827656.437016:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827656.437047:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827656/real 1787827656] req@ffff985610f80780 x1874669228255872/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827681 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827656.437061:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80780 x1874669228255872/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827681 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827656.437074:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827656.437077:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827656.437081:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228255872:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827656.437086:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228256000:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827656.437104:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827656.437113:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827656/real 1787827656] req@ffff985610f83c00 x1874669228256000/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827681 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827656.437121:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83c00 x1874669228256000/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827681 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827656.437129:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827656.437131:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827656.437135:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228256000:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827656.437155:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228256128:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827656.437172:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827656.437181:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827656/real 1787827656] req@ffff985610f80000 x1874669228256128/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827681 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827656.437189:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80000 x1874669228256128/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827681 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827656.437195:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827656.437198:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827656.437200:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228256128:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787827656.500829:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827657.396778:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827657.524852:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827658.420800:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827658.548816:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827659.444763:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827659.572817:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827660.468788:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827660.596833:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827661.492752:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787827661.556768:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827661.556778:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827661.556784:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827661.556804:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3807 00000100:00080000:3.0:1787827661.556816:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827661.556823:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787827661.556858:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f803c0 x1874669228256256/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827661.556888:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827661.556892:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827661.556894:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827661.556899:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3807 00000100:00080000:3.0:1787827661.556903:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827661.556908:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827661.556918:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827661.556936:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80b40 x1874669228256384/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827661.556948:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827661.556954:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827661.556960:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:2.0:1787827661.556961:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228256256:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787827661.556973:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3807 00000100:00080000:3.0:1787827661.556979:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827661.556983:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827661.556991:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827661.557005:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f830c0 x1874669228256512/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787827661.557059:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827661.557112:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827661/real 1787827661] req@ffff985610f803c0 x1874669228256256/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827686 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827661.557133:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f803c0 x1874669228256256/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827686 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827661.557142:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827661.557146:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827661.557149:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228256256:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827661.557175:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228256384:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827661.557206:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827661.557220:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827661/real 1787827661] req@ffff985610f80b40 x1874669228256384/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827686 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827661.557235:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80b40 x1874669228256384/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827686 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827661.557244:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827661.557247:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827661.557249:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228256384:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827661.557255:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228256512:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827661.557272:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827661.557280:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827661/real 1787827661] req@ffff985610f830c0 x1874669228256512/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827686 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827661.557292:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f830c0 x1874669228256512/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827686 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827661.557298:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827661.557300:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827661.557302:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228256512:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787827661.620869:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827662.516801:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827662.644781:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827663.540756:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827663.668816:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827664.564785:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827664.692789:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827665.588778:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827665.716900:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827666.612773:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787827666.676800:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827666.676809:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827666.676812:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827666.676821:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3812 00000100:00080000:3.0:1787827666.676831:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827666.676837:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787827666.676868:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81a40 x1874669228256640/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827666.676892:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827666.676895:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827666.676896:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827666.676899:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3812 00000100:00080000:3.0:1787827666.676901:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827666.676903:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827666.676910:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827666.676920:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f812c0 x1874669228256768/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827666.676928:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827666.676931:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827666.676933:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827666.676936:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3812 00000100:00080000:3.0:1787827666.676939:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827666.676942:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827666.676945:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827666.676950:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228256640:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787827666.676953:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83840 x1874669228256896/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787827666.677009:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827666.677036:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827666/real 1787827666] req@ffff985610f81a40 x1874669228256640/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827691 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827666.677049:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81a40 x1874669228256640/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827691 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827666.677057:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827666.677060:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827666.677063:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228256640:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827666.677068:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228256768:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827666.677082:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827666.677090:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827666/real 1787827666] req@ffff985610f812c0 x1874669228256768/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827691 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827666.677098:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f812c0 x1874669228256768/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827691 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827666.677108:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827666.677110:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827666.677112:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228256768:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827666.677130:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228256896:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827666.677141:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827666.677147:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827666/real 1787827666] req@ffff985610f83840 x1874669228256896/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827691 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827666.677154:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83840 x1874669228256896/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827691 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827666.677160:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827666.677162:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827666.677164:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228256896:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787827666.740801:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827667.636775:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827667.764816:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827668.660748:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827668.788801:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827669.684812:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827669.812769:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827670.708812:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827670.836843:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827671.732838:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787827671.796788:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827671.796798:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827671.796802:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827671.796812:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3817 00000100:00080000:2.0:1787827671.796825:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827671.796832:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787827671.796873:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140dbc00 x1874669228257024/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827671.796900:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827671.796903:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827671.796905:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827671.796908:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3817 00000100:00080000:2.0:1787827671.796911:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827671.796913:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787827671.796923:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827671.796933:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140da1c0 x1874669228257152/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827671.796940:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228257024:192.168.203.114@tcp:250:kworker.0 00000100:00080000:2.0:1787827671.796945:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827671.796949:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827671.796950:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827671.796954:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3817 00000100:00080000:2.0:1787827671.796956:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827671.796958:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787827671.796962:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827671.796972:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db840 x1874669228257280/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827671.797015:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827671.797051:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827671/real 1787827671] req@ffff9856140dbc00 x1874669228257024/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827696 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827671.797067:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140dbc00 x1874669228257024/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827696 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827671.797076:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827671.797079:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827671.797082:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228257024:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787827671.797105:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228257152:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787827671.797129:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827671.797153:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827671/real 1787827671] req@ffff9856140da1c0 x1874669228257152/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827696 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827671.797168:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140da1c0 x1874669228257152/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827696 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827671.797174:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827671.797177:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827671.797179:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228257152:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787827671.797184:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228257280:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787827671.797203:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827671.797210:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827671/real 1787827671] req@ffff9856140db840 x1874669228257280/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827696 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827671.797219:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db840 x1874669228257280/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827696 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827671.797224:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827671.797226:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827671.797228:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228257280:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787827671.860866:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827672.756790:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827672.884837:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827673.780823:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827673.908825:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827674.804771:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827674.932864:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827675.828759:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827675.956918:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827676.852837:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787827676.916830:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827676.916841:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827676.916847:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827676.916858:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3822 00000100:00080000:3.0:1787827676.916872:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827676.916879:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787827676.916926:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81680 x1874669228257408/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827676.916964:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827676.916967:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827676.916969:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827676.916973:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3822 00000100:00080000:3.0:1787827676.916976:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827676.916978:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827676.916989:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827676.917001:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80f00 x1874669228257536/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827676.917010:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827676.917013:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827676.917015:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827676.917018:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3822 00000100:00080000:3.0:1787827676.917020:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827676.917022:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827676.917028:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827676.917033:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228257408:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787827676.917041:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81e00 x1874669228257664/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827676.917150:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228257536:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787827676.917185:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228257664:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787827676.980900:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827677.876853:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827678.004874:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827678.900791:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827679.028915:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827679.924834:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000400:00000100:0.0:1787827679.989106:0:2370:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.114:988 00000400:00000100:0.0:1787827679.989151:0:2370:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.114@tcp at host 192.168.203.114 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:0.0:1787827679.989169:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:0.0:1787827679.989223:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:0.0:1787827679.989233:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000100:00100000:1.0:1787827679.989310:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827676/real 1787827679] req@ffff985610f81e00 x1874669228257664/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827701 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827679.989348:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81e00 x1874669228257664/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827701 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827679.989363:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827679.989369:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827679.989375:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228257664:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787827679.989403:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827676/real 1787827679] req@ffff985610f81680 x1874669228257408/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827701 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827679.989421:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81680 x1874669228257408/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827701 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827679.989431:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827679.989436:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827679.989440:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228257408:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787827679.989449:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827676/real 1787827679] req@ffff985610f80f00 x1874669228257536/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827701 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827679.989462:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80f00 x1874669228257536/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827701 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827679.989472:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827679.989476:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827679.989480:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228257536:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787827680.052793:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827680.948807:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827681.076953:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827681.972834:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787827682.036768:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827682.036774:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827682.036777:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827682.036785:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3827 00000100:00080000:3.0:1787827682.036794:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827682.036801:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787827682.036835:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82d00 x1874669228257792/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827682.036856:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827682.036859:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827682.036861:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827682.036864:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3827 00000100:00080000:3.0:1787827682.036866:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827682.036870:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827682.036878:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827682.036892:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82940 x1874669228257920/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827682.036905:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827682.036908:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827682.036911:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1787827682.036914:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228257792:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787827682.036919:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3827 00000100:00080000:3.0:1787827682.036923:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827682.036928:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827682.036933:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827682.036941:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82580 x1874669228258048/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827682.036977:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827682.037006:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827682/real 1787827682] req@ffff985610f82d00 x1874669228257792/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827707 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827682.037019:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82d00 x1874669228257792/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827707 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827682.037031:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827682.037034:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827682.037037:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228257792:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787827682.037056:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228257920:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787827682.037077:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827682.037096:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827682/real 1787827682] req@ffff985610f82940 x1874669228257920/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827707 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827682.037125:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82940 x1874669228257920/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827707 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827682.037132:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827682.037134:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827682.037136:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228257920:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787827682.037143:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228258048:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787827682.037165:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827682.037177:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827682/real 1787827682] req@ffff985610f82580 x1874669228258048/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827707 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827682.037188:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82580 x1874669228258048/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827707 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827682.037197:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827682.037200:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827682.037203:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228258048:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787827682.100902:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827682.996784:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827683.124824:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827684.020835:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827684.148853:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827685.044874:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827685.172813:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827686.068821:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827686.196829:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827687.092892:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787827687.156810:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827687.156822:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827687.156826:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827687.156837:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3832 00000100:00080000:2.0:1787827687.156850:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827687.156858:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787827687.156911:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db840 x1874669228258176/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827687.156942:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827687.156945:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827687.156946:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827687.156949:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3832 00000100:00080000:2.0:1787827687.156952:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827687.156954:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787827687.156965:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827687.156974:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140da1c0 x1874669228258304/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827687.156982:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827687.156985:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827687.156987:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827687.156989:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3832 00000100:00080000:2.0:1787827687.156991:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827687.156994:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787827687.156998:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827687.157006:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228258176:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827687.157015:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140dbc00 x1874669228258432/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827687.157081:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827687.157113:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827687/real 1787827687] req@ffff9856140db840 x1874669228258176/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827712 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827687.157128:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db840 x1874669228258176/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827712 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827687.157137:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827687.157141:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827687.157144:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228258176:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787827687.157150:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228258304:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787827687.157167:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827687.157176:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827687/real 1787827687] req@ffff9856140da1c0 x1874669228258304/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827712 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827687.157185:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140da1c0 x1874669228258304/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827712 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827687.157191:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827687.157193:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827687.157195:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228258304:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787827687.157234:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228258432:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787827687.157251:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827687.157260:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827687/real 1787827687] req@ffff9856140dbc00 x1874669228258432/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827712 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827687.157269:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140dbc00 x1874669228258432/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827712 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827687.157275:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827687.157277:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827687.157280:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228258432:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787827687.220917:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827688.116919:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827688.244869:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827689.140812:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827689.268810:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827690.164824:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827690.292801:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827691.188843:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827691.316855:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827692.212781:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787827692.276795:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827692.276802:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827692.276805:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827692.276814:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3837 00000100:00080000:3.0:1787827692.276823:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827692.276830:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787827692.276861:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83480 x1874669228258560/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827692.276888:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827692.276891:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827692.276892:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827692.276895:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3837 00000100:00080000:3.0:1787827692.276899:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827692.276901:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827692.276910:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827692.276922:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f821c0 x1874669228258688/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827692.276928:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228258560:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787827692.276933:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827692.276937:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827692.276939:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827692.276942:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3837 00000100:00080000:3.0:1787827692.276946:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827692.276948:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827692.276953:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827692.276964:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82580 x1874669228258816/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787827692.276998:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827692.277027:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827692/real 1787827692] req@ffff985610f83480 x1874669228258560/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827717 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827692.277041:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83480 x1874669228258560/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827717 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827692.277048:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827692.277051:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827692.277054:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228258560:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827692.277072:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f821c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228258688:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827692.277097:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827692.277105:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827692/real 1787827692] req@ffff985610f821c0 x1874669228258688/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827717 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827692.277113:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f821c0 x1874669228258688/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827717 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827692.277119:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827692.277121:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827692.277123:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f821c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228258688:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827692.277130:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228258816:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827692.277142:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827692.277148:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827692/real 1787827692] req@ffff985610f82580 x1874669228258816/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827717 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827692.277155:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82580 x1874669228258816/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827717 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827692.277160:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827692.277162:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827692.277164:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228258816:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787827692.340823:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827693.236784:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827693.364798:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827694.260776:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827694.388823:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827695.284779:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827695.412805:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827696.308754:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827696.436836:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827697.332754:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787827697.396755:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827697.396762:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827697.396766:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827697.396776:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3843 00000100:00080000:3.0:1787827697.396787:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827697.396795:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787827697.396831:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82940 x1874669228258944/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827697.396854:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827697.396856:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827697.396858:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827697.396861:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3843 00000100:00080000:3.0:1787827697.396863:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827697.396866:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827697.396874:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827697.396882:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82d00 x1874669228259072/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827697.396889:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827697.396891:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827697.396893:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827697.396896:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3843 00000100:00080000:3.0:1787827697.396898:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827697.396908:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827697.396912:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827697.396918:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80f00 x1874669228259200/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827697.396936:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228258944:192.168.203.114@tcp:250:kworker.0 00000800:00000100:2.0:1787827697.397000:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827697.397032:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827697/real 1787827697] req@ffff985610f82940 x1874669228258944/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827722 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827697.397049:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82940 x1874669228258944/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827722 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827697.397060:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827697.397063:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827697.397068:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228258944:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827697.397091:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228259072:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827697.397112:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827697.397124:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827697/real 1787827697] req@ffff985610f82d00 x1874669228259072/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827722 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827697.397136:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82d00 x1874669228259072/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827722 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827697.397165:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827697.397168:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827697.397172:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228259072:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827697.397220:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228259200:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827697.397241:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827697.397253:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827697/real 1787827697] req@ffff985610f80f00 x1874669228259200/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827722 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827697.397265:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80f00 x1874669228259200/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827722 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827697.397274:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827697.397277:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827697.397280:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228259200:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787827697.460821:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827698.356824:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827698.484817:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827699.380771:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827699.508836:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827700.404798:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827700.532840:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827701.428831:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827701.556883:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827702.452780:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787827702.516800:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827702.516812:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827702.516817:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827702.516829:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3848 00000100:00080000:3.0:1787827702.516842:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827702.516851:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787827702.516898:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81680 x1874669228259328/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827702.516927:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827702.516929:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827702.516931:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827702.516934:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3848 00000100:00080000:3.0:1787827702.516937:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827702.516953:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827702.516975:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827702.516989:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228259328:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787827702.516993:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81e00 x1874669228259456/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827702.517018:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827702.517022:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827702.517025:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827702.517030:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3848 00000100:00080000:3.0:1787827702.517035:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827702.517038:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827702.517043:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827702.517054:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83840 x1874669228259584/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787827702.517085:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827702.517149:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827702/real 1787827702] req@ffff985610f81680 x1874669228259328/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827727 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827702.517173:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81680 x1874669228259328/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827727 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827702.517183:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827702.517187:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827702.517190:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228259328:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827702.517216:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228259456:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827702.517236:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827702.517246:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827702/real 1787827702] req@ffff985610f81e00 x1874669228259456/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827727 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827702.517255:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81e00 x1874669228259456/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827727 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827702.517261:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827702.517264:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827702.517266:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228259456:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827702.517269:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228259584:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827702.517281:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827702.517287:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827702/real 1787827702] req@ffff985610f83840 x1874669228259584/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827727 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827702.517308:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83840 x1874669228259584/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827727 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827702.517317:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827702.517321:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827702.517325:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228259584:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787827702.580931:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827703.476861:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827703.604846:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827704.500895:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827704.628893:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827705.524802:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827705.652872:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827706.548807:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827706.676846:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827707.572766:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787827707.636828:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827707.636841:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827707.636846:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827707.636858:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3853 00000100:00080000:3.0:1787827707.636881:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827707.636890:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787827707.636946:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f812c0 x1874669228259712/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827707.636985:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827707.636990:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827707.636993:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827707.636998:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3853 00000100:00080000:3.0:1787827707.637003:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827707.637009:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827707.637021:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827707.637040:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81a40 x1874669228259840/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827707.637043:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228259712:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787827707.637065:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827707.637087:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827707.637090:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827707.637097:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3853 00000100:00080000:3.0:1787827707.637127:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827707.637135:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000800:00000100:2.0:1787827707.637139:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000008:00100000:3.0:1787827707.637149:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827707.637170:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f830c0 x1874669228259968/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827707.637177:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827707/real 1787827707] req@ffff985610f812c0 x1874669228259712/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827732 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827707.637218:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f812c0 x1874669228259712/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827732 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827707.637228:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827707.637232:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827707.637236:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228259712:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827707.637258:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228259840:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827707.637277:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827707.637286:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827707/real 1787827707] req@ffff985610f81a40 x1874669228259840/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827732 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827707.637296:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81a40 x1874669228259840/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827732 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827707.637308:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827707.637311:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827707.637314:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228259840:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827707.637318:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228259968:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827707.637330:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827707.637337:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827707/real 1787827707] req@ffff985610f830c0 x1874669228259968/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827732 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827707.637345:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f830c0 x1874669228259968/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827732 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827707.637351:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827707.637353:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827707.637356:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228259968:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787827707.700890:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827708.596763:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827708.724772:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827709.620789:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827709.748828:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827710.644862:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827710.772842:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827711.668784:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827711.797019:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827712.692798:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787827712.756741:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827712.756749:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827712.756752:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827712.756760:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3858 00000100:00080000:3.0:1787827712.756769:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827712.756775:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787827712.756821:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80b40 x1874669228260096/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827712.756844:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827712.756847:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827712.756848:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827712.756852:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3858 00000100:00080000:3.0:1787827712.756854:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827712.756858:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827712.756884:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827712.756893:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228260096:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787827712.756897:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f803c0 x1874669228260224/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827712.756912:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827712.756916:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827712.756919:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827712.756924:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3858 00000100:00080000:3.0:1787827712.756929:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827712.756933:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827712.756940:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000800:00000100:2.0:1787827712.756973:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787827712.756976:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80000 x1874669228260352/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827712.757007:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827712/real 1787827712] req@ffff985610f80b40 x1874669228260096/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827737 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827712.757021:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80b40 x1874669228260096/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827737 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827712.757029:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827712.757045:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827712.757049:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228260096:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827712.757073:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228260224:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827712.757109:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827712.757123:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827712/real 1787827712] req@ffff985610f803c0 x1874669228260224/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827737 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827712.757133:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f803c0 x1874669228260224/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827737 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827712.757139:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827712.757141:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827712.757144:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228260224:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827712.757148:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228260352:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827712.757163:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827712.757170:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827712/real 1787827712] req@ffff985610f80000 x1874669228260352/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827737 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827712.757177:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80000 x1874669228260352/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827737 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827712.757182:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827712.757184:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827712.757186:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228260352:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787827712.820844:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827713.716764:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827713.844777:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827714.740789:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827714.868860:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827715.764782:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827715.892865:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827716.788788:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827716.916872:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827717.812751:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787827717.876776:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827717.876785:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827717.876789:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827717.876800:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3863 00000100:00080000:2.0:1787827717.876810:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827717.876817:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787827717.876852:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8f00 x1874669228260480/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827717.876880:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827717.876884:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827717.876886:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827717.876890:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3863 00000100:00080000:2.0:1787827717.876893:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827717.876898:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787827717.876908:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827717.876919:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228260480:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827717.876923:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140dad00 x1874669228260608/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827717.876937:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827717.876940:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827717.876942:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827717.876946:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3863 00000100:00080000:2.0:1787827717.876950:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827717.876954:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787827717.876959:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827717.876970:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db0c0 x1874669228260736/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827717.876979:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827717.877024:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827717/real 1787827717] req@ffff9856140d8f00 x1874669228260480/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827742 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827717.877038:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8f00 x1874669228260480/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827742 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827717.877046:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827717.877048:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827717.877051:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228260480:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787827717.877071:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228260608:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787827717.877086:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827717.877095:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827717/real 1787827717] req@ffff9856140dad00 x1874669228260608/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827742 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827717.877103:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140dad00 x1874669228260608/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827742 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827717.877109:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827717.877111:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827717.877113:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228260608:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787827717.877117:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228260736:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787827717.877127:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827717.877132:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827717/real 1787827717] req@ffff9856140db0c0 x1874669228260736/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827742 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827717.877140:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db0c0 x1874669228260736/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827742 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827717.877145:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827717.877146:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827717.877148:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228260736:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787827717.940901:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827718.836821:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827718.964870:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827719.860761:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827719.988829:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827720.884767:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827721.012823:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827721.908778:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827722.036946:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827722.932824:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787827722.996809:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827722.996818:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827722.996821:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827722.996841:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3868 00000100:00080000:3.0:1787827722.996850:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827722.996856:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787827722.996886:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83c00 x1874669228260864/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827722.996918:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827722.996921:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827722.996923:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827722.996925:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3868 00000100:00080000:3.0:1787827722.996929:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827722.996932:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827722.996939:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827722.996949:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80780 x1874669228260992/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827722.996957:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827722.996960:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827722.996971:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1787827722.996977:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228260864:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787827722.996978:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3868 00000100:00080000:3.0:1787827722.996983:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827722.996988:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827722.996991:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827722.997003:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80000 x1874669228261120/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827722.997050:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827722.997076:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827722/real 1787827722] req@ffff985610f83c00 x1874669228260864/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827747 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827722.997089:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83c00 x1874669228260864/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827747 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827722.997101:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827722.997104:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827722.997107:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228260864:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787827722.997125:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228260992:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787827722.997140:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827722.997148:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827722/real 1787827722] req@ffff985610f80780 x1874669228260992/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827747 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827722.997156:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80780 x1874669228260992/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827747 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827722.997161:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827722.997163:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827722.997166:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228260992:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787827722.997169:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228261120:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787827722.997179:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827722.997185:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827722/real 1787827722] req@ffff985610f80000 x1874669228261120/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827747 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827722.997193:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80000 x1874669228261120/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827747 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827722.997198:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827722.997200:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827722.997202:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228261120:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787827723.060822:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827723.956780:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827724.084855:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827724.980762:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827725.108896:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827726.004820:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827726.132929:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827727.028841:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827727.157063:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827728.052808:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787827728.116771:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827728.116793:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827728.116800:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827728.116814:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3873 00000100:00080000:3.0:1787827728.116829:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827728.116839:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787827728.116892:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f803c0 x1874669228261248/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827728.116921:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827728.116938:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827728.116940:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827728.116960:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3873 00000100:00080000:3.0:1787827728.116965:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827728.116969:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827728.116980:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827728.116994:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80b40 x1874669228261376/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827728.117004:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827728.117024:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827728.117026:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827728.117032:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3873 00000100:00080000:3.0:1787827728.117036:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00100000:1.0:1787827728.117040:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228261248:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787827728.117043:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827728.117052:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827728.117066:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f830c0 x1874669228261504/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827728.117164:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827728.117238:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827728/real 1787827728] req@ffff985610f803c0 x1874669228261248/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827753 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827728.117257:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f803c0 x1874669228261248/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827753 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827728.117268:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827728.117272:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827728.117277:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228261248:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787827728.117283:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228261376:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787827728.117314:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827728.117329:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827728/real 1787827728] req@ffff985610f80b40 x1874669228261376/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827753 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827728.117343:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80b40 x1874669228261376/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827753 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827728.117350:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827728.117353:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827728.117356:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228261376:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787827728.117382:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228261504:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787827728.117399:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827728.117416:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827728/real 1787827728] req@ffff985610f830c0 x1874669228261504/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827753 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827728.117427:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f830c0 x1874669228261504/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827753 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827728.117437:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827728.117441:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827728.117445:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228261504:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787827728.180998:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827729.076762:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827729.204940:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827730.100764:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827730.228802:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827731.124774:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827731.252876:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827732.148845:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827732.276923:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827733.172845:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787827733.236868:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827733.236879:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827733.236886:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827733.236938:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3878 00000100:00080000:2.0:1787827733.236952:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827733.236960:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787827733.237019:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140da940 x1874669228261632/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827733.237048:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827733.237051:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827733.237053:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827733.237056:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3878 00000100:00080000:2.0:1787827733.237069:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827733.237072:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787827733.237084:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827733.237102:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8b40 x1874669228261760/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827733.237109:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228261632:192.168.203.114@tcp:250:kworker.0 00000100:00080000:2.0:1787827733.237125:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827733.237131:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827733.237133:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827733.237140:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3878 00000100:00080000:2.0:1787827733.237145:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827733.237148:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787827733.237153:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827733.237192:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d9680 x1874669228261888/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827733.237217:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827733.237249:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827733/real 1787827733] req@ffff9856140da940 x1874669228261632/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827758 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827733.237263:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140da940 x1874669228261632/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827758 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827733.237272:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827733.237275:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827733.237278:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228261632:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787827733.237298:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228261760:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787827733.237313:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827733.237321:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827733/real 1787827733] req@ffff9856140d8b40 x1874669228261760/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827758 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827733.237329:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8b40 x1874669228261760/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827758 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827733.237339:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827733.237342:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827733.237344:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228261760:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787827733.237348:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228261888:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787827733.237358:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827733.237364:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827733/real 1787827733] req@ffff9856140d9680 x1874669228261888/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827758 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827733.237371:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d9680 x1874669228261888/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827758 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827733.237376:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827733.237378:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827733.237380:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228261888:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787827733.300862:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827734.196851:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827734.324801:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827735.220822:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827735.348834:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827736.244804:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827736.372784:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827737.268746:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827737.396845:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827738.292792:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787827738.356821:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827738.356831:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827738.356835:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827738.356846:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3883 00000100:00080000:3.0:1787827738.356876:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827738.356902:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787827738.356945:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81a40 x1874669228262016/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827738.356971:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827738.356974:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827738.356976:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827738.356979:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3883 00000100:00080000:3.0:1787827738.356982:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827738.356984:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827738.356993:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827738.357004:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f812c0 x1874669228262144/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827738.357019:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827738.357022:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827738.357024:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827738.357030:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3883 00000100:00100000:1.0:1787827738.357033:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228262016:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787827738.357034:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827738.357042:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827738.357048:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827738.357063:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83840 x1874669228262272/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827738.357131:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827738.357173:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827738/real 1787827738] req@ffff985610f81a40 x1874669228262016/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827763 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827738.357190:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81a40 x1874669228262016/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827763 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827738.357200:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827738.357203:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827738.357207:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228262016:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787827738.357247:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228262144:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787827738.357266:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827738.357276:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827738/real 1787827738] req@ffff985610f812c0 x1874669228262144/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827763 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827738.357285:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f812c0 x1874669228262144/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827763 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827738.357291:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827738.357294:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827738.357296:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228262144:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787827738.357300:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228262272:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787827738.357310:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827738.357317:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827738/real 1787827738] req@ffff985610f83840 x1874669228262272/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827763 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827738.357324:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83840 x1874669228262272/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827763 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827738.357330:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827738.357332:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827738.357349:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228262272:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787827738.420814:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827739.316763:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827739.444916:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827740.340804:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827740.468793:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827741.364763:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827741.492851:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827742.388783:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827742.516912:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827743.412788:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787827743.476783:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827743.476791:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827743.476795:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827743.476803:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3889 00000100:00080000:0.0:1787827743.476812:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827743.476818:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787827743.476865:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7480 x1874669228262400/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827743.476888:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827743.476891:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827743.476892:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827743.476895:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3889 00000100:00080000:0.0:1787827743.476898:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827743.476900:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787827743.476908:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827743.476923:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7840 x1874669228262528/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827743.476932:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827743.476935:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827743.476936:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1787827743.476941:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228262400:192.168.203.114@tcp:250:kworker.0 00000100:00080000:0.0:1787827743.476942:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3889 00000100:00080000:0.0:1787827743.476948:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827743.476952:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787827743.476955:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827743.476963:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df52c0 x1874669228262656/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827743.477038:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228262528:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787827743.477074:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228262656:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787827743.540883:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827744.436763:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827744.564774:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827745.460792:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827745.588775:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827746.484843:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000400:00000100:3.0:1787827746.549055:0:2371:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.114:988 00000400:00000100:3.0:1787827746.549104:0:2371:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.114@tcp at host 192.168.203.114 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:3.0:1787827746.549122:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:3.0:1787827746.549164:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:3.0:1787827746.549171:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000100:00100000:1.0:1787827746.549231:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827743/real 1787827746] req@ffff985607df7840 x1874669228262528/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827768 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827746.549256:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7840 x1874669228262528/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827768 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827746.549267:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827746.549271:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827746.549275:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228262528:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787827746.549282:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827743/real 1787827746] req@ffff985607df52c0 x1874669228262656/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827768 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827746.549290:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df52c0 x1874669228262656/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827768 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827746.549295:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827746.549298:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827746.549301:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228262656:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787827746.549305:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827743/real 1787827746] req@ffff985607df7480 x1874669228262400/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827768 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827746.549313:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7480 x1874669228262400/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827768 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827746.549317:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827746.549320:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827746.549322:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228262400:192.168.203.114@tcp:250:kworker.0 00010000:00010000:0.0:1787827746.612831:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827747.508855:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827747.636902:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827748.532814:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787827748.596855:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827748.596863:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827748.596867:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827748.596877:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3894 00000100:00080000:0.0:1787827748.596887:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827748.596896:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787827748.596925:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6940 x1874669228262784/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827748.596950:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827748.596953:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827748.596960:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827748.596965:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3894 00000100:00080000:0.0:1787827748.596968:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827748.596973:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787827748.596983:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827748.596997:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4780 x1874669228262912/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827748.597030:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:1.0:1787827748.597038:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228262784:192.168.203.114@tcp:250:kworker.0 00000100:00080000:0.0:1787827748.597039:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827748.597047:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827748.597065:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3894 00000100:00080000:0.0:1787827748.597088:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827748.597093:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787827748.597098:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827748.597111:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4b40 x1874669228263040/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827748.597149:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827748.597183:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827748/real 1787827748] req@ffff985607df6940 x1874669228262784/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827773 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827748.597203:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6940 x1874669228262784/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827773 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827748.597214:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827748.597219:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827748.597223:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228262784:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787827748.597248:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228262912:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787827748.597273:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827748.597287:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827748/real 1787827748] req@ffff985607df4780 x1874669228262912/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827773 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827748.597328:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4780 x1874669228262912/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827773 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827748.597345:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827748.597352:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827748.597373:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228262912:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787827748.597380:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228263040:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787827748.597402:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827748.597427:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827748/real 1787827748] req@ffff985607df4b40 x1874669228263040/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827773 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827748.597441:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4b40 x1874669228263040/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827773 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827748.597449:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827748.597453:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827748.597456:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228263040:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787827748.660814:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827749.556895:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827749.684859:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827750.580820:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827750.708913:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827751.604838:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827751.732826:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827752.628792:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827752.756815:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827753.652788:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787827753.716830:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827753.716838:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827753.716842:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827753.716850:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3899 00000100:00080000:1.0:1787827753.716860:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827753.716866:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787827753.716898:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2940 x1874669228263168/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827753.716923:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827753.716926:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827753.716927:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827753.716930:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3899 00000100:00080000:1.0:1787827753.716932:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827753.716935:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787827753.716943:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827753.716951:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a21c0 x1874669228263296/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827753.716960:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827753.716962:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827753.716964:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:2.0:1787827753.716969:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228263168:192.168.203.114@tcp:250:kworker.0 00000100:00080000:1.0:1787827753.716970:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3899 00000100:00080000:1.0:1787827753.716975:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827753.716980:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787827753.716983:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827753.716990:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3480 x1874669228263424/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787827753.717041:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827753.717085:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827753/real 1787827753] req@ffff9856085a2940 x1874669228263168/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827778 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827753.717103:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2940 x1874669228263168/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827778 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827753.717115:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827753.717120:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827753.717125:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228263168:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827753.717149:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228263296:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827753.717171:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827753.717184:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827753/real 1787827753] req@ffff9856085a21c0 x1874669228263296/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827778 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827753.717214:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a21c0 x1874669228263296/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827778 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827753.717223:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827753.717227:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827753.717230:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228263296:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827753.717237:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228263424:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827753.717256:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827753.717267:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827753/real 1787827753] req@ffff9856085a3480 x1874669228263424/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827778 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827753.717279:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3480 x1874669228263424/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827778 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827753.717288:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827753.717292:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827753.717295:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228263424:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787827753.780871:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827754.676769:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827754.804855:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827755.700784:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827755.828814:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827756.724768:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827756.852813:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827757.748787:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827757.876857:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827758.772773:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787827758.836858:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827758.836871:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827758.836876:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827758.836889:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3904 00000100:00080000:1.0:1787827758.836903:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827758.836913:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787827758.836980:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0f00 x1874669228263552/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827758.837025:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827758.837028:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827758.837030:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827758.837033:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3904 00000100:00080000:1.0:1787827758.837036:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827758.837039:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787827758.837066:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827758.837080:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a30c0 x1874669228263680/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827758.837090:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827758.837094:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:2.0:1787827758.837094:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228263552:192.168.203.114@tcp:250:kworker.0 00000100:00080000:1.0:1787827758.837099:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827758.837107:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3904 00000100:00080000:1.0:1787827758.837110:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827758.837113:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787827758.837117:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827758.837157:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3480 x1874669228263808/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787827758.837202:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827758.837245:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827758/real 1787827758] req@ffff9856085a0f00 x1874669228263552/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827783 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827758.837261:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0f00 x1874669228263552/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827783 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827758.837272:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827758.837276:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827758.837297:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228263552:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827758.837322:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228263680:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827758.837350:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827758.837365:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827758/real 1787827758] req@ffff9856085a30c0 x1874669228263680/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827783 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827758.837380:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a30c0 x1874669228263680/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827783 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827758.837389:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827758.837393:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827758.837397:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228263680:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827758.837404:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228263808:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827758.837425:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827758.837449:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827758/real 1787827758] req@ffff9856085a3480 x1874669228263808/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827783 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827758.837463:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3480 x1874669228263808/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827783 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827758.837472:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827758.837476:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827758.837480:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228263808:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787827758.900882:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827759.796810:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827759.924824:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827760.820781:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827760.948849:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827761.844807:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827761.972811:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827762.868777:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827762.996865:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827763.892769:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787827763.956910:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827763.956919:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827763.956923:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827763.956936:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3909 00000100:00080000:3.0:1787827763.956947:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827763.956954:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787827763.956993:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81e00 x1874669228263936/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827763.957020:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827763.957023:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827763.957029:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827763.957033:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3909 00000100:00080000:3.0:1787827763.957037:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827763.957042:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827763.957051:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827763.957062:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228263936:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787827763.957068:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81680 x1874669228264064/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827763.957083:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827763.957088:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827763.957090:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827763.957095:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3909 00000100:00080000:3.0:1787827763.957100:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827763.957106:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827763.957114:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827763.957128:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80f00 x1874669228264192/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827763.957146:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827763.957190:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827763/real 1787827763] req@ffff985610f81e00 x1874669228263936/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827788 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827763.957211:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81e00 x1874669228263936/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827788 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827763.957225:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827763.957231:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827763.957236:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228263936:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787827763.957267:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228264064:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787827763.957291:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827763.957301:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827763/real 1787827763] req@ffff985610f81680 x1874669228264064/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827788 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827763.957313:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81680 x1874669228264064/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827788 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827763.957322:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827763.957326:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827763.957329:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228264064:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787827763.957336:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228264192:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787827763.957358:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827763.957370:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827763/real 1787827763] req@ffff985610f80f00 x1874669228264192/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827788 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827763.957378:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80f00 x1874669228264192/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827788 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827763.957385:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827763.957387:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827763.957390:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228264192:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787827764.020883:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827764.916826:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827765.044888:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827765.940793:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827766.068839:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827766.964815:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827767.092857:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827767.988780:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827768.116864:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827769.012818:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787827769.076788:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827769.076796:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827769.076799:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827769.076807:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3914 00000100:00080000:1.0:1787827769.076816:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827769.076823:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787827769.076855:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a21c0 x1874669228264320/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827769.076878:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827769.076881:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827769.076882:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827769.076884:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3914 00000100:00080000:1.0:1787827769.076887:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827769.076890:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787827769.076897:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827769.076906:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2940 x1874669228264448/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827769.076913:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228264320:192.168.203.114@tcp:250:kworker.0 00000100:00080000:1.0:1787827769.076917:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827769.076920:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827769.076922:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827769.076924:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3914 00000100:00080000:1.0:1787827769.076926:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827769.076928:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787827769.076932:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827769.076940:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a1a40 x1874669228264576/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787827769.076968:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827769.076995:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827769/real 1787827769] req@ffff9856085a21c0 x1874669228264320/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827794 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827769.077008:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a21c0 x1874669228264320/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827794 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827769.077015:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827769.077019:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827769.077021:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228264320:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827769.077048:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228264448:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827769.077064:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827769.077075:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827769/real 1787827769] req@ffff9856085a2940 x1874669228264448/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827794 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827769.077088:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2940 x1874669228264448/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827794 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827769.077104:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827769.077108:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827769.077112:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228264448:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827769.077119:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228264576:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827769.077136:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827769.077144:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827769/real 1787827769] req@ffff9856085a1a40 x1874669228264576/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827794 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827769.077152:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a1a40 x1874669228264576/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827794 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827769.077161:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827769.077164:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827769.077168:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228264576:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787827769.140828:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827770.036764:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827770.164836:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827771.060797:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827771.188810:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827772.084760:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827772.212826:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827773.108784:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827773.236848:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827774.132783:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787827774.196820:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827774.196829:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827774.196834:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827774.196856:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3919 00000100:00080000:2.0:1787827774.196867:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827774.196875:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787827774.196919:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d83c0 x1874669228264704/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827774.196941:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827774.196943:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827774.196945:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827774.196948:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3919 00000100:00080000:2.0:1787827774.196951:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827774.196954:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787827774.196973:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827774.196985:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d9a40 x1874669228264832/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827774.196986:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228264704:192.168.203.114@tcp:250:kworker.0 00000100:00080000:2.0:1787827774.197001:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827774.197004:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827774.197005:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827774.197007:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3919 00000100:00080000:2.0:1787827774.197009:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827774.197011:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787827774.197014:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827774.197036:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db480 x1874669228264960/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827774.197070:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827774.197107:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827774/real 1787827774] req@ffff9856140d83c0 x1874669228264704/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827799 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827774.197126:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d83c0 x1874669228264704/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827799 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827774.197135:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827774.197138:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827774.197141:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228264704:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787827774.197170:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228264832:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787827774.197190:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827774.197200:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827774/real 1787827774] req@ffff9856140d9a40 x1874669228264832/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827799 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827774.197209:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d9a40 x1874669228264832/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827799 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827774.197215:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827774.197217:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827774.197220:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228264832:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787827774.197224:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228264960:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787827774.197242:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827774.197250:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827774/real 1787827774] req@ffff9856140db480 x1874669228264960/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827799 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827774.197257:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db480 x1874669228264960/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827799 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827774.197267:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827774.197269:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827774.197272:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228264960:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787827774.260801:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827775.156792:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827775.284811:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827776.180760:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827776.308820:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827777.204792:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827777.332772:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827778.228766:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827778.356911:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827779.252781:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787827779.316806:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827779.316816:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827779.316820:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827779.316830:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3924 00000100:00080000:2.0:1787827779.316840:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827779.316848:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787827779.316887:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8780 x1874669228265088/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827779.316917:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827779.316921:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827779.316923:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827779.316926:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3924 00000100:00080000:2.0:1787827779.316929:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827779.316932:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787827779.316940:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827779.316953:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db480 x1874669228265216/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827779.316963:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827779.316966:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827779.316968:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827779.316971:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3924 00000100:00080000:2.0:1787827779.316975:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00100000:1.0:1787827779.316979:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228265088:192.168.203.114@tcp:250:kworker.0 00000100:00080000:2.0:1787827779.316980:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787827779.316989:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827779.316999:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d9a40 x1874669228265344/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827779.317070:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827779.317102:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827779/real 1787827779] req@ffff9856140d8780 x1874669228265088/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827804 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827779.317121:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8780 x1874669228265088/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827804 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827779.317132:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827779.317137:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827779.317141:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228265088:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787827779.317146:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228265216:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787827779.317165:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827779.317174:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827779/real 1787827779] req@ffff9856140db480 x1874669228265216/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827804 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827779.317193:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db480 x1874669228265216/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827804 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827779.317199:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827779.317201:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827779.317203:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228265216:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787827779.317224:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228265344:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787827779.317237:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827779.317244:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827779/real 1787827779] req@ffff9856140d9a40 x1874669228265344/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827804 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827779.317256:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d9a40 x1874669228265344/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827804 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827779.317264:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827779.317267:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827779.317271:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228265344:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787827779.380773:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827780.276768:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827780.404821:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827781.300788:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827781.428793:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827782.324876:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827782.452838:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827783.348758:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827783.476803:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827784.372786:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787827784.436839:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827784.436850:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827784.436854:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827784.436864:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3930 00000100:00080000:0.0:1787827784.436877:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827784.436885:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787827784.436924:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7c00 x1874669228265472/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827784.436954:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827784.436958:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827784.436960:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827784.436964:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3930 00000100:00080000:0.0:1787827784.436969:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827784.436972:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787827784.436983:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827784.436998:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4000 x1874669228265600/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827784.437012:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:1.0:1787827784.437013:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228265472:192.168.203.114@tcp:250:kworker.0 00000100:00080000:0.0:1787827784.437022:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827784.437027:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827784.437032:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3930 00000100:00080000:0.0:1787827784.437038:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827784.437042:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787827784.437049:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827784.437066:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4b40 x1874669228265728/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827784.437084:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827784.437119:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827784/real 1787827784] req@ffff985607df7c00 x1874669228265472/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827809 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827784.437134:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7c00 x1874669228265472/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827809 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827784.437144:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827784.437148:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827784.437152:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228265472:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787827784.437172:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228265600:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787827784.437191:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827784.437201:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827784/real 1787827784] req@ffff985607df4000 x1874669228265600/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827809 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827784.437212:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4000 x1874669228265600/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827809 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827784.437219:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827784.437222:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827784.437224:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228265600:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787827784.437229:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228265728:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787827784.437242:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827784.437250:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827784/real 1787827784] req@ffff985607df4b40 x1874669228265728/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827809 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827784.437259:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4b40 x1874669228265728/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827809 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827784.437265:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827784.437267:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827784.437270:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228265728:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787827784.500835:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827785.396806:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827785.524871:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827786.420828:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827786.548895:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827787.444815:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827787.572779:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827788.468863:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827788.596875:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827789.492779:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787827789.556784:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827789.556794:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827789.556799:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827789.556810:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3935 00000100:00080000:0.0:1787827789.556821:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827789.556829:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787827789.556865:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4780 x1874669228265856/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827789.556887:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827789.556890:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827789.556892:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827789.556895:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3935 00000100:00080000:0.0:1787827789.556898:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827789.556901:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787827789.556909:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827789.556919:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6940 x1874669228265984/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827789.556928:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:1.0:1787827789.556931:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228265856:192.168.203.114@tcp:250:kworker.0 00000100:00080000:0.0:1787827789.556932:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827789.556936:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827789.556945:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3935 00000100:00080000:0.0:1787827789.556949:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827789.556954:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787827789.556960:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827789.556976:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7480 x1874669228266112/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827789.556996:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827789.557025:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827789/real 1787827789] req@ffff985607df4780 x1874669228265856/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827814 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827789.557039:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4780 x1874669228265856/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827814 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827789.557052:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827789.557055:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827789.557058:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228265856:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787827789.557076:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228265984:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787827789.557091:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827789.557100:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827789/real 1787827789] req@ffff985607df6940 x1874669228265984/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827814 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827789.557108:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6940 x1874669228265984/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827814 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827789.557114:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827789.557116:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827789.557118:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228265984:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787827789.557122:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228266112:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787827789.557133:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827789.557139:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827789/real 1787827789] req@ffff985607df7480 x1874669228266112/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827814 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827789.557147:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7480 x1874669228266112/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827814 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827789.557153:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827789.557154:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827789.557156:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228266112:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787827789.620853:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827790.516850:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827790.644837:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827791.540795:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827791.668862:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827792.564814:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827792.692784:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827793.588888:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827793.716870:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827794.612818:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787827794.676873:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827794.676882:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827794.676886:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827794.676895:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3940 00000100:00080000:1.0:1787827794.676913:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827794.676922:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787827794.676964:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a12c0 x1874669228266240/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827794.676995:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827794.676998:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827794.677000:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827794.677004:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3940 00000100:00080000:1.0:1787827794.677008:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827794.677013:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787827794.677022:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827794.677035:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2580 x1874669228266368/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827794.677046:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228266240:192.168.203.114@tcp:250:kworker.0 00000100:00080000:1.0:1787827794.677049:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827794.677059:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827794.677064:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827794.677069:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3940 00000100:00080000:1.0:1787827794.677072:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827794.677074:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787827794.677079:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827794.677091:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3840 x1874669228266496/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787827794.677149:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827794.677200:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827794/real 1787827794] req@ffff9856085a12c0 x1874669228266240/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827819 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827794.677216:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a12c0 x1874669228266240/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827819 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827794.677226:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827794.677231:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827794.677236:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228266240:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827794.677277:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228266368:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827794.677305:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827794.677319:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827794/real 1787827794] req@ffff9856085a2580 x1874669228266368/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827819 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827794.677331:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2580 x1874669228266368/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827819 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827794.677337:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827794.677340:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827794.677343:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228266368:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827794.677347:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228266496:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827794.677363:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827794.677370:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827794/real 1787827794] req@ffff9856085a3840 x1874669228266496/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827819 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827794.677378:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3840 x1874669228266496/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827819 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827794.677383:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827794.677385:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827794.677387:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228266496:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787827794.740815:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827795.636815:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827795.764827:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827796.660808:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827796.788802:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827797.684790:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827797.812870:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827798.708870:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827798.836845:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827799.732820:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787827799.796764:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827799.796771:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827799.796775:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827799.796783:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3945 00000100:00080000:1.0:1787827799.796792:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827799.796799:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787827799.796838:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a1680 x1874669228266624/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827799.796858:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827799.796861:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827799.796863:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827799.796865:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3945 00000100:00080000:1.0:1787827799.796868:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827799.796871:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787827799.796878:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827799.796887:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a03c0 x1874669228266752/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827799.796893:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827799.796896:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827799.796897:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827799.796899:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3945 00000100:00080000:1.0:1787827799.796902:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827799.796904:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787827799.796907:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827799.796917:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0780 x1874669228266880/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827799.796978:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228266624:192.168.203.114@tcp:250:kworker.0 00000800:00000100:2.0:1787827799.797065:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827799.797105:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827799/real 1787827799] req@ffff9856085a1680 x1874669228266624/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827824 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827799.797121:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a1680 x1874669228266624/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827824 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827799.797130:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827799.797134:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827799.797137:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228266624:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827799.797143:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228266752:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827799.797159:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827799.797167:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827799/real 1787827799] req@ffff9856085a03c0 x1874669228266752/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827824 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827799.797175:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a03c0 x1874669228266752/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827824 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827799.797180:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827799.797182:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827799.797185:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228266752:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827799.797189:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228266880:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827799.797201:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827799.797207:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827799/real 1787827799] req@ffff9856085a0780 x1874669228266880/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827824 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827799.797216:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0780 x1874669228266880/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827824 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827799.797222:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827799.797224:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827799.797225:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228266880:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787827799.860811:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827800.756796:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827800.884835:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827801.780843:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827801.908859:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827802.804799:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827802.932826:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827803.828817:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827803.956870:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827804.852797:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787827804.916902:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827804.916912:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827804.916918:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827804.916950:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3950 00000100:00080000:3.0:1787827804.916961:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827804.916969:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787827804.917049:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82d00 x1874669228267008/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827804.917089:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827804.917094:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827804.917097:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827804.917101:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3950 00000100:00080000:3.0:1787827804.917106:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827804.917111:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827804.917123:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827804.917159:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82940 x1874669228267136/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827804.917167:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228267008:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787827804.917194:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827804.917198:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827804.917201:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827804.917209:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3950 00000100:00080000:3.0:1787827804.917213:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827804.917217:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827804.917225:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827804.917260:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82580 x1874669228267264/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787827804.917283:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827804.917332:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827804/real 1787827804] req@ffff985610f82d00 x1874669228267008/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827829 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827804.917361:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82d00 x1874669228267008/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827829 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827804.917372:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827804.917376:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827804.917381:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228267008:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827804.917447:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228267136:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827804.917497:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827804.917532:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827804/real 1787827804] req@ffff985610f82940 x1874669228267136/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827829 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827804.917547:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82940 x1874669228267136/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827829 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827804.917556:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827804.917559:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827804.917562:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228267136:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827804.917568:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228267264:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827804.917589:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827804.917599:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827804/real 1787827804] req@ffff985610f82580 x1874669228267264/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827829 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827804.917610:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82580 x1874669228267264/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827829 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827804.917628:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827804.917631:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827804.917634:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228267264:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787827804.980894:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827805.876762:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827806.004884:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827806.900814:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827807.028883:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827807.924811:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827808.052826:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827808.948833:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827809.076917:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827809.972828:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787827810.036827:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827810.036838:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827810.036843:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827810.036855:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3955 00000100:00080000:1.0:1787827810.036868:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827810.036877:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787827810.036939:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0000 x1874669228267392/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827810.036974:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827810.036978:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827810.036981:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827810.036986:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3955 00000100:00080000:1.0:1787827810.036990:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827810.036995:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787827810.037007:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827810.037025:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3c00 x1874669228267520/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827810.037042:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827810.037051:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:2.0:1787827810.037051:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228267392:192.168.203.114@tcp:250:kworker.0 00000100:00080000:1.0:1787827810.037059:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827810.037073:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3955 00000100:00080000:1.0:1787827810.037077:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827810.037081:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787827810.037088:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827810.037104:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0b40 x1874669228267648/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827810.037204:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228267520:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827810.037268:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228267648:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787827810.100883:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827810.996809:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827811.124864:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827812.020807:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827812.148813:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827813.044809:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000400:00000100:3.0:1787827813.108916:0:2372:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.114:988 00000400:00000100:3.0:1787827813.108953:0:2372:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.114@tcp at host 192.168.203.114 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:3.0:1787827813.108968:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:3.0:1787827813.109016:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:3.0:1787827813.109026:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000100:00100000:2.0:1787827813.109059:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827810/real 1787827813] req@ffff9856085a3c00 x1874669228267520/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827835 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827813.109088:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3c00 x1874669228267520/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827835 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827813.109101:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827813.109106:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827813.109111:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228267520:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827813.109120:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827810/real 1787827813] req@ffff9856085a0b40 x1874669228267648/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827835 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827813.109132:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0b40 x1874669228267648/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827835 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827813.109141:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827813.109145:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827813.109149:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228267648:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827813.109157:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827810/real 1787827813] req@ffff9856085a0000 x1874669228267392/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827835 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827813.109168:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0000 x1874669228267392/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827835 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827813.109177:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827813.109181:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827813.109184:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228267392:192.168.203.114@tcp:250:kworker.0 00010000:00010000:0.0:1787827813.172835:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827814.068847:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827814.196923:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827815.092798:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787827815.156896:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827815.156906:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827815.156911:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827815.156923:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3960 00000100:00080000:0.0:1787827815.156936:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827815.156945:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787827815.156990:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df52c0 x1874669228267776/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827815.157030:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827815.157038:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827815.157043:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827815.157051:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3960 00000100:00080000:0.0:1787827815.157060:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827815.157070:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787827815.157086:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827815.157102:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228267776:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787827815.157120:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7840 x1874669228267904/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827815.157144:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827815.157153:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827815.157157:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827815.157166:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3960 00000100:00080000:0.0:1787827815.157176:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827815.157184:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787827815.157196:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000800:00000100:2.0:1787827815.157212:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787827815.157227:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df61c0 x1874669228268032/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827815.157284:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827815/real 1787827815] req@ffff985607df52c0 x1874669228267776/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827840 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827815.157320:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df52c0 x1874669228267776/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827840 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827815.157370:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827815.157378:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827815.157386:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228267776:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827815.157428:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228267904:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827815.157510:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827815.157539:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827815/real 1787827815] req@ffff985607df7840 x1874669228267904/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827840 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827815.157567:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7840 x1874669228267904/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827840 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827815.157599:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827815.157608:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827815.157615:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228267904:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827815.157628:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228268032:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827815.157675:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827815.157743:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827815/real 1787827815] req@ffff985607df61c0 x1874669228268032/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827840 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827815.157773:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df61c0 x1874669228268032/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827840 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827815.157790:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827815.157797:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827815.157804:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228268032:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787827815.220905:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827816.116796:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827816.244807:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827817.140781:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827817.268804:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827818.164788:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827818.292831:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827819.188760:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827819.316855:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827820.212783:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787827820.276851:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827820.276860:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827820.276865:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827820.276875:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3965 00000100:00080000:2.0:1787827820.276896:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827820.276904:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787827820.276945:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d83c0 x1874669228268160/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827820.276984:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827820.276986:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827820.276988:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827820.276990:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3965 00000100:00080000:2.0:1787827820.276994:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827820.276997:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787827820.277004:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827820.277013:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d9680 x1874669228268288/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827820.277019:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827820.277021:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827820.277023:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827820.277025:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3965 00000100:00080000:2.0:1787827820.277027:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827820.277028:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787827820.277031:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827820.277037:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8b40 x1874669228268416/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827820.277103:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228268160:192.168.203.114@tcp:250:kworker.0 00000800:00000100:1.0:1787827820.277224:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827820.277271:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827820/real 1787827820] req@ffff9856140d83c0 x1874669228268160/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827845 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827820.277328:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d83c0 x1874669228268160/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827845 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827820.277342:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827820.277347:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827820.277351:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228268160:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787827820.277358:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228268288:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787827820.277379:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827820.277390:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827820/real 1787827820] req@ffff9856140d9680 x1874669228268288/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827845 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827820.277399:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d9680 x1874669228268288/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827845 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827820.277405:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827820.277408:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827820.277410:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228268288:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787827820.277414:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228268416:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787827820.277426:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827820.277432:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827820/real 1787827820] req@ffff9856140d8b40 x1874669228268416/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827845 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827820.277440:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8b40 x1874669228268416/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827845 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827820.277445:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827820.277447:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827820.277449:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228268416:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787827820.340861:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827821.236902:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827821.364896:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827822.260830:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827822.388904:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827823.284859:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827823.412866:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827824.308861:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827824.436951:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827825.332769:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787827825.396831:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827825.396844:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827825.396851:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827825.396865:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3971 00000100:00080000:2.0:1787827825.396881:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827825.396902:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787827825.397000:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140da940 x1874669228268544/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827825.397047:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827825.397055:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827825.397059:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827825.397066:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3971 00000100:00080000:2.0:1787827825.397073:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827825.397078:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787827825.397092:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827825.397110:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db0c0 x1874669228268672/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827825.397173:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228268544:192.168.203.114@tcp:250:kworker.0 00000100:00080000:2.0:1787827825.397186:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827825.397196:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827825.397199:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827825.397210:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3971 00000100:00080000:2.0:1787827825.397219:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827825.397227:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787827825.397241:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827825.397304:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140dad00 x1874669228268800/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827825.397348:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827825.397426:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827825/real 1787827825] req@ffff9856140da940 x1874669228268544/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827850 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827825.397464:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140da940 x1874669228268544/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827850 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827825.397485:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827825.397492:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827825.397499:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228268544:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787827825.397538:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228268672:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787827825.397575:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827825.397592:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827825/real 1787827825] req@ffff9856140db0c0 x1874669228268672/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827850 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827825.397608:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db0c0 x1874669228268672/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827850 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827825.397619:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827825.397623:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827825.397628:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228268672:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787827825.397635:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228268800:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787827825.397655:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827825.397666:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827825/real 1787827825] req@ffff9856140dad00 x1874669228268800/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827850 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827825.397690:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140dad00 x1874669228268800/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827850 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827825.397768:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827825.397773:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827825.397777:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228268800:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787827825.460914:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827826.356806:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827826.484876:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827827.380779:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827827.508801:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827828.404780:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827828.532804:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827829.428793:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827829.556882:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827830.452765:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787827830.516788:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827830.516796:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827830.516800:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827830.516808:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3976 00000100:00080000:1.0:1787827830.516817:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827830.516823:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787827830.516854:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2d00 x1874669228268928/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827830.516877:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827830.516881:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827830.516883:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827830.516888:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3976 00000100:00080000:1.0:1787827830.516891:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827830.516896:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787827830.516906:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827830.516915:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228268928:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787827830.516920:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0000 x1874669228269056/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827830.516930:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827830.516934:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827830.516936:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827830.516939:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3976 00000100:00080000:1.0:1787827830.516943:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827830.516946:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787827830.516951:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827830.516960:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0b40 x1874669228269184/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787827830.516983:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827830.517015:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827830/real 1787827830] req@ffff9856085a2d00 x1874669228268928/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827855 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827830.517030:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2d00 x1874669228268928/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827855 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827830.517043:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827830.517046:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827830.517049:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228268928:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827830.517068:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228269056:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827830.517085:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827830.517094:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827830/real 1787827830] req@ffff9856085a0000 x1874669228269056/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827855 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827830.517102:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0000 x1874669228269056/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827855 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827830.517107:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827830.517110:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827830.517112:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228269056:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827830.517115:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228269184:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827830.517126:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827830.517132:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827830/real 1787827830] req@ffff9856085a0b40 x1874669228269184/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827855 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827830.517139:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0b40 x1874669228269184/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827855 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827830.517144:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827830.517146:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827830.517148:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228269184:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787827830.580789:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827831.476794:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827831.604809:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827832.500746:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827832.628821:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827833.524826:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827833.652806:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827834.548759:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827834.676883:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827835.572775:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787827835.636807:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827835.636817:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827835.636822:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827835.636831:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3981 00000100:00080000:3.0:1787827835.636841:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827835.636849:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787827835.636884:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f821c0 x1874669228269312/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827835.636913:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827835.636917:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827835.636920:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827835.636925:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3981 00000100:00080000:3.0:1787827835.636929:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827835.636934:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827835.636944:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827835.636946:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f821c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228269312:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787827835.636965:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83480 x1874669228269440/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827835.636984:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827835.636989:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827835.636991:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827835.636997:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3981 00000100:00080000:3.0:1787827835.637003:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000800:00000100:1.0:1787827835.637008:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00080000:3.0:1787827835.637009:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827835.637017:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827835.637031:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82580 x1874669228269568/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827835.637040:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827835/real 1787827835] req@ffff985610f821c0 x1874669228269312/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827860 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827835.637055:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f821c0 x1874669228269312/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827860 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827835.637063:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827835.637067:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827835.637071:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f821c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228269312:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787827835.637088:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228269440:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787827835.637103:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827835.637112:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827835/real 1787827835] req@ffff985610f83480 x1874669228269440/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827860 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827835.637119:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83480 x1874669228269440/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827860 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827835.637124:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827835.637126:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827835.637128:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228269440:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787827835.637132:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228269568:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787827835.637142:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827835.637148:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827835/real 1787827835] req@ffff985610f82580 x1874669228269568/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827860 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827835.637156:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82580 x1874669228269568/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827860 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827835.637161:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827835.637164:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827835.637166:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228269568:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787827835.700772:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827836.596783:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827836.724848:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827837.620823:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827837.748837:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827838.644820:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827838.772854:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827839.668801:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827839.796832:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827840.692815:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787827840.756822:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827840.756840:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827840.756848:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827840.756864:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3986 00000100:00080000:3.0:1787827840.756883:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827840.756897:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787827840.756952:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82940 x1874669228269696/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827840.757002:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827840.757009:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827840.757012:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827840.757020:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3986 00000100:00080000:3.0:1787827840.757026:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827840.757033:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827840.757047:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827840.757076:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82d00 x1874669228269824/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827840.757092:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827840.757101:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:1.0:1787827840.757103:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228269696:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787827840.757105:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827840.757133:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3986 00000100:00080000:3.0:1787827840.757165:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827840.757175:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827840.757186:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000800:00000100:1.0:1787827840.757203:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787827840.757219:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80f00 x1874669228269952/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827840.757244:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827840/real 1787827840] req@ffff985610f82940 x1874669228269696/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827865 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827840.757282:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82940 x1874669228269696/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827865 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827840.757295:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827840.757300:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827840.757305:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228269696:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787827840.757330:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228269824:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787827840.757350:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827840.757360:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827840/real 1787827840] req@ffff985610f82d00 x1874669228269824/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827865 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827840.757368:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82d00 x1874669228269824/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827865 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827840.757378:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827840.757381:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827840.757383:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228269824:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787827840.757388:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228269952:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787827840.757399:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827840.757407:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827840/real 1787827840] req@ffff985610f80f00 x1874669228269952/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827865 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827840.757420:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80f00 x1874669228269952/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827865 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827840.757428:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827840.757432:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827840.757436:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228269952:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787827840.820846:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827841.716762:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827841.844838:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827842.740782:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827842.868794:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827843.764786:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827843.892793:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827844.788801:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827844.916858:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827845.812806:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787827845.876811:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827845.876821:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827845.876825:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827845.876834:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3991 00000100:00080000:3.0:1787827845.876853:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827845.876862:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787827845.876907:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81680 x1874669228270080/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827845.876937:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827845.876940:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827845.876942:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827845.876945:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3991 00000100:00080000:3.0:1787827845.876949:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827845.876953:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827845.876961:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827845.876974:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81e00 x1874669228270208/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827845.876983:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827845.876986:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827845.876988:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827845.876991:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3991 00000100:00080000:3.0:1787827845.876993:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827845.876997:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787827845.877001:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228270080:192.168.203.114@tcp:250:kworker.0 00000008:00100000:3.0:1787827845.877003:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827845.877017:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83840 x1874669228270336/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827845.877069:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827845.877102:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827845/real 1787827845] req@ffff985610f81680 x1874669228270080/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827870 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827845.877140:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81680 x1874669228270080/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827870 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827845.877150:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827845.877153:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827845.877157:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228270080:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787827845.877163:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228270208:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787827845.877183:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827845.877193:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827845/real 1787827845] req@ffff985610f81e00 x1874669228270208/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827870 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827845.877203:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81e00 x1874669228270208/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827870 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827845.877209:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827845.877211:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827845.877214:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228270208:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787827845.877235:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228270336:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787827845.877255:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827845.877285:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827845/real 1787827845] req@ffff985610f83840 x1874669228270336/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827870 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827845.877301:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83840 x1874669228270336/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827870 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827845.877311:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827845.877315:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827845.877320:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228270336:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787827845.940834:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827846.836864:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827846.966848:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827847.860775:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827847.988872:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827848.884774:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827849.012832:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827849.908804:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827850.036913:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827850.932821:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787827850.996845:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827850.996856:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827850.996860:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827850.996871:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 3996 00000100:00080000:2.0:1787827850.996882:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827850.996907:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787827850.996951:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8f00 x1874669228270464/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827850.996980:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827850.996983:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827850.996985:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827850.996988:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3996 00000100:00080000:2.0:1787827850.996991:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827850.996994:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787827850.997004:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827850.997013:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140dbc00 x1874669228270592/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827850.997023:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827850.997026:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827850.997027:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827850.997030:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 3996 00000100:00080000:2.0:1787827850.997032:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827850.997039:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787827850.997040:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228270464:192.168.203.114@tcp:250:kworker.0 00000008:00100000:2.0:1787827850.997046:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827850.997056:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140da1c0 x1874669228270720/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827850.997265:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827850.997349:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827850/real 1787827850] req@ffff9856140d8f00 x1874669228270464/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827875 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827850.997388:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8f00 x1874669228270464/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827875 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827850.997398:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827850.997402:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827850.997405:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228270464:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787827850.997435:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228270592:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787827850.997455:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827850.997465:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827850/real 1787827850] req@ffff9856140dbc00 x1874669228270592/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827875 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827850.997474:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140dbc00 x1874669228270592/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827875 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827850.997480:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827850.997482:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827850.997485:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228270592:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787827850.997489:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228270720:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787827850.997500:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827850.997507:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827850/real 1787827850] req@ffff9856140da1c0 x1874669228270720/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827875 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827850.997514:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140da1c0 x1874669228270720/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827875 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827850.997524:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827850.997526:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827850.997529:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228270720:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787827851.060840:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827851.956786:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827852.084973:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827852.980834:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827853.108814:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827854.004870:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827854.132846:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827855.028840:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827855.156878:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827856.052776:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787827856.116798:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827856.116808:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827856.116812:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827856.116823:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4001 00000100:00080000:3.0:1787827856.116833:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827856.116840:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787827856.116880:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f812c0 x1874669228270848/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827856.116906:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827856.116909:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827856.116910:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827856.116913:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4001 00000100:00080000:3.0:1787827856.116915:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827856.116918:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827856.116926:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827856.116934:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81a40 x1874669228270976/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827856.116940:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827856.116943:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827856.116944:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827856.116946:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4001 00000100:00080000:3.0:1787827856.116947:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827856.116949:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827856.116952:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827856.116958:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f830c0 x1874669228271104/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827856.116984:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228270848:192.168.203.114@tcp:250:kworker.0 00000800:00000100:1.0:1787827856.117059:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827856.117100:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827856/real 1787827856] req@ffff985610f812c0 x1874669228270848/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827881 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827856.117113:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f812c0 x1874669228270848/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827881 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827856.117122:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827856.117125:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827856.117128:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228270848:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787827856.117133:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228270976:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787827856.117147:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827856.117155:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827856/real 1787827856] req@ffff985610f81a40 x1874669228270976/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827881 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827856.117162:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81a40 x1874669228270976/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827881 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827856.117168:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827856.117170:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827856.117172:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228270976:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787827856.117176:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228271104:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787827856.117186:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827856.117192:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827856/real 1787827856] req@ffff985610f830c0 x1874669228271104/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827881 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827856.117199:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f830c0 x1874669228271104/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827881 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827856.117204:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827856.117206:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827856.117207:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228271104:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787827856.180862:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827857.076767:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827857.204782:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827858.100908:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827858.228860:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827859.124811:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827859.252791:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827860.148826:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827860.276878:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827861.172867:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787827861.236886:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827861.236896:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827861.236902:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827861.236913:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4006 00000100:00080000:3.0:1787827861.236925:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827861.236942:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787827861.236984:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80b40 x1874669228271232/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827861.237029:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827861.237034:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827861.237054:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827861.237060:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4006 00000100:00080000:3.0:1787827861.237065:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827861.237071:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827861.237084:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827861.237104:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f803c0 x1874669228271360/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827861.237113:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228271232:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787827861.237119:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827861.237123:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827861.237125:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827861.237130:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4006 00000100:00080000:3.0:1787827861.237135:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827861.237140:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827861.237146:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827861.237157:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80000 x1874669228271488/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827861.237229:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827861.237284:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827861/real 1787827861] req@ffff985610f80b40 x1874669228271232/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827886 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827861.237302:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80b40 x1874669228271232/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827886 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827861.237312:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827861.237315:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827861.237318:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228271232:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787827861.237341:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228271360:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787827861.237360:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827861.237379:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827861/real 1787827861] req@ffff985610f803c0 x1874669228271360/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827886 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827861.237389:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f803c0 x1874669228271360/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827886 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827861.237395:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827861.237398:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827861.237400:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228271360:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787827861.237404:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228271488:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787827861.237415:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827861.237422:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827861/real 1787827861] req@ffff985610f80000 x1874669228271488/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827886 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827861.237430:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80000 x1874669228271488/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827886 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827861.237435:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827861.237437:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827861.237439:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228271488:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787827861.300786:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827862.196828:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827862.324816:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827863.220835:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827863.348874:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827864.244826:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827864.372809:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827865.268758:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827865.396831:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827866.292865:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787827866.356810:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827866.356821:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827866.356825:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827866.356834:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4011 00000100:00080000:1.0:1787827866.356844:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827866.356851:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787827866.356889:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3c00 x1874669228271616/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827866.356915:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827866.356918:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827866.356919:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827866.356922:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4011 00000100:00080000:1.0:1787827866.356924:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827866.356927:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787827866.356936:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827866.356945:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0780 x1874669228271744/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827866.356950:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228271616:192.168.203.114@tcp:250:kworker.0 00000100:00080000:1.0:1787827866.356956:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827866.356960:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827866.356961:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827866.356964:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4011 00000100:00080000:1.0:1787827866.356979:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827866.356982:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787827866.356987:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827866.356996:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a03c0 x1874669228271872/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787827866.357044:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827866.357080:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827866/real 1787827866] req@ffff9856085a3c00 x1874669228271616/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827891 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827866.357093:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3c00 x1874669228271616/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827891 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827866.357108:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827866.357111:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827866.357115:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228271616:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827866.357136:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228271744:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827866.357152:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827866.357160:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827866/real 1787827866] req@ffff9856085a0780 x1874669228271744/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827891 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827866.357167:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0780 x1874669228271744/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827891 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827866.357173:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827866.357175:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827866.357177:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228271744:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827866.357186:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228271872:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827866.357198:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827866.357204:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827866/real 1787827866] req@ffff9856085a03c0 x1874669228271872/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827891 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827866.357212:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a03c0 x1874669228271872/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827891 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827866.357217:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827866.357219:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827866.357220:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228271872:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787827866.420856:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827867.316810:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827867.444880:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827868.340892:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827868.468819:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827869.364810:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827869.492897:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827870.388932:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827870.516990:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827871.412861:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787827871.476919:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827871.476955:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827871.476962:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827871.476978:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4017 00000100:00080000:3.0:1787827871.477005:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827871.477019:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787827871.477084:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80780 x1874669228272000/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827871.477164:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827871.477187:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827871.477190:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827871.477196:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4017 00000100:00080000:3.0:1787827871.477202:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827871.477208:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827871.477221:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827871.477244:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83c00 x1874669228272128/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827871.477251:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228272000:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787827871.477271:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827871.477279:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827871.477281:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827871.477289:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4017 00000100:00080000:3.0:1787827871.477296:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827871.477304:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827871.477310:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827871.477328:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80000 x1874669228272256/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827871.477367:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827871.477465:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827871/real 1787827871] req@ffff985610f80780 x1874669228272000/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827896 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827871.477525:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80780 x1874669228272000/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827896 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827871.477550:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827871.477561:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827871.477584:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228272000:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787827871.477616:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228272128:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787827871.477650:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827871.477678:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827871/real 1787827871] req@ffff985610f83c00 x1874669228272128/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827896 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827871.477700:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83c00 x1874669228272128/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827896 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827871.477717:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827871.477774:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827871.477779:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228272128:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787827871.477788:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228272256:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787827871.477819:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827871.477841:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827871/real 1787827871] req@ffff985610f80000 x1874669228272256/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827896 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827871.477876:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80000 x1874669228272256/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827896 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827871.477898:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827871.477904:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827871.477911:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228272256:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787827871.540890:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827872.436830:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827872.564789:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827873.460789:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827873.588969:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827874.484817:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827874.612810:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827875.508846:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827875.636897:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827876.532849:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787827876.596853:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827876.596866:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827876.596871:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827876.596885:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4022 00000100:00080000:3.0:1787827876.596935:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827876.596945:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787827876.596999:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f803c0 x1874669228272384/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827876.597032:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827876.597036:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827876.597038:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827876.597043:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4022 00000100:00080000:3.0:1787827876.597047:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827876.597050:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827876.597062:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827876.597076:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80b40 x1874669228272512/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827876.597088:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827876.597095:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:1.0:1787827876.597096:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228272384:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787827876.597098:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827876.597123:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4022 00000100:00080000:3.0:1787827876.597125:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827876.597128:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827876.597131:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827876.597142:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f830c0 x1874669228272640/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827876.597231:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228272512:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787827876.597261:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228272640:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787827876.660804:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827877.556782:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827877.684861:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827878.580838:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827878.708901:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827879.604806:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000400:00000100:3.0:1787827879.669069:0:2373:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.114:988 00000400:00000100:3.0:1787827879.669132:0:2373:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.114@tcp at host 192.168.203.114 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:3.0:1787827879.669152:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:3.0:1787827879.669195:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:3.0:1787827879.669206:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000100:00100000:1.0:1787827879.669269:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827876/real 1787827879] req@ffff985610f80b40 x1874669228272512/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827901 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827879.669299:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80b40 x1874669228272512/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827901 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827879.669313:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827879.669318:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827879.669323:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228272512:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787827879.669332:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827876/real 1787827879] req@ffff985610f830c0 x1874669228272640/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827901 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827879.669346:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f830c0 x1874669228272640/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827901 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827879.669355:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827879.669359:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827879.669364:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228272640:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787827879.669371:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827876/real 1787827879] req@ffff985610f803c0 x1874669228272384/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827901 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827879.669383:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f803c0 x1874669228272384/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827901 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827879.669390:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827879.669394:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827879.669396:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228272384:192.168.203.114@tcp:250:kworker.0 00010000:00010000:0.0:1787827879.732812:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827880.628772:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827880.756887:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827881.652822:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787827881.716811:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827881.716821:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827881.716825:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827881.716835:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4027 00000100:00080000:1.0:1787827881.716845:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827881.716851:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787827881.716903:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a1680 x1874669228272768/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827881.716934:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827881.716938:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827881.716940:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827881.716945:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4027 00000100:00080000:1.0:1787827881.716949:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827881.716953:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787827881.716964:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827881.716980:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3840 x1874669228272896/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827881.716988:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827881.716992:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:2.0:1787827881.716993:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228272768:192.168.203.114@tcp:250:kworker.0 00000100:00080000:1.0:1787827881.716994:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827881.717002:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4027 00000100:00080000:1.0:1787827881.717007:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827881.717010:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787827881.717015:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827881.717029:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2580 x1874669228273024/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787827881.717067:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827881.717115:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827881/real 1787827881] req@ffff9856085a1680 x1874669228272768/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827906 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827881.717133:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a1680 x1874669228272768/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827906 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827881.717145:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827881.717150:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827881.717155:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228272768:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827881.717183:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228272896:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827881.717209:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827881.717224:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827881/real 1787827881] req@ffff9856085a3840 x1874669228272896/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827906 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827881.717243:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3840 x1874669228272896/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827906 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827881.717249:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827881.717252:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827881.717254:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228272896:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827881.717261:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228273024:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827881.717281:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827881.717294:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827881/real 1787827881] req@ffff9856085a2580 x1874669228273024/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827906 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827881.717306:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2580 x1874669228273024/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827906 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827881.717314:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827881.717317:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827881.717320:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228273024:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787827881.780828:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827882.676763:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827882.804822:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827883.700752:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827883.828830:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827884.724771:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827884.852878:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827885.748771:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827885.876856:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827886.772810:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787827886.836784:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827886.836794:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827886.836799:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827886.836808:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4032 00000100:00080000:1.0:1787827886.836840:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827886.836849:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787827886.836893:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a12c0 x1874669228273152/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827886.836926:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827886.836930:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827886.836932:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827886.836937:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4032 00000100:00080000:1.0:1787827886.836941:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827886.836945:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787827886.836954:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827886.836965:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a1a40 x1874669228273280/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827886.836973:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827886.836978:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:2.0:1787827886.836979:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228273152:192.168.203.114@tcp:250:kworker.0 00000100:00080000:1.0:1787827886.836980:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827886.836987:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4032 00000100:00080000:1.0:1787827886.836990:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827886.836992:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787827886.836995:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827886.837007:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2940 x1874669228273408/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787827886.837043:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827886.837073:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827886/real 1787827886] req@ffff9856085a12c0 x1874669228273152/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827911 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827886.837085:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a12c0 x1874669228273152/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827911 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827886.837098:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827886.837102:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827886.837105:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228273152:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827886.837124:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228273280:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827886.837138:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827886.837146:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827886/real 1787827886] req@ffff9856085a1a40 x1874669228273280/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827911 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827886.837154:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a1a40 x1874669228273280/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827911 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827886.837160:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827886.837161:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827886.837163:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228273280:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827886.837167:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228273408:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827886.837178:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827886.837183:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827886/real 1787827886] req@ffff9856085a2940 x1874669228273408/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827911 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827886.837191:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2940 x1874669228273408/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827911 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827886.837196:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827886.837197:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827886.837199:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228273408:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787827886.900790:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827887.796750:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827887.924768:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827888.820749:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827888.948887:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827889.844810:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827889.972833:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827890.868762:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827890.996904:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827891.892779:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787827891.956837:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827891.956851:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827891.956857:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827891.956869:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4037 00000100:00080000:3.0:1787827891.956880:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827891.956889:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787827891.956934:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81a40 x1874669228273536/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827891.956965:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827891.956969:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827891.956971:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827891.956975:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4037 00000100:00080000:3.0:1787827891.956980:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827891.956985:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827891.956997:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827891.957007:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228273536:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787827891.957014:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f812c0 x1874669228273664/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827891.957028:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827891.957032:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827891.957035:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827891.957039:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4037 00000100:00080000:3.0:1787827891.957056:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827891.957061:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000800:00000100:2.0:1787827891.957067:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000008:00100000:3.0:1787827891.957069:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827891.957086:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83840 x1874669228273792/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827891.957115:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827891/real 1787827891] req@ffff985610f81a40 x1874669228273536/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827916 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827891.957136:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81a40 x1874669228273536/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827916 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827891.957149:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827891.957153:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827891.957157:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228273536:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827891.957181:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228273664:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827891.957199:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827891.957208:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827891/real 1787827891] req@ffff985610f812c0 x1874669228273664/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827916 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827891.957216:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f812c0 x1874669228273664/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827916 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827891.957222:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827891.957224:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827891.957227:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228273664:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827891.957230:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228273792:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827891.957241:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827891.957246:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827891/real 1787827891] req@ffff985610f83840 x1874669228273792/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827916 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827891.957255:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83840 x1874669228273792/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827916 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827891.957260:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827891.957261:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827891.957263:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228273792:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787827892.020820:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827892.916746:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827893.044842:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827893.940811:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827894.068856:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827894.964778:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827895.092830:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827895.988804:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827896.116842:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827897.012748:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787827897.076803:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827897.076813:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827897.076817:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827897.076827:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4042 00000100:00080000:3.0:1787827897.076837:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827897.076843:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787827897.076882:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81e00 x1874669228273920/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827897.076906:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827897.076909:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827897.076911:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827897.076914:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4042 00000100:00080000:3.0:1787827897.076916:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827897.076919:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827897.076928:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827897.076938:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81680 x1874669228274048/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827897.076946:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1787827897.076951:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228273920:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787827897.076951:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827897.076955:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827897.076962:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4042 00000100:00080000:3.0:1787827897.076965:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827897.076967:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827897.076969:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827897.076979:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80f00 x1874669228274176/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787827897.077015:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827897.077046:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827897/real 1787827897] req@ffff985610f81e00 x1874669228273920/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827922 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827897.077058:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81e00 x1874669228273920/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827922 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827897.077068:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827897.077071:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827897.077074:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228273920:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827897.077095:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228274048:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827897.077109:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827897.077117:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827897/real 1787827897] req@ffff985610f81680 x1874669228274048/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827922 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827897.077124:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81680 x1874669228274048/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827922 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827897.077135:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827897.077137:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827897.077139:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228274048:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827897.077143:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228274176:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827897.077153:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827897.077159:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827897/real 1787827897] req@ffff985610f80f00 x1874669228274176/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827922 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827897.077165:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80f00 x1874669228274176/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827922 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827897.077171:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827897.077172:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827897.077174:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228274176:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787827897.140839:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827898.036838:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827898.164785:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827899.060789:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827899.188830:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827900.084792:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827900.212794:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827901.108803:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827901.236841:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827902.132794:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787827902.196807:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827902.196817:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827902.196822:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827902.196833:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4047 00000100:00080000:2.0:1787827902.196846:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827902.196854:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787827902.196895:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db840 x1874669228274304/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827902.196927:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827902.196933:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827902.196935:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827902.196939:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4047 00000100:00080000:2.0:1787827902.196944:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827902.196948:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787827902.196959:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827902.196980:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228274304:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827902.196980:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140da1c0 x1874669228274432/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827902.196998:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827902.197003:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827902.197005:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827902.197010:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4047 00000100:00080000:2.0:1787827902.197015:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827902.197019:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787827902.197025:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827902.197039:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140dbc00 x1874669228274560/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827902.197056:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827902.197091:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827902/real 1787827902] req@ffff9856140db840 x1874669228274304/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827927 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827902.197105:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db840 x1874669228274304/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827927 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827902.197113:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827902.197116:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827902.197119:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228274304:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787827902.197141:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228274432:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787827902.197165:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827902.197175:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827902/real 1787827902] req@ffff9856140da1c0 x1874669228274432/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827927 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827902.197183:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140da1c0 x1874669228274432/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827927 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827902.197189:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827902.197192:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827902.197194:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228274432:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787827902.197198:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228274560:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787827902.197209:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827902.197215:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827902/real 1787827902] req@ffff9856140dbc00 x1874669228274560/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827927 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827902.197223:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140dbc00 x1874669228274560/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827927 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827902.197228:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827902.197230:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827902.197232:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228274560:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787827902.260801:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827903.156833:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827903.284830:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827904.180801:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827904.308869:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827905.204827:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827905.332817:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827906.228780:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827906.356916:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827907.252823:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787827907.316809:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827907.316821:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827907.316826:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827907.316837:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4052 00000100:00080000:3.0:1787827907.316851:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827907.316858:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787827907.316906:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82d00 x1874669228274688/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827907.316939:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827907.316943:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827907.316946:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827907.316952:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4052 00000100:00080000:3.0:1787827907.316958:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827907.316964:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787827907.316976:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228274688:192.168.203.114@tcp:250:kworker.0 00000008:00100000:3.0:1787827907.316977:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827907.317009:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82940 x1874669228274816/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827907.317021:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827907.317042:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827907.317044:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827907.317049:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4052 00000100:00080000:3.0:1787827907.317062:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827907.317066:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827907.317073:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000800:00000100:2.0:1787827907.317078:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827907.317136:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827907/real 1787827907] req@ffff985610f82d00 x1874669228274688/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827932 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827907.317161:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82d00 x1874669228274688/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827932 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827907.317178:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00100000:3.0:1787827907.317184:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82580 x1874669228274944/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827907.317188:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827907.317198:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228274688:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827907.317237:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228274816:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827907.317255:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827907.317266:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827907/real 1787827907] req@ffff985610f82940 x1874669228274816/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827932 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827907.317276:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82940 x1874669228274816/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827932 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827907.317283:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827907.317285:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827907.317288:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228274816:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827907.317292:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228274944:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827907.317304:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827907.317312:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827907/real 1787827907] req@ffff985610f82580 x1874669228274944/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827932 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827907.317321:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82580 x1874669228274944/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827932 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827907.317344:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827907.317347:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827907.317350:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228274944:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787827907.380827:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827908.276768:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827908.404871:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827909.300760:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827909.428801:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827910.324757:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827910.452804:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827911.348835:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827911.476805:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827912.372761:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787827912.436771:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827912.436780:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827912.436785:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827912.436795:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4058 00000100:00080000:3.0:1787827912.436804:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827912.436811:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787827912.436849:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83480 x1874669228275072/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827912.436879:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827912.436883:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827912.436884:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827912.436888:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4058 00000100:00080000:3.0:1787827912.436891:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827912.436894:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827912.436903:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827912.436914:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f821c0 x1874669228275200/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827912.436923:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1787827912.436926:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228275072:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787827912.436926:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827912.436930:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827912.436937:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4058 00000100:00080000:3.0:1787827912.436939:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827912.436941:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827912.436944:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827912.436955:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82580 x1874669228275328/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787827912.436990:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827912.437018:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827912/real 1787827912] req@ffff985610f83480 x1874669228275072/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827937 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827912.437032:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83480 x1874669228275072/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827937 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827912.437040:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827912.437043:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827912.437046:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228275072:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827912.437066:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f821c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228275200:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827912.437081:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827912.437089:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827912/real 1787827912] req@ffff985610f821c0 x1874669228275200/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827937 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827912.437096:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f821c0 x1874669228275200/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827937 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827912.437102:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827912.437112:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827912.437115:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f821c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228275200:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827912.437121:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228275328:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827912.437133:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827912.437139:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827912/real 1787827912] req@ffff985610f82580 x1874669228275328/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827937 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827912.437148:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82580 x1874669228275328/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827937 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827912.437153:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827912.437154:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827912.437156:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228275328:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787827912.500806:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827913.396792:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827913.524790:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827914.420789:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827914.548815:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827915.444790:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827915.572836:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827916.468755:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827916.596846:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827917.492812:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787827917.556836:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827917.556846:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827917.556852:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827917.556862:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4063 00000100:00080000:1.0:1787827917.556872:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827917.556878:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787827917.556916:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a21c0 x1874669228275456/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827917.556942:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827917.556945:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827917.556946:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827917.556949:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4063 00000100:00080000:1.0:1787827917.556951:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827917.556954:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787827917.556962:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827917.556971:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3480 x1874669228275584/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827917.556978:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827917.556982:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827917.556983:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827917.556986:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4063 00000100:00080000:1.0:1787827917.556990:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827917.556994:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787827917.556997:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827917.557003:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a30c0 x1874669228275712/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827917.557030:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228275456:192.168.203.114@tcp:250:kworker.0 00000800:00000100:2.0:1787827917.557107:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827917.557146:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827917/real 1787827917] req@ffff9856085a21c0 x1874669228275456/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827942 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827917.557160:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a21c0 x1874669228275456/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827942 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827917.557170:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827917.557174:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827917.557177:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228275456:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827917.557182:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228275584:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827917.557197:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827917.557206:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827917/real 1787827917] req@ffff9856085a3480 x1874669228275584/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827942 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827917.557214:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3480 x1874669228275584/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827942 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827917.557225:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827917.557227:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827917.557229:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228275584:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827917.557252:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228275712:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827917.557263:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827917.557269:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827917/real 1787827917] req@ffff9856085a30c0 x1874669228275712/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827942 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827917.557276:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a30c0 x1874669228275712/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827942 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827917.557282:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827917.557284:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827917.557286:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228275712:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787827917.620798:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827918.516789:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827918.644804:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827919.540769:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827919.668781:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827920.564815:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827920.692806:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827921.588798:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827921.716816:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827922.612797:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787827922.676761:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827922.676778:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827922.676782:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827922.676791:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4068 00000100:00080000:1.0:1787827922.676802:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827922.676808:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787827922.676843:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0f00 x1874669228275840/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827922.676868:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827922.676871:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827922.676872:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827922.676875:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4068 00000100:00080000:1.0:1787827922.676878:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827922.676880:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787827922.676888:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827922.676899:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a30c0 x1874669228275968/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827922.676906:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827922.676909:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827922.676911:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827922.676914:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4068 00000100:00080000:1.0:1787827922.676916:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827922.676919:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787827922.676923:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827922.676932:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228275840:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787827922.676933:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3480 x1874669228276096/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787827922.677025:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827922.677066:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827922/real 1787827922] req@ffff9856085a0f00 x1874669228275840/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827947 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827922.677090:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0f00 x1874669228275840/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827947 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827922.677099:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827922.677102:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827922.677106:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228275840:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827922.677112:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228275968:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827922.677132:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827922.677140:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827922/real 1787827922] req@ffff9856085a30c0 x1874669228275968/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827947 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827922.677148:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a30c0 x1874669228275968/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827947 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827922.677153:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827922.677156:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827922.677158:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228275968:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827922.677181:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228276096:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827922.677192:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827922.677199:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827922/real 1787827922] req@ffff9856085a3480 x1874669228276096/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827947 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827922.677206:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3480 x1874669228276096/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827947 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827922.677211:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827922.677213:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827922.677215:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228276096:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787827922.740785:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827923.636838:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827923.764858:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827924.660820:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827924.788891:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827925.684769:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827925.812840:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827926.708848:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827926.836933:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827927.732794:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787827927.796878:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827927.796898:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827927.796906:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827927.796925:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4073 00000100:00080000:3.0:1787827927.796958:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827927.796970:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787827927.797030:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82940 x1874669228276224/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827927.797071:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827927.797076:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827927.797079:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827927.797084:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4073 00000100:00080000:3.0:1787827927.797089:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827927.797093:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827927.797106:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827927.797124:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82d00 x1874669228276352/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827927.797141:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827927.797145:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827927.797148:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:2.0:1787827927.797161:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228276224:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787827927.797161:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4073 00000100:00080000:3.0:1787827927.797181:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827927.797188:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827927.797195:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827927.797214:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80f00 x1874669228276480/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787827927.797273:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827927.797324:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827927/real 1787827927] req@ffff985610f82940 x1874669228276224/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827952 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827927.797357:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82940 x1874669228276224/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827952 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827927.797396:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827927.797405:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827927.797414:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228276224:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827927.797454:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228276352:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827927.797490:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827927.797508:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827927/real 1787827927] req@ffff985610f82d00 x1874669228276352/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827952 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827927.797525:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82d00 x1874669228276352/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827952 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827927.797536:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827927.797570:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827927.797576:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228276352:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827927.797585:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228276480:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827927.797624:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827927.797639:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827927/real 1787827927] req@ffff985610f80f00 x1874669228276480/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827952 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827927.797655:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80f00 x1874669228276480/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827952 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827927.797677:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827927.797684:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827927.797743:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228276480:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787827927.860880:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827928.756852:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827928.884799:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827929.780847:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827929.908817:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827930.804894:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827930.932934:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827931.828784:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827931.956924:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827932.852777:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787827932.916771:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827932.916781:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827932.916786:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827932.916814:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4078 00000100:00080000:3.0:1787827932.916824:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827932.916831:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787827932.916887:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81680 x1874669228276608/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827932.916914:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827932.916918:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827932.916920:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827932.916923:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4078 00000100:00080000:3.0:1787827932.916926:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827932.916929:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827932.916938:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827932.916948:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81e00 x1874669228276736/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827932.916962:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827932.916965:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827932.916967:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827932.916971:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4078 00000100:00080000:3.0:1787827932.916974:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827932.916977:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827932.916981:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827932.916988:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83840 x1874669228276864/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827932.917003:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228276608:192.168.203.114@tcp:250:kworker.0 00000800:00000100:2.0:1787827932.917089:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827932.917133:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827932/real 1787827932] req@ffff985610f81680 x1874669228276608/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827957 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827932.917177:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81680 x1874669228276608/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827957 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827932.917189:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827932.917193:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827932.917197:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228276608:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827932.917205:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228276736:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827932.917227:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827932.917240:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827932/real 1787827932] req@ffff985610f81e00 x1874669228276736/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827957 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827932.917277:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81e00 x1874669228276736/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827957 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827932.917290:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827932.917295:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827932.917299:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228276736:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827932.917337:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228276864:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827932.917370:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827932.917402:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827932/real 1787827932] req@ffff985610f83840 x1874669228276864/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827957 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827932.917417:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83840 x1874669228276864/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827957 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827932.917427:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827932.917431:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827932.917436:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228276864:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787827932.980869:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827933.876837:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827934.004889:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827934.900823:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827935.028872:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827935.924827:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827936.052887:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827936.948866:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827937.076958:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827937.972819:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787827938.036814:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827938.036821:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827938.036826:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827938.036836:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4083 00000100:00080000:2.0:1787827938.036846:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827938.036852:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787827938.036891:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8f00 x1874669228276992/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827938.036938:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827938.036944:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827938.036946:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827938.036951:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4083 00000100:00080000:2.0:1787827938.036955:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827938.036960:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787827938.036970:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827938.036984:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140dad00 x1874669228277120/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827938.036995:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827938.036998:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827938.037000:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827938.037004:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4083 00000100:00080000:2.0:1787827938.037009:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827938.037014:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787827938.037020:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827938.037033:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db0c0 x1874669228277248/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827938.037068:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228276992:192.168.203.114@tcp:250:kworker.0 00000800:00000100:1.0:1787827938.037182:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827938.037225:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827938/real 1787827938] req@ffff9856140d8f00 x1874669228276992/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827963 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827938.037242:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8f00 x1874669228276992/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827963 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827938.037253:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827938.037259:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827938.037264:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228276992:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787827938.037275:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228277120:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787827938.037306:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827938.037322:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827938/real 1787827938] req@ffff9856140dad00 x1874669228277120/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827963 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827938.037340:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140dad00 x1874669228277120/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827963 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827938.037351:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827938.037354:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827938.037357:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228277120:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787827938.037390:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228277248:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787827938.037412:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827938.037422:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827938/real 1787827938] req@ffff9856140db0c0 x1874669228277248/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827963 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827938.037432:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db0c0 x1874669228277248/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827963 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827938.037438:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827938.037441:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787827938.037444:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228277248:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787827938.100819:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827938.996871:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827939.124887:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827940.020887:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827940.148903:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827941.044771:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827941.172820:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827942.068841:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827942.196922:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827943.092816:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787827943.156879:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827943.156895:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827943.156902:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827943.156933:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4088 00000100:00080000:3.0:1787827943.156951:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827943.156964:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787827943.157010:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f812c0 x1874669228277376/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827943.157044:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827943.157048:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827943.157050:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827943.157055:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4088 00000100:00080000:3.0:1787827943.157061:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827943.157068:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827943.157081:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827943.157097:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228277376:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787827943.157100:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81a40 x1874669228277504/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827943.157121:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827943.157125:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827943.157127:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827943.157133:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4088 00000100:00080000:3.0:1787827943.157138:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827943.157141:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827943.157145:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827943.157166:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f803c0 x1874669228277632/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827943.157299:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228277504:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827943.157341:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228277632:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787827943.220916:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827944.116825:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827944.244819:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827945.140794:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827945.268787:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827946.164780:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000400:00000100:1.0:1787827946.228975:0:2370:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.114:988 00000400:00000100:1.0:1787827946.229022:0:2370:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.114@tcp at host 192.168.203.114 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1787827946.229037:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:1.0:1787827946.229074:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:1.0:1787827946.229084:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000100:00100000:2.0:1787827946.229120:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827943/real 1787827946] req@ffff985610f81a40 x1874669228277504/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827968 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827946.229157:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81a40 x1874669228277504/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827968 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827946.229170:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827946.229175:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827946.229179:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228277504:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827946.229187:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827943/real 1787827946] req@ffff985610f803c0 x1874669228277632/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827968 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827946.229199:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f803c0 x1874669228277632/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827968 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827946.229212:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827946.229216:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827946.229220:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228277632:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827946.229227:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827943/real 1787827946] req@ffff985610f812c0 x1874669228277376/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827968 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827946.229240:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f812c0 x1874669228277376/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827968 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827946.229249:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827946.229253:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827946.229256:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228277376:192.168.203.114@tcp:250:kworker.0 00010000:00010000:0.0:1787827946.292784:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827947.188761:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827947.316871:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827948.212808:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787827948.276782:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827948.276790:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827948.276794:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827948.276805:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4093 00000100:00080000:0.0:1787827948.276816:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827948.276824:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787827948.276864:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6d00 x1874669228277760/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827948.276893:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827948.276896:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827948.276898:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827948.276901:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4093 00000100:00080000:0.0:1787827948.276904:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827948.276907:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787827948.276918:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827948.276929:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4f00 x1874669228277888/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827948.276940:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827948.276946:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:2.0:1787827948.276949:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228277760:192.168.203.114@tcp:250:kworker.0 00000100:00080000:0.0:1787827948.276950:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827948.276964:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4093 00000100:00080000:0.0:1787827948.276969:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827948.276975:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787827948.276983:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827948.277001:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5e00 x1874669228278016/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787827948.277037:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827948.277077:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827948/real 1787827948] req@ffff985607df6d00 x1874669228277760/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827973 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827948.277099:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6d00 x1874669228277760/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827973 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827948.277113:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827948.277126:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827948.277131:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228277760:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827948.277167:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228277888:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827948.277197:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827948.277213:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827948/real 1787827948] req@ffff985607df4f00 x1874669228277888/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827973 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827948.277228:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4f00 x1874669228277888/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827973 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827948.277238:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827948.277242:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827948.277247:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228277888:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827948.277254:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228278016:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827948.277276:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827948.277289:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827948/real 1787827948] req@ffff985607df5e00 x1874669228278016/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827973 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827948.277305:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5e00 x1874669228278016/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827973 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827948.277315:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827948.277319:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827948.277324:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228278016:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787827948.340810:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827949.236759:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827949.364875:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827950.260777:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827950.388777:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827951.284754:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827951.412839:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827952.308845:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827952.436860:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827953.332834:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787827953.396788:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827953.396797:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827953.396800:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827953.396810:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4099 00000100:00080000:1.0:1787827953.396823:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827953.396841:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787827953.396884:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a21c0 x1874669228278144/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827953.396909:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827953.396912:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827953.396914:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827953.396917:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4099 00000100:00080000:1.0:1787827953.396930:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827953.396933:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787827953.396943:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827953.396951:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2940 x1874669228278272/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827953.396958:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827953.396961:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827953.396962:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827953.396964:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4099 00000100:00080000:1.0:1787827953.396966:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827953.396968:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787827953.396971:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827953.396976:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a1a40 x1874669228278400/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827953.397094:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228278144:192.168.203.114@tcp:250:kworker.0 00000800:00000100:2.0:1787827953.397225:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827953.397295:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827953/real 1787827953] req@ffff9856085a21c0 x1874669228278144/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827978 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827953.397320:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a21c0 x1874669228278144/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827978 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827953.397335:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827953.397341:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827953.397346:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228278144:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827953.397356:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228278272:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827953.397422:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827953.397447:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827953/real 1787827953] req@ffff9856085a2940 x1874669228278272/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827978 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827953.397472:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2940 x1874669228278272/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827978 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827953.397483:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827953.397488:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827953.397492:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228278272:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827953.397500:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228278400:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827953.397527:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827953.397540:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827953/real 1787827953] req@ffff9856085a1a40 x1874669228278400/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827978 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827953.397554:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a1a40 x1874669228278400/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827978 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827953.397563:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827953.397567:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827953.397602:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228278400:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787827953.460849:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827954.356840:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827954.484853:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827955.380815:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827955.508864:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827956.404803:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827956.532800:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827957.428785:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827957.556823:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827958.452855:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787827958.516829:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827958.516842:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827958.516848:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827958.516861:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4104 00000100:00080000:0.0:1787827958.516874:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827958.516883:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787827958.516926:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df43c0 x1874669228278528/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827958.516952:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827958.516954:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827958.516956:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827958.516960:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4104 00000100:00080000:0.0:1787827958.516963:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827958.516967:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787827958.516976:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827958.516987:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5680 x1874669228278656/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827958.516991:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228278528:192.168.203.114@tcp:250:kworker.0 00000100:00080000:0.0:1787827958.517002:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827958.517005:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827958.517007:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827958.517010:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4104 00000100:00080000:0.0:1787827958.517012:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827958.517016:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787827958.517019:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827958.517029:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df70c0 x1874669228278784/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787827958.517061:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827958.517094:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827958/real 1787827958] req@ffff985607df43c0 x1874669228278528/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827983 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827958.517126:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df43c0 x1874669228278528/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827983 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827958.517136:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827958.517139:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827958.517143:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228278528:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827958.517166:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228278656:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827958.517183:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827958.517192:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827958/real 1787827958] req@ffff985607df5680 x1874669228278656/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827983 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827958.517201:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5680 x1874669228278656/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827983 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827958.517207:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827958.517209:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827958.517212:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228278656:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827958.517216:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228278784:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827958.517227:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827958.517233:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827958/real 1787827958] req@ffff985607df70c0 x1874669228278784/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827983 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827958.517241:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df70c0 x1874669228278784/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827983 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827958.517252:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827958.517255:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827958.517257:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228278784:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787827958.580793:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827959.476815:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827959.604825:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827960.500815:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827960.628950:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827961.524875:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827961.652907:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827962.548803:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827962.676925:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827963.572894:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787827963.636809:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827963.636835:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827963.636841:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827963.636856:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4109 00000100:00080000:0.0:1787827963.636871:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827963.636882:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787827963.636941:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5a40 x1874669228278912/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827963.636981:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827963.636990:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827963.636993:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827963.636997:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4109 00000100:00080000:0.0:1787827963.637002:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827963.637006:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787827963.637029:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827963.637077:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6580 x1874669228279040/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827963.637094:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228278912:192.168.203.114@tcp:250:kworker.0 00000100:00080000:0.0:1787827963.637106:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827963.637116:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827963.637120:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827963.637161:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4109 00000100:00080000:0.0:1787827963.637170:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827963.637180:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787827963.637189:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827963.637213:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df70c0 x1874669228279168/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787827963.637229:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827963.637347:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827963/real 1787827963] req@ffff985607df5a40 x1874669228278912/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827988 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827963.637380:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5a40 x1874669228278912/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827988 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827963.637417:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827963.637425:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827963.637433:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228278912:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827963.637494:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228279040:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827963.637539:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827963.637563:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827963/real 1787827963] req@ffff985607df6580 x1874669228279040/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827988 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827963.637586:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6580 x1874669228279040/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827988 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827963.637603:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827963.637609:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827963.637616:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228279040:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827963.637628:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228279168:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827963.637660:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827963.637678:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827963/real 1787827963] req@ffff985607df70c0 x1874669228279168/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827988 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827963.637697:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df70c0 x1874669228279168/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827988 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827963.637710:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827963.637715:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827963.637720:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228279168:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787827963.700822:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827964.596793:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827964.724857:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827965.620893:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827965.748883:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827966.644853:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827966.772904:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827967.668837:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827967.796900:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827968.692812:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787827968.756849:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827968.756860:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827968.756866:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827968.756880:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4114 00000100:00080000:1.0:1787827968.756895:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827968.756905:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787827968.756967:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a12c0 x1874669228279296/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827968.757017:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827968.757022:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827968.757025:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827968.757030:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4114 00000100:00080000:1.0:1787827968.757034:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827968.757039:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787827968.757053:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827968.757070:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2580 x1874669228279424/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827968.757085:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827968.757091:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827968.757097:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827968.757103:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4114 00000100:00080000:1.0:1787827968.757107:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00100000:2.0:1787827968.757111:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228279296:192.168.203.114@tcp:250:kworker.0 00000100:00080000:1.0:1787827968.757127:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787827968.757134:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827968.757151:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3840 x1874669228279552/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787827968.757207:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827968.757280:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827968/real 1787827968] req@ffff9856085a12c0 x1874669228279296/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827993 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827968.757305:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a12c0 x1874669228279296/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827993 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827968.757321:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827968.757326:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827968.757332:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228279296:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827968.757364:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228279424:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827968.757392:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827968.757424:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827968/real 1787827968] req@ffff9856085a2580 x1874669228279424/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827993 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827968.757441:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2580 x1874669228279424/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827993 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827968.757451:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827968.757456:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827968.757460:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228279424:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827968.757467:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228279552:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827968.757489:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827968.757513:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827968/real 1787827968] req@ffff9856085a3840 x1874669228279552/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827993 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827968.757532:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3840 x1874669228279552/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827993 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827968.757543:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827968.757547:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827968.757550:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228279552:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787827968.820926:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827969.716824:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827969.844835:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827970.740854:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827970.868915:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827971.764832:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827971.892852:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827972.788826:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827972.917006:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827973.812837:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787827973.876867:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827973.876878:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827973.876885:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827973.876899:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4119 00000100:00080000:0.0:1787827973.876914:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827973.876923:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787827973.876993:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5680 x1874669228279680/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827973.877029:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827973.877034:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827973.877037:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827973.877041:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4119 00000100:00080000:0.0:1787827973.877046:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827973.877051:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787827973.877065:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827973.877081:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df43c0 x1874669228279808/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827973.877095:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827973.877100:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827973.877103:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827973.877108:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4119 00000100:00080000:0.0:1787827973.877112:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827973.877116:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787827973.877123:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827973.877131:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228279680:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787827973.877142:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5e00 x1874669228279936/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787827973.877278:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827973.877343:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827973/real 1787827973] req@ffff985607df5680 x1874669228279680/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827998 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827973.877379:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5680 x1874669228279680/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827998 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827973.877400:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827973.877409:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827973.877418:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228279680:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827973.877434:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228279808:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827973.877481:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827973.877506:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827973/real 1787827973] req@ffff985607df43c0 x1874669228279808/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827998 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827973.877530:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df43c0 x1874669228279808/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827998 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827973.877546:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827973.877553:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827973.877559:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228279808:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827973.877603:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228279936:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827973.877652:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827973.877677:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827973/real 1787827973] req@ffff985607df5e00 x1874669228279936/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827998 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827973.877754:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5e00 x1874669228279936/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827998 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827973.877770:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827973.877777:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827973.877783:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228279936:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787827973.940801:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827974.836846:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827974.964835:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827975.860794:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827975.988885:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827976.884856:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827977.012878:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827977.908789:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827978.036902:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827978.932825:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787827978.996817:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827978.996829:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827978.996835:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827978.996849:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4124 00000100:00080000:0.0:1787827978.996876:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827978.996888:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787827978.996947:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4f00 x1874669228280064/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827978.996988:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827978.996993:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827978.996996:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827978.997001:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4124 00000100:00080000:0.0:1787827978.997005:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827978.997010:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787827978.997024:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827978.997053:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6d00 x1874669228280192/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827978.997074:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1787827978.997074:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228280064:192.168.203.114@tcp:250:kworker.0 00000100:00080000:0.0:1787827978.997087:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827978.997094:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827978.997101:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4124 00000100:00080000:0.0:1787827978.997105:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827978.997109:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787827978.997116:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827978.997136:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df61c0 x1874669228280320/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787827978.997180:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827978.997261:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827978/real 1787827978] req@ffff985607df4f00 x1874669228280064/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828003 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827978.997301:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4f00 x1874669228280064/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828003 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827978.997323:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827978.997332:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827978.997342:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228280064:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827978.997388:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228280192:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827978.997441:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827978.997470:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827978/real 1787827978] req@ffff985607df6d00 x1874669228280192/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828003 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827978.997498:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6d00 x1874669228280192/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828003 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827978.997517:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827978.997526:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827978.997534:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228280192:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827978.997549:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228280320:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827978.997607:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827978.997630:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827978/real 1787827978] req@ffff985607df61c0 x1874669228280320/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828003 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827978.997654:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df61c0 x1874669228280320/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828003 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827978.997671:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827978.997677:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827978.997684:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228280320:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787827979.060945:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827979.956795:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827980.084830:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827980.980872:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827981.108814:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827982.004879:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827982.132897:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827983.028826:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827983.156883:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827984.052858:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787827984.116877:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827984.116888:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827984.116894:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827984.116926:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4129 00000100:00080000:0.0:1787827984.116940:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827984.116950:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787827984.117010:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7840 x1874669228280448/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827984.117052:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827984.117057:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827984.117060:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827984.117064:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4129 00000100:00080000:0.0:1787827984.117068:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827984.117073:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787827984.117087:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827984.117106:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df52c0 x1874669228280576/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827984.117124:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228280448:192.168.203.114@tcp:250:kworker.0 00000100:00080000:0.0:1787827984.117125:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827984.117141:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827984.117144:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827984.117174:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4129 00000100:00080000:0.0:1787827984.117197:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827984.117202:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787827984.117209:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827984.117229:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7480 x1874669228280704/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787827984.117270:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827984.117330:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827984/real 1787827984] req@ffff985607df7840 x1874669228280448/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828009 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827984.117356:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7840 x1874669228280448/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828009 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827984.117371:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827984.117376:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827984.117382:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228280448:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827984.117431:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228280576:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827984.117466:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827984.117483:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827984/real 1787827984] req@ffff985607df52c0 x1874669228280576/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828009 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827984.117519:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df52c0 x1874669228280576/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828009 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827984.117531:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827984.117535:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827984.117539:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228280576:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827984.117548:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228280704:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827984.117573:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827984.117585:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827984/real 1787827984] req@ffff985607df7480 x1874669228280704/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828009 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827984.117600:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7480 x1874669228280704/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828009 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827984.117610:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827984.117614:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827984.117618:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228280704:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787827984.180949:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827985.076860:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827985.204837:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827986.100771:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827986.228843:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827987.124792:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827987.252790:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827988.148765:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827988.276853:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827989.172785:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787827989.236815:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827989.236824:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827989.236829:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827989.236839:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4134 00000100:00080000:3.0:1787827989.236850:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827989.236857:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787827989.236900:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f830c0 x1874669228280832/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827989.236924:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827989.236927:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827989.236928:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827989.236931:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4134 00000100:00080000:3.0:1787827989.236933:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827989.236936:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827989.236944:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827989.236953:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80b40 x1874669228280960/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827989.236961:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827989.236965:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827989.236966:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827989.236971:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4134 00000100:00100000:2.0:1787827989.236972:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228280832:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787827989.236974:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827989.236980:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827989.236985:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827989.236992:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80000 x1874669228281088/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787827989.237039:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827989.237072:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827989/real 1787827989] req@ffff985610f830c0 x1874669228280832/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828014 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827989.237086:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f830c0 x1874669228280832/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828014 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827989.237095:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827989.237098:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827989.237101:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228280832:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827989.237137:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228280960:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827989.237155:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827989.237164:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827989/real 1787827989] req@ffff985610f80b40 x1874669228280960/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828014 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827989.237172:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80b40 x1874669228280960/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828014 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827989.237178:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827989.237180:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827989.237182:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228280960:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827989.237187:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228281088:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827989.237197:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827989.237203:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827989/real 1787827989] req@ffff985610f80000 x1874669228281088/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828014 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827989.237211:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80000 x1874669228281088/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828014 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827989.237216:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827989.237217:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827989.237219:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228281088:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787827989.300806:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827990.196817:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827990.324850:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827991.220893:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827991.348846:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827992.244805:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827992.372938:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827993.268822:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827993.396899:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827994.292799:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787827994.356777:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827994.356789:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827994.356794:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827994.356807:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4139 00000100:00080000:3.0:1787827994.356840:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827994.356854:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787827994.356917:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83c00 x1874669228281216/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827994.356973:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827994.356979:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827994.356982:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827994.356987:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4139 00000100:00080000:3.0:1787827994.356993:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827994.356999:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827994.357013:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827994.357031:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80780 x1874669228281344/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827994.357045:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827994.357049:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827994.357052:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827994.357056:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4139 00000100:00080000:3.0:1787827994.357060:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827994.357067:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787827994.357074:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827994.357143:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228281216:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787827994.357144:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80000 x1874669228281472/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787827994.357321:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827994.357389:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827994/real 1787827994] req@ffff985610f83c00 x1874669228281216/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828019 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827994.357422:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83c00 x1874669228281216/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828019 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827994.357443:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827994.357452:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827994.357461:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228281216:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827994.357491:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228281344:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827994.357537:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827994.357573:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827994/real 1787827994] req@ffff985610f80780 x1874669228281344/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828019 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827994.357593:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80780 x1874669228281344/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828019 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827994.357605:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827994.357611:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827994.357617:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228281344:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827994.357658:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228281472:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827994.357690:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827994.357743:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827994/real 1787827994] req@ffff985610f80000 x1874669228281472/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828019 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827994.357758:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80000 x1874669228281472/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828019 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827994.357767:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827994.357771:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827994.357774:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228281472:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787827994.420877:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827995.316872:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827995.444842:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827996.340868:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827996.468868:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827997.364817:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787827997.492863:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827998.388899:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787827998.516920:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827999.412856:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787827999.476873:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827999.476886:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827999.476893:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827999.476910:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4145 00000100:00080000:1.0:1787827999.476943:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827999.476955:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787827999.477024:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a1680 x1874669228281600/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827999.477121:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827999.477127:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827999.477131:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827999.477138:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4145 00000100:00080000:1.0:1787827999.477145:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827999.477163:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787827999.477179:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827999.477205:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a03c0 x1874669228281728/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827999.477214:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228281600:192.168.203.114@tcp:250:kworker.0 00000100:00080000:1.0:1787827999.477243:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827999.477249:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827999.477253:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827999.477262:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4145 00000100:00080000:1.0:1787827999.477272:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827999.477280:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787827999.477290:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000800:00000100:2.0:1787827999.477311:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787827999.477318:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0780 x1874669228281856/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827999.477399:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827999/real 1787827999] req@ffff9856085a1680 x1874669228281600/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828024 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827999.477427:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a1680 x1874669228281600/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828024 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827999.477442:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827999.477448:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827999.477453:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228281600:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787827999.477517:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228281728:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827999.477558:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827999.477583:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827999/real 1787827999] req@ffff9856085a03c0 x1874669228281728/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828024 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827999.477606:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a03c0 x1874669228281728/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828024 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827999.477620:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827999.477624:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827999.477629:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228281728:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787827999.477636:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228281856:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787827999.477663:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787827999.477677:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827999/real 1787827999] req@ffff9856085a0780 x1874669228281856/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828024 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827999.477691:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0780 x1874669228281856/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828024 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827999.477700:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827999.477703:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787827999.477707:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228281856:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787827999.540874:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828000.436847:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828000.564840:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828001.460762:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828001.588814:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828002.484852:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828002.612921:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828003.508843:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787828003.636910:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828004.532798:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787828004.596878:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828004.596896:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828004.596903:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828004.596922:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4150 00000100:00080000:3.0:1787828004.596940:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828004.596951:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828004.597045:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80b40 x1874669228281984/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828004.597084:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828004.597089:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828004.597092:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828004.597097:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4150 00000100:00080000:3.0:1787828004.597104:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828004.597109:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828004.597125:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828004.597151:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f830c0 x1874669228282112/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828004.597155:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228281984:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787828004.597187:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828004.597193:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828004.597196:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828004.597204:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4150 00000100:00080000:3.0:1787828004.597208:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828004.597213:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828004.597219:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828004.597245:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f812c0 x1874669228282240/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828004.597263:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828004.597312:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828004/real 1787828004] req@ffff985610f80b40 x1874669228281984/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828029 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828004.597329:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80b40 x1874669228281984/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828029 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828004.597338:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828004.597342:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828004.597345:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228281984:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787828004.597371:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228282112:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828004.597389:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828004.597398:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828004/real 1787828004] req@ffff985610f830c0 x1874669228282112/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828029 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828004.597407:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f830c0 x1874669228282112/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828029 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828004.597414:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828004.597416:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828004.597418:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228282112:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787828004.597422:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228282240:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828004.597434:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828004.597439:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828004/real 1787828004] req@ffff985610f812c0 x1874669228282240/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828029 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828004.597455:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f812c0 x1874669228282240/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828029 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828004.597461:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828004.597464:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828004.597466:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228282240:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787828004.660809:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828005.556885:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828005.684852:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828006.580829:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828006.708851:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828007.604804:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828007.732877:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828008.628809:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787828008.756886:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828009.652828:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787828009.716778:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828009.716791:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828009.716797:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828009.716813:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4155 00000100:00080000:3.0:1787828009.716829:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828009.716841:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828009.716892:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f803c0 x1874669228282368/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828009.716924:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828009.716929:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828009.716931:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828009.716935:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4155 00000100:00080000:3.0:1787828009.716939:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828009.716943:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828009.716955:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828009.716986:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81a40 x1874669228282496/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828009.717001:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1787828009.717009:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228282368:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787828009.717009:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828009.717014:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828009.717025:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4155 00000100:00080000:3.0:1787828009.717029:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828009.717035:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828009.717040:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828009.717057:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83840 x1874669228282624/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828009.717155:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228282496:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787828009.717195:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228282624:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787828009.780870:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828010.676859:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828010.804828:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828011.700766:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828011.828855:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828012.724871:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000400:00000100:3.0:1787828012.789165:0:2371:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.114:988 00000400:00000100:3.0:1787828012.789246:0:2371:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.114@tcp at host 192.168.203.114 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:3.0:1787828012.789266:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:3.0:1787828012.789325:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:3.0:1787828012.789343:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000100:00100000:2.0:1787828012.789437:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828009/real 1787828012] req@ffff985610f81a40 x1874669228282496/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828034 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828012.789499:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81a40 x1874669228282496/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828034 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828012.789521:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828012.789530:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828012.789554:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228282496:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787828012.789568:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828009/real 1787828012] req@ffff985610f83840 x1874669228282624/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828034 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828012.789590:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83840 x1874669228282624/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828034 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828012.789605:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828012.789611:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828012.789617:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228282624:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787828012.789629:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828009/real 1787828012] req@ffff985610f803c0 x1874669228282368/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828034 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828012.789650:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f803c0 x1874669228282368/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828034 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828012.789665:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828012.789670:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828012.789677:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228282368:192.168.203.114@tcp:250:kworker.0 00010000:00010000:0.0:1787828012.852835:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828013.748866:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787828013.876927:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828014.772823:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787828014.837007:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828014.837022:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828014.837028:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828014.837043:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4160 00000100:00080000:3.0:1787828014.837059:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828014.837069:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828014.837142:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81e00 x1874669228282752/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828014.837201:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828014.837209:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828014.837213:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828014.837221:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4160 00000100:00080000:3.0:1787828014.837229:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828014.837239:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828014.837258:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828014.837291:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228282752:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787828014.837309:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81680 x1874669228282880/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828014.837336:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828014.837343:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828014.837348:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828014.837357:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4160 00000100:00080000:3.0:1787828014.837407:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828014.837415:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000800:00000100:2.0:1787828014.837423:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000008:00100000:3.0:1787828014.837429:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828014.837455:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80f00 x1874669228283008/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828014.837481:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828014/real 1787828014] req@ffff985610f81e00 x1874669228282752/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828039 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828014.837512:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81e00 x1874669228282752/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828039 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828014.837530:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828014.837538:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828014.837545:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228282752:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787828014.837603:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228282880:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828014.837650:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828014.837676:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828014/real 1787828014] req@ffff985610f81680 x1874669228282880/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828039 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828014.837746:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81680 x1874669228282880/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828039 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828014.837763:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828014.837771:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828014.837778:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228282880:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787828014.837790:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228283008:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828014.837832:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828014.837861:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828014/real 1787828014] req@ffff985610f80f00 x1874669228283008/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828039 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828014.837885:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80f00 x1874669228283008/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828039 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828014.837899:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828014.837905:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828014.837910:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228283008:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787828014.900866:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828015.796848:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828015.924855:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828016.820777:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828016.948796:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828017.844821:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828017.972827:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828018.868862:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787828018.996873:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828019.892771:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787828019.956820:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828019.956830:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828019.956833:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828019.956843:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4165 00000100:00080000:1.0:1787828019.956865:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828019.956874:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787828019.956926:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3c00 x1874669228283136/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828019.956965:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828019.956968:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828019.956970:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828019.956973:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4165 00000100:00080000:1.0:1787828019.956976:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828019.956979:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787828019.956987:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828019.957007:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0b40 x1874669228283264/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828019.957015:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828019.957018:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828019.957019:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828019.957022:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4165 00000100:00080000:1.0:1787828019.957024:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828019.957029:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787828019.957029:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228283136:192.168.203.114@tcp:250:kworker.0 00000008:00100000:1.0:1787828019.957032:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828019.957043:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0000 x1874669228283392/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828019.957104:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828019.957137:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828019/real 1787828019] req@ffff9856085a3c00 x1874669228283136/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828044 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828019.957150:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3c00 x1874669228283136/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828044 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828019.957159:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828019.957171:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828019.957175:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228283136:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787828019.957181:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228283264:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828019.957198:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828019.957208:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828019/real 1787828019] req@ffff9856085a0b40 x1874669228283264/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828044 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828019.957225:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0b40 x1874669228283264/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828044 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828019.957231:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828019.957233:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828019.957236:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228283264:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787828019.957259:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228283392:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828019.957270:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828019.957277:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828019/real 1787828019] req@ffff9856085a0000 x1874669228283392/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828044 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828019.957284:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0000 x1874669228283392/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828044 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828019.957290:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828019.957292:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828019.957294:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228283392:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787828020.020793:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828020.916778:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828021.044858:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828021.940796:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828022.068832:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828022.964912:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828023.092927:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828023.988790:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787828024.116867:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828025.012771:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787828025.076966:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828025.076977:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828025.076983:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828025.076995:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4170 00000100:00080000:3.0:1787828025.077006:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828025.077014:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828025.077069:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82d00 x1874669228283520/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828025.077109:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828025.077115:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828025.077118:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828025.077124:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4170 00000100:00080000:3.0:1787828025.077130:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828025.077136:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828025.077149:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828025.077166:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82940 x1874669228283648/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828025.077180:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228283520:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787828025.077184:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828025.077192:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828025.077194:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828025.077199:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4170 00000100:00080000:3.0:1787828025.077203:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828025.077207:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828025.077212:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828025.077225:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82580 x1874669228283776/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828025.077280:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828025.077329:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828025/real 1787828025] req@ffff985610f82d00 x1874669228283520/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828050 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828025.077357:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82d00 x1874669228283520/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828050 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828025.077386:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828025.077389:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828025.077393:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228283520:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787828025.077416:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228283648:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828025.077437:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828025.077447:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828025/real 1787828025] req@ffff985610f82940 x1874669228283648/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828050 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828025.077456:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82940 x1874669228283648/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828050 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828025.077463:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828025.077465:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828025.077468:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228283648:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787828025.077472:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228283776:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828025.077483:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828025.077490:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828025/real 1787828025] req@ffff985610f82580 x1874669228283776/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828050 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828025.077499:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82580 x1874669228283776/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828050 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828025.077505:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828025.077507:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828025.077510:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228283776:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787828025.140855:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828026.036742:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828026.164808:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828027.060754:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828027.188938:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828028.084826:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828028.212991:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828029.108825:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787828029.237052:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828030.132783:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787828030.196808:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828030.196818:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828030.196823:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828030.196850:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4175 00000100:00080000:3.0:1787828030.196862:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828030.196870:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828030.196944:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f821c0 x1874669228283904/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828030.197000:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828030.197004:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828030.197006:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828030.197010:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4175 00000100:00080000:3.0:1787828030.197013:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828030.197017:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828030.197027:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828030.197047:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83480 x1874669228284032/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828030.197059:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1787828030.197062:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f821c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228283904:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787828030.197063:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828030.197068:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828030.197075:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4175 00000100:00080000:3.0:1787828030.197078:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828030.197082:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828030.197086:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828030.197098:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82580 x1874669228284160/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828030.197151:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828030.197221:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828030/real 1787828030] req@ffff985610f821c0 x1874669228283904/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828055 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828030.197246:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f821c0 x1874669228283904/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828055 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828030.197261:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828030.197267:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828030.197272:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f821c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228283904:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787828030.197303:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228284032:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828030.197333:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828030.197347:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828030/real 1787828030] req@ffff985610f83480 x1874669228284032/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828055 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828030.197363:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83480 x1874669228284032/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828055 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828030.197382:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828030.197387:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828030.197391:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228284032:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787828030.197398:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228284160:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828030.197417:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828030.197428:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828030/real 1787828030] req@ffff985610f82580 x1874669228284160/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828055 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828030.197442:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82580 x1874669228284160/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828055 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828030.197451:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828030.197455:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828030.197458:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228284160:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787828030.260816:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828031.156850:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828031.284895:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828032.180781:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828032.308894:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828033.204797:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828033.332962:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828034.228771:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787828034.356860:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828035.252838:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787828035.316836:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828035.316851:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828035.316859:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828035.316876:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4180 00000100:00080000:3.0:1787828035.316905:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828035.316918:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828035.316973:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82940 x1874669228284288/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828035.317012:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828035.317017:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828035.317020:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828035.317025:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4180 00000100:00080000:3.0:1787828035.317030:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828035.317035:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828035.317048:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828035.317065:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82d00 x1874669228284416/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828035.317083:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828035.317088:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828035.317092:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:2.0:1787828035.317105:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228284288:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787828035.317106:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4180 00000100:00080000:3.0:1787828035.317113:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828035.317124:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828035.317132:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828035.317146:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80f00 x1874669228284544/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828035.317253:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828035.317328:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828035/real 1787828035] req@ffff985610f82940 x1874669228284288/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828060 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828035.317364:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82940 x1874669228284288/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828060 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828035.317380:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828035.317385:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828035.317391:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228284288:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787828035.317426:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228284416:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828035.317461:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828035.317477:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828035/real 1787828035] req@ffff985610f82d00 x1874669228284416/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828060 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828035.317492:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82d00 x1874669228284416/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828060 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828035.317503:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828035.317508:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828035.317512:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228284416:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787828035.317518:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228284544:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828035.317538:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828035.317550:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828035/real 1787828035] req@ffff985610f80f00 x1874669228284544/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828060 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828035.317563:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80f00 x1874669228284544/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828060 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828035.317573:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828035.317586:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828035.317590:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228284544:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787828035.380905:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828036.276820:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828036.404897:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828037.300852:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828037.428861:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828038.324901:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828038.452954:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828039.348844:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787828039.476921:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828040.372771:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787828040.436865:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828040.436893:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828040.436898:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828040.436911:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4186 00000100:00080000:3.0:1787828040.436925:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828040.436933:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828040.437007:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81680 x1874669228284672/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828040.437066:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828040.437071:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828040.437074:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828040.437078:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4186 00000100:00080000:3.0:1787828040.437082:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828040.437087:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828040.437114:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828040.437133:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81e00 x1874669228284800/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828040.437148:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828040.437152:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828040.437155:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828040.437161:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4186 00000100:00080000:3.0:1787828040.437164:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00100000:2.0:1787828040.437173:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228284672:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787828040.437173:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828040.437183:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828040.437196:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f803c0 x1874669228284928/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828040.437272:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828040.437316:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828040/real 1787828040] req@ffff985610f81680 x1874669228284672/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828065 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828040.437333:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81680 x1874669228284672/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828065 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828040.437344:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828040.437349:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828040.437353:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228284672:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787828040.437378:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228284800:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828040.437408:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828040.437418:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828040/real 1787828040] req@ffff985610f81e00 x1874669228284800/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828065 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828040.437428:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81e00 x1874669228284800/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828065 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828040.437435:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828040.437438:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828040.437440:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228284800:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787828040.437445:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228284928:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828040.437458:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828040.437465:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828040/real 1787828040] req@ffff985610f803c0 x1874669228284928/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828065 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828040.437474:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f803c0 x1874669228284928/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828065 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828040.437486:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828040.437489:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828040.437492:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228284928:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787828040.500960:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828041.396822:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828041.524863:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828042.420866:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828042.548939:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828043.444839:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828043.572860:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828044.468764:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787828044.596927:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828045.492758:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787828045.556853:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828045.556866:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828045.556872:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828045.556886:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4191 00000100:00080000:2.0:1787828045.556900:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828045.556911:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787828045.556962:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140da940 x1874669228285056/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828045.557008:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828045.557014:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828045.557017:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828045.557023:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4191 00000100:00080000:2.0:1787828045.557028:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828045.557034:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787828045.557047:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828045.557064:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228285056:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787828045.557066:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8b40 x1874669228285184/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828045.557099:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828045.557105:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828045.557110:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828045.557118:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4191 00000100:00080000:2.0:1787828045.557122:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828045.557127:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787828045.557134:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000800:00000100:1.0:1787828045.557154:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828045.557161:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d9680 x1874669228285312/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787828045.557208:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828045/real 1787828045] req@ffff9856140da940 x1874669228285056/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828070 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828045.557231:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140da940 x1874669228285056/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828070 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828045.557240:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828045.557244:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828045.557247:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228285056:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787828045.557272:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228285184:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828045.557288:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828045.557296:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828045/real 1787828045] req@ffff9856140d8b40 x1874669228285184/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828070 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828045.557304:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8b40 x1874669228285184/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828070 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828045.557309:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828045.557311:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828045.557313:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228285184:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787828045.557317:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228285312:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828045.557328:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828045.557334:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828045/real 1787828045] req@ffff9856140d9680 x1874669228285312/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828070 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828045.557342:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d9680 x1874669228285312/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828070 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828045.557346:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828045.557348:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828045.557350:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228285312:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787828045.620784:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828046.516823:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828046.644939:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828047.540823:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828047.668884:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828048.564779:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828048.692877:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828049.588800:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787828049.716960:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828050.612836:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787828050.676886:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828050.676901:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828050.676908:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828050.676935:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4196 00000100:00080000:2.0:1787828050.676949:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828050.676961:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787828050.677025:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d83c0 x1874669228285440/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828050.677074:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828050.677079:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828050.677082:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828050.677087:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4196 00000100:00080000:2.0:1787828050.677091:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828050.677096:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787828050.677108:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828050.677127:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d9a40 x1874669228285568/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828050.677152:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1787828050.677155:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228285440:192.168.203.114@tcp:250:kworker.0 00000100:00080000:2.0:1787828050.677159:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828050.677166:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828050.677175:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4196 00000100:00080000:2.0:1787828050.677181:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828050.677187:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787828050.677195:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828050.677215:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db480 x1874669228285696/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828050.677260:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828050.677306:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828050/real 1787828050] req@ffff9856140d83c0 x1874669228285440/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828075 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828050.677330:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d83c0 x1874669228285440/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828075 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828050.677453:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828050.677461:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828050.677467:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228285440:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787828050.677501:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228285568:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787828050.677911:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828050.678053:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828050/real 1787828050] req@ffff9856140d9a40 x1874669228285568/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828075 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828050.678072:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d9a40 x1874669228285568/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828075 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828050.678085:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828050.678089:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828050.678093:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228285568:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787828050.678102:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228285696:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787828050.678131:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828050.678145:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828050/real 1787828050] req@ffff9856140db480 x1874669228285696/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828075 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828050.678160:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db480 x1874669228285696/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828075 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828050.678170:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828050.678174:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828050.678178:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228285696:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787828050.740867:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828051.636834:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828051.764783:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828052.660777:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828052.788830:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828053.684796:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828053.812840:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828054.708846:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787828054.836964:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828055.732870:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787828055.796855:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828055.796867:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828055.796871:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828055.796884:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4201 00000100:00080000:2.0:1787828055.796917:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828055.796928:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787828055.796981:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8780 x1874669228285824/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828055.797019:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828055.797024:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828055.797027:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828055.797032:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4201 00000100:00080000:2.0:1787828055.797037:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828055.797042:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787828055.797058:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828055.797077:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db480 x1874669228285952/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828055.797111:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828055.797115:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828055.797117:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1787828055.797126:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228285824:192.168.203.114@tcp:250:kworker.0 00000100:00080000:2.0:1787828055.797131:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4201 00000100:00080000:2.0:1787828055.797145:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828055.797150:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787828055.797156:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828055.797210:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d9a40 x1874669228286080/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828055.797269:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828055.797338:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828055/real 1787828055] req@ffff9856140d8780 x1874669228285824/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828080 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828055.797367:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8780 x1874669228285824/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828080 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828055.797384:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828055.797389:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828055.797395:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228285824:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787828055.797430:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228285952:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787828055.797492:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828055.797510:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828055/real 1787828055] req@ffff9856140db480 x1874669228285952/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828080 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828055.797525:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db480 x1874669228285952/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828080 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828055.797543:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828055.797547:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828055.797551:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228285952:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787828055.797557:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228286080:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787828055.797601:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828055.797613:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828055/real 1787828055] req@ffff9856140d9a40 x1874669228286080/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828080 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828055.797627:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d9a40 x1874669228286080/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828080 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828055.797638:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828055.797641:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828055.797645:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228286080:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787828055.860908:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828056.756832:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828056.884847:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828057.780808:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828057.908890:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828058.804774:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828058.932909:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828059.828828:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787828059.956946:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828060.852798:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787828060.916828:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828060.916837:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828060.916841:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828060.916852:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4206 00000100:00080000:0.0:1787828060.916862:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828060.916870:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787828060.916909:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6940 x1874669228286208/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828060.916935:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828060.916938:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828060.916939:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828060.916943:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4206 00000100:00080000:0.0:1787828060.916946:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828060.916949:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828060.916957:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828060.916966:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4780 x1874669228286336/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828060.916974:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828060.916978:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828060.916982:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1787828060.916982:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228286208:192.168.203.114@tcp:250:kworker.0 00000100:00080000:0.0:1787828060.916991:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4206 00000100:00080000:0.0:1787828060.916993:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828060.916995:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828060.916998:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828060.917006:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4b40 x1874669228286464/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787828060.917045:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828060.917078:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828060/real 1787828060] req@ffff985607df6940 x1874669228286208/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828085 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828060.917091:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6940 x1874669228286208/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828085 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828060.917099:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828060.917102:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828060.917105:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228286208:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787828060.917124:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228286336:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828060.917139:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828060.917147:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828060/real 1787828060] req@ffff985607df4780 x1874669228286336/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828085 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828060.917156:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4780 x1874669228286336/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828085 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828060.917166:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828060.917168:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828060.917170:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228286336:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787828060.917174:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228286464:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828060.917184:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828060.917190:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828060/real 1787828060] req@ffff985607df4b40 x1874669228286464/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828085 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828060.917197:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4b40 x1874669228286464/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828085 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828060.917203:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828060.917205:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828060.917207:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228286464:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787828060.980823:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828061.876848:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828062.004919:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828062.900800:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828063.028839:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828063.924816:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828064.052885:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828064.948831:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787828065.076893:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828065.972885:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787828066.036818:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828066.036832:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828066.036838:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828066.036853:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4211 00000100:00080000:0.0:1787828066.036869:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828066.036879:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787828066.036935:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4000 x1874669228286592/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828066.036978:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828066.036983:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828066.036985:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828066.036990:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4211 00000100:00080000:0.0:1787828066.036995:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828066.036999:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828066.037014:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828066.037031:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7c00 x1874669228286720/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787828066.037058:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228286592:192.168.203.114@tcp:250:kworker.0 00000100:00080000:0.0:1787828066.037059:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828066.037075:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828066.037080:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828066.037086:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4211 00000100:00080000:0.0:1787828066.037090:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828066.037094:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828066.037101:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828066.037120:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4b40 x1874669228286848/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787828066.037163:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828066.037212:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828066/real 1787828066] req@ffff985607df4000 x1874669228286592/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828091 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828066.037237:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4000 x1874669228286592/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828091 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828066.037252:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828066.037257:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828066.037263:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228286592:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787828066.037294:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228286720:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828066.037320:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828066.037335:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828066/real 1787828066] req@ffff985607df7c00 x1874669228286720/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828091 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828066.037351:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7c00 x1874669228286720/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828091 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828066.037362:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828066.037366:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828066.037370:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228286720:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787828066.037377:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228286848:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828066.037395:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828066.037406:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828066/real 1787828066] req@ffff985607df4b40 x1874669228286848/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828091 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828066.037420:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4b40 x1874669228286848/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828091 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828066.037430:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828066.037433:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828066.037437:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228286848:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787828066.100895:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828066.996846:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828067.124914:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828068.020852:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828068.148946:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828069.044769:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828069.172841:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828070.068831:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787828070.196945:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828071.092839:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787828071.156855:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828071.156870:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828071.156876:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828071.156889:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4216 00000100:00080000:3.0:1787828071.156918:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828071.156928:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828071.156979:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83840 x1874669228286976/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828071.157031:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828071.157037:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828071.157039:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828071.157063:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4216 00000100:00080000:3.0:1787828071.157067:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828071.157072:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828071.157084:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828071.157102:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81a40 x1874669228287104/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828071.157116:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:1.0:1787828071.157126:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228286976:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787828071.157134:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828071.157140:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828071.157146:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4216 00000100:00080000:3.0:1787828071.157151:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828071.157155:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828071.157161:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828071.157175:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f812c0 x1874669228287232/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787828071.157242:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828071.157289:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828071/real 1787828071] req@ffff985610f83840 x1874669228286976/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828096 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828071.157311:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83840 x1874669228286976/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828096 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828071.157323:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828071.157328:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828071.157333:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228286976:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787828071.157364:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228287104:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828071.157402:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828071.157417:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828071/real 1787828071] req@ffff985610f81a40 x1874669228287104/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828096 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828071.157432:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81a40 x1874669228287104/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828096 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828071.157442:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828071.157445:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828071.157449:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228287104:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787828071.157457:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228287232:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828071.157477:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828071.157487:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828071/real 1787828071] req@ffff985610f812c0 x1874669228287232/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828096 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828071.157501:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f812c0 x1874669228287232/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828096 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828071.157519:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828071.157522:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828071.157525:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228287232:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787828071.220914:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828072.116777:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828072.244850:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828073.140828:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828073.268876:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828074.164808:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828074.292807:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828075.188775:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787828075.316845:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828076.212790:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787828076.276805:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828076.276826:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828076.276831:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828076.276846:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4221 00000100:00080000:3.0:1787828076.276861:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828076.276871:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828076.276921:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f830c0 x1874669228287360/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828076.276962:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828076.276969:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828076.276973:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828076.276981:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4221 00000100:00080000:3.0:1787828076.276988:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828076.277000:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828076.277018:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828076.277046:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228287360:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787828076.277050:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80b40 x1874669228287488/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828076.277075:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828076.277080:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828076.277083:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828076.277088:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4221 00000100:00080000:3.0:1787828076.277094:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828076.277101:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828076.277109:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828076.277127:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80000 x1874669228287616/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787828076.277203:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228287488:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787828076.277237:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228287616:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787828076.340916:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828077.236806:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828077.364945:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828078.260803:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828078.388847:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828079.284868:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000400:00000100:1.0:1787828079.349360:0:2372:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.114:988 00000400:00000100:1.0:1787828079.349474:0:2372:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.114@tcp at host 192.168.203.114 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1787828079.349537:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:1.0:1787828079.349607:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:1.0:1787828079.349620:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000100:00100000:0.0:1787828079.349684:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828076/real 1787828079] req@ffff985610f80b40 x1874669228287488/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828101 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828079.349827:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80b40 x1874669228287488/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828101 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828079.349852:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828079.349864:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828079.349873:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228287488:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787828079.349888:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828076/real 1787828079] req@ffff985610f80000 x1874669228287616/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828101 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828079.349915:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80000 x1874669228287616/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828101 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828079.349932:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828079.349938:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828079.349945:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228287616:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787828079.349958:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828076/real 1787828079] req@ffff985610f830c0 x1874669228287360/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828101 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828079.349982:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f830c0 x1874669228287360/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828101 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828079.349998:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828079.350005:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828079.350013:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228287360:192.168.203.114@tcp:250:kworker.0 00010000:00010000:0.0:1787828079.412878:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828080.308859:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787828080.436912:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828081.332802:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787828081.396902:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828081.396915:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828081.396920:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828081.396937:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4227 00000100:00080000:0.0:1787828081.396953:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828081.396965:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787828081.397014:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4780 x1874669228287744/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828081.397057:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828081.397062:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828081.397065:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828081.397070:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4227 00000100:00080000:0.0:1787828081.397075:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828081.397081:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828081.397095:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828081.397122:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6940 x1874669228287872/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787828081.397141:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228287744:192.168.203.114@tcp:250:kworker.0 00000100:00080000:0.0:1787828081.397154:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828081.397163:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828081.397167:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828081.397177:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4227 00000100:00080000:0.0:1787828081.397187:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828081.397197:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828081.397211:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828081.397237:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7480 x1874669228288000/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787828081.397267:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828081.397365:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828081/real 1787828081] req@ffff985607df4780 x1874669228287744/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828106 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828081.397395:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4780 x1874669228287744/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828106 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828081.397416:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828081.397423:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828081.397430:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228287744:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787828081.397500:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228287872:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828081.397543:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828081.397566:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828081/real 1787828081] req@ffff985607df6940 x1874669228287872/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828106 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828081.397588:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6940 x1874669228287872/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828106 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828081.397602:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828081.397608:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828081.397614:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228287872:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787828081.397641:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228288000:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828081.397681:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828081.397766:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828081/real 1787828081] req@ffff985607df7480 x1874669228288000/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828106 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828081.397804:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7480 x1874669228288000/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828106 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828081.397817:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828081.397823:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828081.397828:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228288000:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787828081.460888:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828082.356759:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828082.484839:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828083.380783:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828083.508848:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828084.404815:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828084.532863:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828085.428759:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787828085.556834:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828086.452797:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787828086.516928:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828086.516950:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828086.516959:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828086.516978:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4232 00000100:00080000:3.0:1787828086.516995:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828086.517006:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828086.517062:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80780 x1874669228288128/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828086.517107:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828086.517113:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828086.517117:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828086.517122:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4232 00000100:00080000:3.0:1787828086.517129:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828086.517135:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828086.517150:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828086.517169:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228288128:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787828086.517177:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83c00 x1874669228288256/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828086.517200:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828086.517208:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828086.517213:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828086.517223:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4232 00000100:00080000:3.0:1787828086.517235:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828086.517245:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828086.517256:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000800:00000100:0.0:1787828086.517275:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828086.517288:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f830c0 x1874669228288384/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787828086.517331:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828086/real 1787828086] req@ffff985610f80780 x1874669228288128/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828111 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828086.517358:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80780 x1874669228288128/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828111 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828086.517374:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828086.517381:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828086.517387:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228288128:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787828086.517473:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228288256:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787828086.517517:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828086.517535:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828086/real 1787828086] req@ffff985610f83c00 x1874669228288256/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828111 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828086.517550:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83c00 x1874669228288256/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828111 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828086.517560:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828086.517564:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828086.517568:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228288256:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787828086.517575:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228288384:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787828086.517597:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828086.517610:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828086/real 1787828086] req@ffff985610f830c0 x1874669228288384/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828111 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828086.517625:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f830c0 x1874669228288384/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828111 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828086.517636:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828086.517640:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828086.517644:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228288384:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787828086.580872:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828087.476826:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828087.604867:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828088.500808:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828088.628900:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828089.524755:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828089.652915:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828090.548799:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787828090.676988:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828091.572830:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787828091.636863:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828091.636894:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828091.636902:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828091.636919:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4237 00000100:00080000:0.0:1787828091.636936:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828091.636947:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787828091.637015:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df52c0 x1874669228288512/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828091.637068:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828091.637073:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828091.637076:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828091.637081:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4237 00000100:00080000:0.0:1787828091.637086:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828091.637093:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828091.637108:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828091.637129:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7840 x1874669228288640/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787828091.637140:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228288512:192.168.203.114@tcp:250:kworker.0 00000100:00080000:0.0:1787828091.637150:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828091.637157:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828091.637160:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828091.637166:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4237 00000100:00080000:0.0:1787828091.637169:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828091.637174:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828091.637180:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828091.637196:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df61c0 x1874669228288768/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787828091.637257:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828091.637320:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828091/real 1787828091] req@ffff985607df52c0 x1874669228288512/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828116 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828091.637346:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df52c0 x1874669228288512/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828116 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828091.637361:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828091.637367:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828091.637372:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228288512:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787828091.637431:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228288640:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828091.637474:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828091.637491:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828091/real 1787828091] req@ffff985607df7840 x1874669228288640/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828116 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828091.637506:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7840 x1874669228288640/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828116 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828091.637518:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828091.637522:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828091.637526:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228288640:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787828091.637533:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228288768:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828091.637554:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828091.637573:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828091/real 1787828091] req@ffff985607df61c0 x1874669228288768/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828116 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828091.637588:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df61c0 x1874669228288768/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828116 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828091.637606:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828091.637610:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828091.637614:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228288768:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787828091.700864:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828092.596836:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828092.724930:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828093.620859:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828093.748878:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828094.644785:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828094.772856:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828095.668819:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787828095.796870:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828096.692839:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787828096.756826:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828096.756842:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828096.756849:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828096.756868:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4242 00000100:00080000:0.0:1787828096.756887:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828096.756902:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787828096.756964:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6d00 x1874669228288896/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828096.757029:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828096.757037:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828096.757042:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828096.757050:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4242 00000100:00080000:0.0:1787828096.757058:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828096.757067:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828096.757098:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828096.757126:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4f00 x1874669228289024/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828096.757153:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:1.0:1787828096.757166:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228288896:192.168.203.114@tcp:250:kworker.0 00000100:00080000:0.0:1787828096.757168:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828096.757179:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828096.757195:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4242 00000100:00080000:0.0:1787828096.757205:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828096.757217:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828096.757231:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828096.757264:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5e00 x1874669228289152/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787828096.757289:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828096.757387:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828096/real 1787828096] req@ffff985607df6d00 x1874669228288896/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828121 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828096.757422:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6d00 x1874669228288896/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828121 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828096.757438:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828096.757445:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828096.757451:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228288896:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787828096.757498:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228289024:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828096.757548:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828096.757573:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828096/real 1787828096] req@ffff985607df4f00 x1874669228289024/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828121 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828096.757591:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4f00 x1874669228289024/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828121 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828096.757603:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828096.757608:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828096.757612:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228289024:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787828096.757622:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228289152:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828096.757650:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828096.757663:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828096/real 1787828096] req@ffff985607df5e00 x1874669228289152/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828121 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828096.757677:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5e00 x1874669228289152/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828121 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828096.757687:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828096.757757:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828096.757763:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228289152:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787828096.820843:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828097.716818:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828097.844938:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828098.740768:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828098.868875:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828099.764837:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828099.892853:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828100.788814:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787828100.916951:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828101.812790:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787828101.876837:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828101.876848:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828101.876853:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828101.876863:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4247 00000100:00080000:0.0:1787828101.876874:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828101.876881:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787828101.876925:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df43c0 x1874669228289280/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828101.876950:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828101.876953:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828101.876954:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828101.876957:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4247 00000100:00080000:0.0:1787828101.876959:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828101.876962:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828101.876970:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828101.876979:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5680 x1874669228289408/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828101.876986:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828101.876989:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828101.876990:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828101.876993:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4247 00000100:00080000:0.0:1787828101.876995:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828101.876997:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828101.877000:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828101.877008:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df70c0 x1874669228289536/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787828101.877031:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228289280:192.168.203.114@tcp:250:kworker.0 00000800:00000100:1.0:1787828101.877101:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828101.877142:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828101/real 1787828101] req@ffff985607df43c0 x1874669228289280/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828126 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828101.877162:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df43c0 x1874669228289280/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828126 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828101.877190:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828101.877195:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828101.877200:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228289280:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787828101.877208:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228289408:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828101.877233:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828101.877244:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828101/real 1787828101] req@ffff985607df5680 x1874669228289408/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828126 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828101.877254:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5680 x1874669228289408/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828126 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828101.877261:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828101.877263:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828101.877265:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228289408:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787828101.877287:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228289536:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828101.877300:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828101.877307:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828101/real 1787828101] req@ffff985607df70c0 x1874669228289536/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828126 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828101.877315:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df70c0 x1874669228289536/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828126 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828101.877321:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828101.877322:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828101.877324:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228289536:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787828101.940849:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828102.836829:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828102.964855:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828103.860827:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828103.988876:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828104.884828:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828105.012883:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828105.908804:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787828106.036922:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828106.932847:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787828106.996845:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828106.996863:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828106.996870:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828106.996886:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4252 00000100:00080000:3.0:1787828106.996902:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828106.996914:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828106.996975:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80000 x1874669228289664/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828106.997019:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828106.997025:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828106.997028:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828106.997034:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4252 00000100:00080000:3.0:1787828106.997040:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828106.997045:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828106.997059:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828106.997081:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80b40 x1874669228289792/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787828106.997100:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228289664:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787828106.997103:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828106.997115:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828106.997118:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828106.997125:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4252 00000100:00080000:3.0:1787828106.997132:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828106.997139:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828106.997148:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828106.997186:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f812c0 x1874669228289920/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787828106.997210:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828106.997264:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828106/real 1787828106] req@ffff985610f80000 x1874669228289664/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828131 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828106.997288:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80000 x1874669228289664/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828131 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828106.997367:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828106.997373:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828106.997377:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228289664:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787828106.997408:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228289792:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828106.997436:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828106.997450:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828106/real 1787828106] req@ffff985610f80b40 x1874669228289792/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828131 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828106.997462:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80b40 x1874669228289792/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828131 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828106.997471:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828106.997475:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828106.997478:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228289792:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787828106.997483:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228289920:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828106.997499:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828106.997508:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828106/real 1787828106] req@ffff985610f812c0 x1874669228289920/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828131 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828106.997519:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f812c0 x1874669228289920/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828131 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828106.997527:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828106.997530:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828106.997533:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228289920:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787828107.060865:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828107.956836:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828108.084879:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828108.980817:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828109.108925:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828110.004759:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828110.132864:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828111.028799:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787828111.156912:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828112.052835:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787828112.116811:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828112.116821:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828112.116826:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828112.116840:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4257 00000100:00080000:3.0:1787828112.116857:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828112.116867:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828112.116922:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81a40 x1874669228290048/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828112.116963:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828112.116968:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828112.116971:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828112.116976:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4257 00000100:00080000:3.0:1787828112.116980:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828112.116985:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828112.117000:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828112.117016:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83840 x1874669228290176/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828112.117031:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828112.117036:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828112.117038:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828112.117044:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4257 00000100:00080000:3.0:1787828112.117047:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828112.117052:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828112.117061:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828112.117074:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f803c0 x1874669228290304/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787828112.117092:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228290048:192.168.203.114@tcp:250:kworker.0 00000800:00000100:1.0:1787828112.117217:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828112.117305:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828112/real 1787828112] req@ffff985610f81a40 x1874669228290048/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828137 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828112.117344:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81a40 x1874669228290048/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828137 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828112.117361:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828112.117367:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828112.117373:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228290048:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787828112.117384:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228290176:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828112.117420:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828112.117444:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828112/real 1787828112] req@ffff985610f83840 x1874669228290176/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828137 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828112.117468:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83840 x1874669228290176/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828137 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828112.117486:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828112.117493:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828112.117500:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228290176:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787828112.117545:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228290304:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828112.117576:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828112.117591:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828112/real 1787828112] req@ffff985610f803c0 x1874669228290304/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828137 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828112.117607:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f803c0 x1874669228290304/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828137 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828112.117618:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828112.117622:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828112.117626:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228290304:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787828112.180859:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828113.076786:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828113.204832:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828114.100775:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828114.228860:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828115.124775:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828115.252792:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828116.148786:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787828116.276958:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828117.172829:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787828117.236880:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828117.236893:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828117.236900:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828117.236914:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4262 00000100:00080000:3.0:1787828117.236929:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828117.236939:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828117.236999:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81e00 x1874669228290432/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828117.237042:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828117.237049:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828117.237053:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828117.237062:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4262 00000100:00080000:3.0:1787828117.237068:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828117.237077:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828117.237095:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828117.237121:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81680 x1874669228290560/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787828117.237137:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228290432:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787828117.237142:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828117.237148:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828117.237151:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828117.237158:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4262 00000100:00080000:3.0:1787828117.237164:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828117.237171:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828117.237178:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828117.237194:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80f00 x1874669228290688/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787828117.237263:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828117.237325:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828117/real 1787828117] req@ffff985610f81e00 x1874669228290432/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828142 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828117.237349:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81e00 x1874669228290432/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828142 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828117.237363:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828117.237368:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828117.237374:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228290432:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787828117.237405:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228290560:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828117.237438:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828117.237454:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828117/real 1787828117] req@ffff985610f81680 x1874669228290560/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828142 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828117.237469:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81680 x1874669228290560/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828142 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828117.237487:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828117.237491:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828117.237496:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228290560:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787828117.237502:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228290688:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828117.237521:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828117.237533:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828117/real 1787828117] req@ffff985610f80f00 x1874669228290688/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828142 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828117.237546:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80f00 x1874669228290688/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828142 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828117.237556:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828117.237559:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828117.237563:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228290688:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787828117.300876:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828118.196790:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828118.324863:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828119.220832:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828119.348856:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828120.244856:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828120.372885:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828121.268843:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787828121.396855:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828122.292771:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787828122.356773:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828122.356782:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828122.356787:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828122.356797:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4267 00000100:00080000:3.0:1787828122.356808:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828122.356816:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828122.356856:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82d00 x1874669228290816/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828122.356883:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828122.356887:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828122.356889:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828122.356892:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4267 00000100:00080000:3.0:1787828122.356894:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828122.356897:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828122.356906:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828122.356915:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82940 x1874669228290944/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828122.356921:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828122.356924:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828122.356925:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828122.356928:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4267 00000100:00080000:3.0:1787828122.356929:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828122.356932:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828122.356934:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828122.356940:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82580 x1874669228291072/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787828122.357047:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228290816:192.168.203.114@tcp:250:kworker.0 00000800:00000100:1.0:1787828122.357170:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828122.357258:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828122/real 1787828122] req@ffff985610f82d00 x1874669228290816/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828147 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828122.357290:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82d00 x1874669228290816/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828147 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828122.357311:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828122.357319:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828122.357328:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228290816:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787828122.357344:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228290944:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828122.357394:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828122.357413:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828122/real 1787828122] req@ffff985610f82940 x1874669228290944/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828147 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828122.357431:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82940 x1874669228290944/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828147 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828122.357460:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828122.357465:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828122.357470:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228290944:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787828122.357479:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228291072:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828122.357507:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828122.357533:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828122/real 1787828122] req@ffff985610f82580 x1874669228291072/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828147 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828122.357549:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82580 x1874669228291072/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828147 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828122.357560:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828122.357569:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828122.357580:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228291072:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787828122.420823:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828123.316798:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828123.444846:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828124.340784:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828124.468836:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828125.364825:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828125.492853:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828126.388845:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787828126.516904:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828127.412851:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787828127.476840:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828127.476852:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828127.476858:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828127.476873:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4273 00000100:00080000:3.0:1787828127.476901:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828127.476915:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828127.476996:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83480 x1874669228291200/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828127.477058:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828127.477063:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828127.477066:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828127.477072:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4273 00000100:00080000:3.0:1787828127.477077:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828127.477083:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828127.477097:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828127.477117:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f821c0 x1874669228291328/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828127.477131:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828127.477136:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828127.477139:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828127.477144:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4273 00000100:00080000:3.0:1787828127.477148:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828127.477194:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787828127.477199:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228291200:192.168.203.114@tcp:250:kworker.0 00000008:00100000:3.0:1787828127.477201:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828127.477235:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82580 x1874669228291456/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787828127.477316:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828127.477366:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828127/real 1787828127] req@ffff985610f83480 x1874669228291200/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828152 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828127.477390:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83480 x1874669228291200/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828152 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828127.477405:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828127.477411:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828127.477416:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228291200:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787828127.477425:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f821c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228291328:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828127.477453:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828127.477485:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828127/real 1787828127] req@ffff985610f821c0 x1874669228291328/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828152 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828127.477511:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f821c0 x1874669228291328/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828152 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828127.477521:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828127.477525:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828127.477530:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f821c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228291328:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787828127.477560:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228291456:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828127.477589:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828127.477601:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828127/real 1787828127] req@ffff985610f82580 x1874669228291456/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828152 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828127.477615:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82580 x1874669228291456/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828152 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828127.477634:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828127.477646:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828127.477651:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228291456:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787828127.540836:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828128.436826:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828128.564873:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828129.460828:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828129.588902:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828130.484819:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828130.612837:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828131.508795:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787828131.636920:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828132.532866:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787828132.596856:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828132.596869:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828132.596875:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828132.596890:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4278 00000100:00080000:1.0:1787828132.596906:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828132.596916:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787828132.596974:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2d00 x1874669228291584/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828132.597029:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828132.597034:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828132.597037:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828132.597041:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4278 00000100:00080000:1.0:1787828132.597046:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828132.597051:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787828132.597066:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828132.597086:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0000 x1874669228291712/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828132.597102:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828132.597107:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828132.597111:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1787828132.597141:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228291584:192.168.203.114@tcp:250:kworker.0 00000100:00080000:1.0:1787828132.597150:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4278 00000100:00080000:1.0:1787828132.597163:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828132.597180:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787828132.597189:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828132.597207:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0b40 x1874669228291840/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787828132.597297:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828132.597359:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828132/real 1787828132] req@ffff9856085a2d00 x1874669228291584/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828157 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828132.597390:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2d00 x1874669228291584/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828157 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828132.597403:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828132.597408:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828132.597412:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228291584:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787828132.597444:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228291712:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787828132.597470:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828132.597482:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828132/real 1787828132] req@ffff9856085a0000 x1874669228291712/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828157 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828132.597494:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0000 x1874669228291712/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828157 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828132.597503:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828132.597507:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828132.597520:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228291712:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787828132.597527:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228291840:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787828132.597546:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828132.597557:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828132/real 1787828132] req@ffff9856085a0b40 x1874669228291840/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828157 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828132.597576:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0b40 x1874669228291840/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828157 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828132.597598:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828132.597604:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828132.597609:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228291840:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787828132.660845:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828133.556928:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828133.684924:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828134.580831:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828134.708812:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828135.604825:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828135.732900:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828136.628889:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787828136.756904:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828137.652841:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787828137.716842:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828137.716852:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828137.716857:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828137.716869:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4283 00000100:00080000:3.0:1787828137.716882:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828137.716890:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828137.716949:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82940 x1874669228291968/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828137.716981:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828137.716984:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828137.716986:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828137.716989:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4283 00000100:00080000:3.0:1787828137.716992:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828137.716995:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828137.717005:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828137.717016:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82d00 x1874669228292096/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828137.717026:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828137.717032:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:0.0:1787828137.717035:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228291968:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787828137.717035:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828137.717045:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4283 00000100:00080000:3.0:1787828137.717049:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828137.717052:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828137.717056:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828137.717068:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80f00 x1874669228292224/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787828137.717129:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828137.717178:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828137/real 1787828137] req@ffff985610f82940 x1874669228291968/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828162 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828137.717194:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82940 x1874669228291968/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828162 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828137.717205:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828137.717208:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828137.717212:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228291968:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787828137.717235:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228292096:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787828137.717253:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828137.717264:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828137/real 1787828137] req@ffff985610f82d00 x1874669228292096/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828162 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828137.717274:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82d00 x1874669228292096/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828162 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828137.717281:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828137.717283:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828137.717286:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228292096:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787828137.717290:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228292224:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787828137.717303:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828137.717311:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828137/real 1787828137] req@ffff985610f80f00 x1874669228292224/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828162 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828137.717320:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80f00 x1874669228292224/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828162 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828137.717326:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828137.717328:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828137.717330:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228292224:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787828137.780894:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828138.676834:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828138.804874:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828139.700803:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828139.828845:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828140.724829:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828140.852824:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828141.748804:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787828141.876894:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828142.772833:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787828142.836803:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828142.836814:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828142.836819:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828142.836832:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4288 00000100:00080000:3.0:1787828142.836847:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828142.836857:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828142.836907:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81680 x1874669228292352/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828142.836959:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828142.836971:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828142.836974:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828142.836980:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4288 00000100:00080000:3.0:1787828142.836986:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828142.836998:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828142.837012:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828142.837024:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228292352:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787828142.837037:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81e00 x1874669228292480/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828142.837076:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828142.837080:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828142.837084:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828142.837090:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4288 00000100:00080000:3.0:1787828142.837095:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828142.837100:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828142.837108:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828142.837127:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f803c0 x1874669228292608/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787828142.837222:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228292480:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787828142.837288:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228292608:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787828142.900900:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828143.796842:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828143.924930:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828144.820876:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828144.948803:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828145.844799:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000400:00000100:1.0:1787828145.908999:0:2373:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.114:988 00000400:00000100:1.0:1787828145.909071:0:2373:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.114@tcp at host 192.168.203.114 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1787828145.909089:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:1.0:1787828145.909135:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:1.0:1787828145.909143:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000100:00100000:0.0:1787828145.909197:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828142/real 1787828145] req@ffff985610f81e00 x1874669228292480/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828167 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828145.909238:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81e00 x1874669228292480/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828167 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828145.909253:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828145.909259:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828145.909265:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228292480:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787828145.909286:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828142/real 1787828145] req@ffff985610f803c0 x1874669228292608/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828167 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828145.909302:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f803c0 x1874669228292608/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828167 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828145.909312:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828145.909317:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828145.909321:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228292608:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787828145.909330:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828142/real 1787828145] req@ffff985610f81680 x1874669228292352/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828167 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828145.909346:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81680 x1874669228292352/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828167 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828145.909357:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828145.909361:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828145.909366:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228292352:192.168.203.114@tcp:250:kworker.0 00010000:00010000:0.0:1787828145.972857:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828146.868834:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787828146.996873:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828147.892812:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787828147.956919:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828147.956932:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828147.956938:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828147.956953:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4293 00000100:00080000:0.0:1787828147.956968:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828147.956978:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787828147.957084:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6580 x1874669228292736/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828147.957143:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828147.957151:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828147.957156:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828147.957169:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4293 00000100:00080000:0.0:1787828147.957180:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828147.957207:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828147.957225:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828147.957228:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228292736:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787828147.957269:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5a40 x1874669228292864/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828147.957297:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828147.957303:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828147.957307:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828147.957332:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4293 00000100:00080000:0.0:1787828147.957348:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000800:00000100:1.0:1787828147.957349:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00080000:0.0:1787828147.957368:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828147.957382:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828147.957413:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828147/real 1787828147] req@ffff985607df6580 x1874669228292736/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828172 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828147.957419:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df70c0 x1874669228292992/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787828147.957463:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6580 x1874669228292736/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828172 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828147.957486:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828147.957495:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828147.957504:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228292736:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787828147.957552:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228292864:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828147.957608:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828147.957632:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828147/real 1787828147] req@ffff985607df5a40 x1874669228292864/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828172 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828147.957654:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5a40 x1874669228292864/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828172 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828147.957669:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828147.957675:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828147.957681:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228292864:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787828147.957731:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228292992:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828147.957769:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828147.957790:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828147/real 1787828147] req@ffff985607df70c0 x1874669228292992/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828172 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828147.957810:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df70c0 x1874669228292992/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828172 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828147.957822:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828147.957827:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828147.957832:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228292992:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787828148.020926:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828148.916829:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828149.044868:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828149.940862:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828150.068870:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828150.964794:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828151.092907:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828151.988787:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787828152.116955:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828153.012852:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787828153.076895:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828153.076913:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828153.076921:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828153.076941:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4298 00000100:00080000:2.0:1787828153.076959:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828153.076971:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787828153.077097:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d83c0 x1874669228293120/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828153.077146:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828153.077168:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828153.077172:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828153.077179:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4298 00000100:00080000:2.0:1787828153.077188:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828153.077196:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787828153.077212:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828153.077220:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228293120:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787828153.077241:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d9680 x1874669228293248/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828153.077264:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828153.077270:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828153.077273:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828153.077297:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4298 00000100:00080000:2.0:1787828153.077306:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828153.077314:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787828153.077324:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000800:00000100:1.0:1787828153.077335:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828153.077369:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8b40 x1874669228293376/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787828153.077414:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828153/real 1787828153] req@ffff9856140d83c0 x1874669228293120/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828178 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828153.077436:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d83c0 x1874669228293120/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828178 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828153.077448:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828153.077453:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828153.077457:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228293120:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787828153.077484:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228293248:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828153.077520:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828153.077532:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828153/real 1787828153] req@ffff9856140d9680 x1874669228293248/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828178 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828153.077543:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d9680 x1874669228293248/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828178 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828153.077555:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828153.077559:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828153.077564:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228293248:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787828153.077572:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228293376:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828153.077600:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828153.077617:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828153/real 1787828153] req@ffff9856140d8b40 x1874669228293376/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828178 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828153.077650:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8b40 x1874669228293376/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828178 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828153.077660:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828153.077664:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828153.077668:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228293376:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787828153.140854:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828154.036835:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828154.164850:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828155.060861:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828155.188913:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828156.084866:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828156.212900:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828157.108785:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787828157.237041:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828158.132837:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787828158.196808:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828158.196822:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828158.196827:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828158.196858:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4303 00000100:00080000:1.0:1787828158.196870:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828158.196879:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787828158.196923:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3c00 x1874669228293504/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828158.196973:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828158.196978:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828158.196981:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828158.196987:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4303 00000100:00080000:1.0:1787828158.196992:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828158.196999:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787828158.197011:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828158.197031:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228293504:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787828158.197033:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0780 x1874669228293632/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828158.197071:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828158.197076:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828158.197079:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828158.197086:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4303 00000100:00080000:1.0:1787828158.197092:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828158.197098:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787828158.197106:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828158.197126:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a03c0 x1874669228293760/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787828158.197141:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828158.197186:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828158/real 1787828158] req@ffff9856085a3c00 x1874669228293504/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828183 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828158.197208:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3c00 x1874669228293504/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828183 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828158.197219:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828158.197224:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828158.197230:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228293504:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787828158.197260:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228293632:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787828158.197292:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828158.197309:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828158/real 1787828158] req@ffff9856085a0780 x1874669228293632/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828183 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828158.197325:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0780 x1874669228293632/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828183 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828158.197332:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828158.197335:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828158.197338:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228293632:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787828158.197343:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228293760:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787828158.197380:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828158.197391:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828158/real 1787828158] req@ffff9856085a03c0 x1874669228293760/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828183 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828158.197400:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a03c0 x1874669228293760/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828183 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828158.197406:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828158.197408:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828158.197410:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228293760:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787828158.260852:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828159.156768:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828159.284842:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828160.180819:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828160.308859:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828161.204773:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828161.332816:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828162.228784:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787828162.356922:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828163.252825:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787828163.317010:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828163.317023:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828163.317029:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828163.317043:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4308 00000100:00080000:2.0:1787828163.317059:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828163.317069:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787828163.317121:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140da940 x1874669228293888/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828163.317176:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828163.317181:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828163.317184:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828163.317189:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4308 00000100:00080000:2.0:1787828163.317194:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828163.317198:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787828163.317212:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828163.317227:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db0c0 x1874669228294016/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828163.317239:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828163.317244:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828163.317247:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828163.317253:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4308 00000100:00080000:2.0:1787828163.317258:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828163.317263:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787828163.317270:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828163.317283:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140dad00 x1874669228294144/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787828163.317305:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228293888:192.168.203.114@tcp:250:kworker.0 00000800:00000100:0.0:1787828163.317436:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828163.317498:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828163/real 1787828163] req@ffff9856140da940 x1874669228293888/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828188 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828163.317523:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140da940 x1874669228293888/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828188 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828163.317539:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828163.317544:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828163.317549:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228293888:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787828163.317559:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228294016:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787828163.317587:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828163.317601:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828163/real 1787828163] req@ffff9856140db0c0 x1874669228294016/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828188 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828163.317626:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db0c0 x1874669228294016/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828188 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828163.317637:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828163.317640:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828163.317644:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228294016:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787828163.317726:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228294144:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787828163.317754:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828163.317775:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828163/real 1787828163] req@ffff9856140dad00 x1874669228294144/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828188 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828163.317791:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140dad00 x1874669228294144/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828188 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828163.317802:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828163.317806:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828163.317810:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228294144:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787828163.380866:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828164.276804:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828164.404815:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828165.300865:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828165.428872:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828166.324817:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828166.452973:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828167.348952:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787828167.476951:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828168.372837:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787828168.436758:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828168.436768:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828168.436772:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828168.436781:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4314 00000100:00080000:2.0:1787828168.436793:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828168.436801:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787828168.436838:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8f00 x1874669228294272/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828168.436865:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828168.436879:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828168.436881:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828168.436884:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4314 00000100:00080000:2.0:1787828168.436888:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828168.436892:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787828168.436901:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828168.436914:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140dbc00 x1874669228294400/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787828168.436925:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228294272:192.168.203.114@tcp:250:kworker.0 00000100:00080000:2.0:1787828168.436929:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828168.436939:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828168.436941:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828168.436945:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4314 00000100:00080000:2.0:1787828168.436949:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828168.436954:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787828168.436958:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828168.436969:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140da1c0 x1874669228294528/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787828168.437000:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828168.437044:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828168/real 1787828168] req@ffff9856140d8f00 x1874669228294272/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828193 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828168.437058:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8f00 x1874669228294272/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828193 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828168.437068:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828168.437073:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828168.437078:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228294272:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787828168.437111:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228294400:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787828168.437146:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828168.437156:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828168/real 1787828168] req@ffff9856140dbc00 x1874669228294400/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828193 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828168.437165:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140dbc00 x1874669228294400/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828193 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828168.437172:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828168.437174:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828168.437176:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228294400:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787828168.437181:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228294528:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787828168.437193:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828168.437199:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828168/real 1787828168] req@ffff9856140da1c0 x1874669228294528/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828193 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828168.437207:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140da1c0 x1874669228294528/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828193 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828168.437212:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828168.437214:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828168.437216:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228294528:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787828168.500842:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828169.396814:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828169.524808:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828170.420772:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828170.548833:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828171.444793:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828171.572775:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828172.468877:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787828172.596988:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828173.492862:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787828173.556806:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828173.556823:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828173.556847:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828173.556865:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4319 00000100:00080000:2.0:1787828173.556882:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828173.556911:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787828173.556962:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db840 x1874669228294656/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828173.556994:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828173.556998:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828173.557000:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828173.557004:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4319 00000100:00080000:2.0:1787828173.557008:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828173.557012:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787828173.557023:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828173.557045:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140da1c0 x1874669228294784/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787828173.557058:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228294656:192.168.203.114@tcp:250:kworker.0 00000100:00080000:2.0:1787828173.557069:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828173.557076:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828173.557079:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828173.557087:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4319 00000100:00080000:2.0:1787828173.557094:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828173.557101:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787828173.557108:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828173.557123:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140dbc00 x1874669228294912/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787828173.557138:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828173.557177:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828173/real 1787828173] req@ffff9856140db840 x1874669228294656/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828198 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828173.557195:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db840 x1874669228294656/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828198 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828173.557207:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828173.557212:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828173.557217:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228294656:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787828173.557243:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228294784:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787828173.557265:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828173.557289:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828173/real 1787828173] req@ffff9856140da1c0 x1874669228294784/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828198 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828173.557302:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140da1c0 x1874669228294784/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828198 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828173.557311:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828173.557314:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828173.557318:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228294784:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787828173.557323:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228294912:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787828173.557341:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828173.557358:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828173/real 1787828173] req@ffff9856140dbc00 x1874669228294912/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828198 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828173.557377:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140dbc00 x1874669228294912/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828198 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828173.557385:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828173.557388:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828173.557391:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228294912:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787828173.620804:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828174.516788:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828174.644845:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828175.540861:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828175.668906:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828176.564881:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828176.692931:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828177.588884:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787828177.717024:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828178.612787:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787828178.676828:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828178.676839:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828178.676844:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828178.676856:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4324 00000100:00080000:0.0:1787828178.676869:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828178.676877:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787828178.676923:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5680 x1874669228295040/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828178.676953:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828178.676957:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828178.676959:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828178.676963:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4324 00000100:00080000:0.0:1787828178.676967:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828178.676973:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828178.676985:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828178.677001:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df43c0 x1874669228295168/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787828178.677004:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228295040:192.168.203.114@tcp:250:kworker.0 00000100:00080000:0.0:1787828178.677017:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828178.677020:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828178.677021:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828178.677024:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4324 00000100:00080000:0.0:1787828178.677028:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828178.677031:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828178.677035:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828178.677045:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5e00 x1874669228295296/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787828178.677071:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828178.677111:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828178/real 1787828178] req@ffff985607df5680 x1874669228295040/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828203 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828178.677125:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5680 x1874669228295040/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828203 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828178.677134:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828178.677137:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828178.677140:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228295040:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787828178.677161:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228295168:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828178.677177:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828178.677185:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828178/real 1787828178] req@ffff985607df43c0 x1874669228295168/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828203 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828178.677200:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df43c0 x1874669228295168/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828203 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828178.677206:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828178.677208:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828178.677210:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228295168:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787828178.677214:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228295296:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828178.677225:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828178.677232:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828178/real 1787828178] req@ffff985607df5e00 x1874669228295296/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828203 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828178.677240:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5e00 x1874669228295296/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828203 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828178.677245:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828178.677247:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828178.677249:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228295296:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787828178.740840:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828179.636863:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828179.764841:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828180.660820:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828180.788812:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828181.684860:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828181.812885:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828182.708827:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787828182.836957:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828183.732803:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787828183.796831:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828183.796839:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828183.796844:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828183.796854:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4329 00000100:00080000:2.0:1787828183.796866:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828183.796875:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787828183.796918:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8f00 x1874669228295424/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828183.796951:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828183.796955:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828183.796957:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828183.796960:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4329 00000100:00080000:2.0:1787828183.796964:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828183.796967:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787828183.796977:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828183.796989:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140dad00 x1874669228295552/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828183.797000:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828183.797004:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828183.797006:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828183.797009:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4329 00000100:00080000:2.0:1787828183.797019:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00100000:1.0:1787828183.797023:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228295424:192.168.203.114@tcp:250:kworker.0 00000100:00080000:2.0:1787828183.797025:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787828183.797035:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828183.797046:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db0c0 x1874669228295680/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787828183.797131:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828183.797174:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828183/real 1787828183] req@ffff9856140d8f00 x1874669228295424/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828208 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828183.797193:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8f00 x1874669228295424/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828208 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828183.797204:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828183.797208:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828183.797211:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228295424:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787828183.797219:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228295552:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828183.797265:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828183.797282:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828183/real 1787828183] req@ffff9856140dad00 x1874669228295552/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828208 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828183.797322:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140dad00 x1874669228295552/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828208 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828183.797340:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828183.797347:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828183.797354:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228295552:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787828183.797405:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228295680:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828183.797469:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828183.797480:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828183/real 1787828183] req@ffff9856140db0c0 x1874669228295680/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828208 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828183.797491:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db0c0 x1874669228295680/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828208 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828183.797498:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828183.797501:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828183.797504:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228295680:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787828183.860878:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828184.756875:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828184.884878:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828185.780779:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828185.908826:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828186.804803:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828186.932786:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828187.828770:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787828187.956920:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828188.852763:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787828188.916801:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828188.916809:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828188.916813:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828188.916822:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4334 00000100:00080000:1.0:1787828188.916833:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828188.916839:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787828188.916874:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a1680 x1874669228295808/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828188.916901:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828188.916904:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828188.916906:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828188.916909:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4334 00000100:00080000:1.0:1787828188.916911:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828188.916913:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787828188.916922:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828188.916931:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3840 x1874669228295936/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828188.916939:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828188.916942:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828188.916943:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828188.916947:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4334 00000100:00100000:0.0:1787828188.916949:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228295808:192.168.203.114@tcp:250:kworker.0 00000100:00080000:1.0:1787828188.916950:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828188.916957:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787828188.916961:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828188.916968:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2580 x1874669228296064/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787828188.917027:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828188.917061:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828188/real 1787828188] req@ffff9856085a1680 x1874669228295808/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828213 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828188.917084:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a1680 x1874669228295808/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828213 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828188.917093:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828188.917097:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828188.917100:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228295808:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787828188.917121:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228295936:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787828188.917138:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828188.917147:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828188/real 1787828188] req@ffff9856085a3840 x1874669228295936/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828213 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828188.917155:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3840 x1874669228295936/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828213 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828188.917161:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828188.917164:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828188.917166:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228295936:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787828188.917176:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228296064:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787828188.917188:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828188.917194:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828188/real 1787828188] req@ffff9856085a2580 x1874669228296064/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828213 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828188.917202:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2580 x1874669228296064/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828213 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828188.917208:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828188.917209:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828188.917211:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228296064:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787828188.980874:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828189.876816:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828190.004903:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828190.900861:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828191.028825:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828191.924787:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828192.052821:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828192.948802:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787828193.076833:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828193.972771:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787828194.036782:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828194.036792:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828194.036796:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828194.036804:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4339 00000100:00080000:1.0:1787828194.036816:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828194.036823:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787828194.036859:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a12c0 x1874669228296192/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828194.036886:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828194.036889:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828194.036895:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828194.036898:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4339 00000100:00080000:1.0:1787828194.036900:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828194.036904:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787828194.036913:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828194.036924:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a1a40 x1874669228296320/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828194.036932:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828194.036935:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828194.036936:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1787828194.036942:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228296192:192.168.203.114@tcp:250:kworker.0 00000100:00080000:1.0:1787828194.036943:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4339 00000100:00080000:1.0:1787828194.036948:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828194.036953:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787828194.036956:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828194.036964:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2940 x1874669228296448/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787828194.037019:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828194.037053:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828194/real 1787828194] req@ffff9856085a12c0 x1874669228296192/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828219 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828194.037068:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a12c0 x1874669228296192/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828219 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828194.037077:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828194.037081:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828194.037085:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228296192:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787828194.037111:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228296320:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787828194.037144:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828194.037161:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828194/real 1787828194] req@ffff9856085a1a40 x1874669228296320/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828219 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828194.037171:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a1a40 x1874669228296320/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828219 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828194.037178:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828194.037180:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828194.037183:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228296320:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787828194.037191:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228296448:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787828194.037208:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828194.037219:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828194/real 1787828194] req@ffff9856085a2940 x1874669228296448/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828219 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828194.037228:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2940 x1874669228296448/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828219 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828194.037233:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828194.037236:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828194.037238:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228296448:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787828194.100815:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828194.996771:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828195.124828:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828196.020794:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828196.148881:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828197.044810:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828197.172858:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828198.068789:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828198.196898:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828199.092764:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787828199.156790:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828199.156802:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828199.156806:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828199.156818:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4344 00000100:00080000:0.0:1787828199.156834:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828199.156844:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787828199.156880:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4f00 x1874669228296576/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828199.156917:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828199.156922:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828199.156924:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828199.156928:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4344 00000100:00080000:0.0:1787828199.156932:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828199.156936:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828199.156945:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828199.156957:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6d00 x1874669228296704/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787828199.156968:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228296576:192.168.203.114@tcp:250:kworker.0 00000100:00080000:0.0:1787828199.156985:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828199.156988:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828199.156990:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828199.156994:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4344 00000100:00080000:0.0:1787828199.156997:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828199.157001:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828199.157005:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828199.157014:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df61c0 x1874669228296832/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787828199.157057:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828199.157094:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828199/real 1787828199] req@ffff985607df4f00 x1874669228296576/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828224 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828199.157108:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4f00 x1874669228296576/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828224 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828199.157118:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828199.157122:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828199.157125:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228296576:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787828199.157147:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228296704:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828199.157164:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828199.157173:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828199/real 1787828199] req@ffff985607df6d00 x1874669228296704/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828224 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828199.157181:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6d00 x1874669228296704/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828224 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828199.157186:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828199.157188:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828199.157190:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228296704:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787828199.157194:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228296832:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828199.157206:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828199.157211:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828199/real 1787828199] req@ffff985607df61c0 x1874669228296832/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828224 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828199.157223:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df61c0 x1874669228296832/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828224 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828199.157228:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828199.157230:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828199.157232:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228296832:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787828199.220798:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828200.116770:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828200.244824:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828201.140766:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828201.268784:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828202.164760:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828202.292798:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828203.188789:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828203.316871:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828204.212760:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787828204.276831:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828204.276842:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828204.276847:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828204.276856:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4349 00000100:00080000:2.0:1787828204.276865:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828204.276878:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787828204.276917:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140da940 x1874669228296960/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828204.276939:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828204.276941:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828204.276943:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828204.276946:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4349 00000100:00080000:2.0:1787828204.276948:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828204.276950:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787828204.276958:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828204.276968:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8b40 x1874669228297088/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828204.276976:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828204.276979:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828204.276982:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1787828204.276985:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228296960:192.168.203.114@tcp:250:kworker.0 00000100:00080000:2.0:1787828204.276987:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4349 00000100:00080000:2.0:1787828204.276993:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828204.276997:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787828204.277000:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828204.277014:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d9680 x1874669228297216/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787828204.277057:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828204.277089:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828204/real 1787828204] req@ffff9856140da940 x1874669228296960/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828229 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828204.277103:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140da940 x1874669228296960/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828229 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828204.277111:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828204.277114:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828204.277118:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228296960:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787828204.277146:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228297088:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828204.277161:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828204.277169:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828204/real 1787828204] req@ffff9856140d8b40 x1874669228297088/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828229 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828204.277177:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8b40 x1874669228297088/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828229 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828204.277188:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828204.277190:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828204.277192:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228297088:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787828204.277196:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228297216:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828204.277207:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828204.277213:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828204/real 1787828204] req@ffff9856140d9680 x1874669228297216/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828229 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828204.277220:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d9680 x1874669228297216/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828229 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828204.277225:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828204.277226:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828204.277228:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228297216:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787828204.340803:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828205.236767:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828205.364803:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828206.260767:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828206.388861:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828207.284770:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828207.412801:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828208.308768:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828208.436823:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828209.332760:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787828209.396827:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828209.396835:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828209.396839:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828209.396849:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4355 00000100:00080000:1.0:1787828209.396860:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828209.396866:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787828209.396901:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a21c0 x1874669228297344/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828209.396929:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828209.396932:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828209.396934:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828209.396937:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4355 00000100:00080000:1.0:1787828209.396939:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828209.396942:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787828209.396950:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828209.396959:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3480 x1874669228297472/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828209.396968:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828209.396971:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828209.396972:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828209.396975:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4355 00000100:00080000:1.0:1787828209.396977:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828209.396979:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787828209.396982:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828209.396989:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228297344:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787828209.396991:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a30c0 x1874669228297600/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787828209.397089:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228297472:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787828209.397122:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228297600:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787828209.460813:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828210.356795:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828210.484799:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828211.380762:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828211.508838:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828212.404744:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000400:00000100:1.0:1787828212.468932:0:2370:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.114:988 00000400:00000100:1.0:1787828212.468973:0:2370:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.114@tcp at host 192.168.203.114 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1787828212.468989:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:1.0:1787828212.469023:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:1.0:1787828212.469031:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000100:00100000:0.0:1787828212.469078:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828209/real 1787828212] req@ffff9856085a30c0 x1874669228297600/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828234 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828212.469109:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a30c0 x1874669228297600/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828234 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828212.469123:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828212.469129:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828212.469133:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228297600:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787828212.469142:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828209/real 1787828212] req@ffff9856085a21c0 x1874669228297344/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828234 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828212.469155:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a21c0 x1874669228297344/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828234 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828212.469162:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828212.469166:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828212.469169:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228297344:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787828212.469179:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828209/real 1787828212] req@ffff9856085a3480 x1874669228297472/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828234 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828212.469191:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3480 x1874669228297472/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828234 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828212.469199:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828212.469203:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828212.469206:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228297472:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787828212.532788:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828213.428759:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828213.556833:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828214.452772:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787828214.516786:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828214.516798:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828214.516803:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828214.516815:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4360 00000100:00080000:2.0:1787828214.516827:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828214.516834:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787828214.516870:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d83c0 x1874669228297728/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828214.516898:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828214.516901:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828214.516903:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828214.516905:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4360 00000100:00080000:2.0:1787828214.516908:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828214.516911:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787828214.516921:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828214.516933:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d9a40 x1874669228297856/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828214.516947:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1787828214.516955:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228297728:192.168.203.114@tcp:250:kworker.0 00000100:00080000:2.0:1787828214.516957:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828214.516962:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828214.516970:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4360 00000100:00080000:2.0:1787828214.516977:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828214.516983:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787828214.516989:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828214.517002:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db480 x1874669228297984/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787828214.517044:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828214.517085:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828214/real 1787828214] req@ffff9856140d83c0 x1874669228297728/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828239 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828214.517106:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d83c0 x1874669228297728/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828239 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828214.517124:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828214.517129:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828214.517134:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228297728:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787828214.517164:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228297856:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787828214.517192:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828214.517205:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828214/real 1787828214] req@ffff9856140d9a40 x1874669228297856/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828239 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828214.517224:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d9a40 x1874669228297856/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828239 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828214.517233:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828214.517236:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828214.517240:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228297856:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787828214.517247:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228297984:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787828214.517270:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828214.517281:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828214/real 1787828214] req@ffff9856140db480 x1874669228297984/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828239 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828214.517294:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db480 x1874669228297984/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828239 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828214.517302:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828214.517305:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828214.517309:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228297984:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787828214.580845:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828215.476830:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828215.604852:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828216.500822:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828216.628826:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828217.524810:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828217.652869:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828218.548790:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828218.676837:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828219.572780:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787828219.636775:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828219.636786:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828219.636791:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828219.636802:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4365 00000100:00080000:2.0:1787828219.636815:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828219.636824:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787828219.636871:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8780 x1874669228298112/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828219.636896:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828219.636899:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828219.636900:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828219.636903:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4365 00000100:00080000:2.0:1787828219.636906:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828219.636909:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787828219.636918:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828219.636929:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db480 x1874669228298240/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828219.636937:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828219.636940:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828219.636942:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828219.636945:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4365 00000100:00080000:2.0:1787828219.636947:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828219.636949:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787828219.636952:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828219.636958:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228298112:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787828219.636962:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d9a40 x1874669228298368/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787828219.637049:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828219.637094:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828219/real 1787828219] req@ffff9856140d8780 x1874669228298112/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828244 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828219.637116:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8780 x1874669228298112/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828244 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828219.637128:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828219.637132:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828219.637137:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228298112:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787828219.637146:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228298240:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787828219.637171:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828219.637186:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828219/real 1787828219] req@ffff9856140db480 x1874669228298240/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828244 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828219.637200:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db480 x1874669228298240/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828244 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828219.637210:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828219.637214:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828219.637218:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228298240:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787828219.637249:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228298368:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787828219.637272:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828219.637284:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828219/real 1787828219] req@ffff9856140d9a40 x1874669228298368/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828244 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828219.637298:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d9a40 x1874669228298368/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828244 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828219.637314:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828219.637318:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828219.637321:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228298368:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787828219.700807:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828220.596742:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828220.724800:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828221.620802:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828221.748825:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828222.644810:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828222.772780:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828223.668769:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828223.796800:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828224.692782:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787828224.756755:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828224.756765:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828224.756768:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828224.756778:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4370 00000100:00080000:2.0:1787828224.756788:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828224.756795:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787828224.756842:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d83c0 x1874669228298496/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828224.756866:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828224.756868:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828224.756870:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828224.756872:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4370 00000100:00080000:2.0:1787828224.756875:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828224.756877:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787828224.756887:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828224.756895:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d9680 x1874669228298624/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828224.756902:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828224.756904:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828224.756905:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828224.756907:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4370 00000100:00080000:2.0:1787828224.756909:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828224.756911:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787828224.756913:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828224.756918:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8b40 x1874669228298752/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787828224.756955:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228298496:192.168.203.114@tcp:250:kworker.0 00000800:00000100:0.0:1787828224.757015:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828224.757047:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828224/real 1787828224] req@ffff9856140d83c0 x1874669228298496/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828249 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828224.757061:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d83c0 x1874669228298496/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828249 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828224.757069:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828224.757072:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828224.757075:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228298496:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787828224.757080:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228298624:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787828224.757093:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828224.757101:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828224/real 1787828224] req@ffff9856140d9680 x1874669228298624/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828249 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828224.757109:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d9680 x1874669228298624/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828249 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828224.757115:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828224.757117:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828224.757119:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228298624:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787828224.757123:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228298752:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787828224.757133:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828224.757138:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828224/real 1787828224] req@ffff9856140d8b40 x1874669228298752/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828249 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828224.757145:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8b40 x1874669228298752/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828249 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828224.757150:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828224.757152:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828224.757154:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228298752:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787828224.820789:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828225.716819:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828225.844803:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828226.740762:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828226.868806:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828227.764810:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828227.892831:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828228.788749:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828228.916888:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828229.812775:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787828229.876771:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828229.876778:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828229.876781:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828229.876796:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4375 00000100:00080000:0.0:1787828229.876808:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828229.876817:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787828229.876861:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7840 x1874669228298880/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828229.876889:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828229.876892:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828229.876893:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828229.876896:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4375 00000100:00080000:0.0:1787828229.876899:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828229.876903:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828229.876911:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828229.876924:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df52c0 x1874669228299008/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828229.876932:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228298880:192.168.203.114@tcp:250:kworker.0 00000100:00080000:0.0:1787828229.876937:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828229.876942:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828229.876945:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828229.876959:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4375 00000100:00080000:0.0:1787828229.876965:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828229.876969:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828229.876976:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828229.876993:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7480 x1874669228299136/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828229.877003:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828229.877036:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828229/real 1787828229] req@ffff985607df7840 x1874669228298880/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828254 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828229.877054:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7840 x1874669228298880/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828254 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828229.877065:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828229.877069:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828229.877074:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228298880:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787828229.877101:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228299008:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828229.877122:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828229.877133:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828229/real 1787828229] req@ffff985607df52c0 x1874669228299008/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828254 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828229.877142:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df52c0 x1874669228299008/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828254 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828229.877148:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828229.877150:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828229.877152:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228299008:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787828229.877157:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228299136:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828229.877169:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828229.877185:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828229/real 1787828229] req@ffff985607df7480 x1874669228299136/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828254 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828229.877194:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7480 x1874669228299136/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828254 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828229.877200:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828229.877202:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828229.877205:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228299136:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787828229.940771:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828230.836761:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828230.964823:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828231.860749:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828231.988804:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828232.884773:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828233.012804:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828233.908782:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828234.036817:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828234.932811:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787828234.996862:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828234.996879:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828234.996886:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828234.996903:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4380 00000100:00080000:2.0:1787828234.996935:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828234.996946:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787828234.997013:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140da940 x1874669228299264/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828234.997060:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828234.997080:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828234.997083:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828234.997087:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4380 00000100:00080000:2.0:1787828234.997091:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828234.997096:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787828234.997105:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828234.997109:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228299264:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787828234.997119:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db0c0 x1874669228299392/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828234.997128:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828234.997131:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828234.997132:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828234.997135:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4380 00000100:00080000:2.0:1787828234.997137:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828234.997139:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787828234.997142:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828234.997150:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140dad00 x1874669228299520/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787828234.997182:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828234.997212:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828234/real 1787828234] req@ffff9856140da940 x1874669228299264/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828259 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828234.997226:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140da940 x1874669228299264/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828259 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828234.997235:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828234.997238:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828234.997242:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228299264:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787828234.997264:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228299392:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787828234.997281:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828234.997290:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828234/real 1787828234] req@ffff9856140db0c0 x1874669228299392/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828259 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828234.997298:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db0c0 x1874669228299392/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828259 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828234.997304:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828234.997306:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828234.997308:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228299392:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787828234.997331:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228299520:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787828234.997351:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828234.997361:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828234/real 1787828234] req@ffff9856140dad00 x1874669228299520/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828259 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828234.997373:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140dad00 x1874669228299520/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828259 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828234.997381:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828234.997383:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828234.997386:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228299520:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787828235.060845:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828235.956831:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828236.084925:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828236.980798:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828237.108878:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828238.004774:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828238.132865:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828239.028816:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828239.156968:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828240.052792:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787828240.116795:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828240.116809:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828240.116815:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828240.116826:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4385 00000100:00080000:2.0:1787828240.116839:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828240.116846:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787828240.116904:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8f00 x1874669228299648/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828240.116931:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828240.116934:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828240.116936:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828240.116940:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4385 00000100:00080000:2.0:1787828240.116943:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828240.116947:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787828240.116958:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828240.116973:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140dbc00 x1874669228299776/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828240.116985:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1787828240.116991:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228299648:192.168.203.114@tcp:250:kworker.0 00000100:00080000:2.0:1787828240.116991:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828240.116997:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828240.117002:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4385 00000100:00080000:2.0:1787828240.117005:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828240.117009:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787828240.117019:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828240.117028:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140da1c0 x1874669228299904/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787828240.117101:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828240.117151:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828240/real 1787828240] req@ffff9856140d8f00 x1874669228299648/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828265 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828240.117176:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8f00 x1874669228299648/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828265 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828240.117206:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828240.117212:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828240.117217:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228299648:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787828240.117269:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228299776:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787828240.117301:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828240.117317:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828240/real 1787828240] req@ffff9856140dbc00 x1874669228299776/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828265 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828240.117332:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140dbc00 x1874669228299776/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828265 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828240.117342:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828240.117345:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828240.117349:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228299776:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787828240.117355:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228299904:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787828240.117417:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828240.117430:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828240/real 1787828240] req@ffff9856140da1c0 x1874669228299904/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828265 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828240.117442:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140da1c0 x1874669228299904/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828265 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828240.117450:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828240.117453:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828240.117456:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228299904:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787828240.180865:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828241.076879:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828241.204849:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828242.100809:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828242.228883:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828243.124846:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828243.252916:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828244.148797:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828244.276835:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828245.172767:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787828245.236816:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828245.236826:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828245.236830:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828245.236840:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4390 00000100:00080000:1.0:1787828245.236849:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828245.236855:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787828245.236894:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0f00 x1874669228300032/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828245.236919:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828245.236922:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828245.236923:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828245.236925:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4390 00000100:00080000:1.0:1787828245.236927:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828245.236930:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787828245.236938:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828245.236949:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3480 x1874669228300160/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828245.236957:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1787828245.236962:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228300032:192.168.203.114@tcp:250:kworker.0 00000100:00080000:1.0:1787828245.236962:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828245.236963:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828245.236971:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4390 00000100:00080000:1.0:1787828245.236974:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828245.236976:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787828245.236979:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828245.236988:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a21c0 x1874669228300288/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787828245.237034:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828245.237071:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828245/real 1787828245] req@ffff9856085a0f00 x1874669228300032/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828270 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828245.237087:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0f00 x1874669228300032/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828270 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828245.237096:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828245.237099:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828245.237101:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228300032:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787828245.237131:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228300160:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787828245.237151:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828245.237160:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828245/real 1787828245] req@ffff9856085a3480 x1874669228300160/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828270 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828245.237168:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3480 x1874669228300160/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828270 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828245.237174:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828245.237176:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828245.237179:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228300160:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787828245.237182:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228300288:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787828245.237193:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828245.237199:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828245/real 1787828245] req@ffff9856085a21c0 x1874669228300288/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828270 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828245.237206:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a21c0 x1874669228300288/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828270 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828245.237216:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828245.237219:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828245.237221:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228300288:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828245.300832:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828246.196813:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828246.324857:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828247.220867:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828247.348907:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828248.244867:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828248.372951:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828249.268850:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828249.396846:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828250.292801:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787828250.356826:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828250.356841:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828250.356846:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828250.356859:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4395 00000100:00080000:1.0:1787828250.356872:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828250.356880:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787828250.356929:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a30c0 x1874669228300416/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828250.356964:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828250.356968:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828250.356970:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828250.356974:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4395 00000100:00080000:1.0:1787828250.356978:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828250.356983:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787828250.356994:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828250.357008:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2940 x1874669228300544/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828250.357019:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828250.357023:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:0.0:1787828250.357027:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228300416:192.168.203.114@tcp:250:kworker.0 00000100:00080000:1.0:1787828250.357027:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828250.357048:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4395 00000100:00080000:1.0:1787828250.357053:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828250.357058:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787828250.357063:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828250.357076:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a1a40 x1874669228300672/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787828250.357107:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828250.357158:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828250/real 1787828250] req@ffff9856085a30c0 x1874669228300416/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828275 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828250.357173:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a30c0 x1874669228300416/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828275 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828250.357181:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828250.357184:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828250.357187:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228300416:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787828250.357210:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228300544:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787828250.357228:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828250.357247:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828250/real 1787828250] req@ffff9856085a2940 x1874669228300544/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828275 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828250.357255:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2940 x1874669228300544/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828275 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828250.357266:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828250.357268:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828250.357270:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228300544:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787828250.357274:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228300672:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787828250.357284:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828250.357290:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828250/real 1787828250] req@ffff9856085a1a40 x1874669228300672/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828275 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828250.357311:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a1a40 x1874669228300672/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828275 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828250.357327:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828250.357329:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828250.357331:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228300672:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828250.420863:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828251.316804:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828251.444847:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828252.340771:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828252.468786:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828253.364830:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828253.492855:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828254.388782:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828254.516833:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828255.412786:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787828255.476812:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828255.476822:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828255.476827:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828255.476838:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4401 00000100:00080000:1.0:1787828255.476850:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828255.476856:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787828255.476894:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a12c0 x1874669228300800/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828255.476920:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828255.476923:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828255.476925:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828255.476927:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4401 00000100:00080000:1.0:1787828255.476930:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828255.476932:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787828255.476948:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828255.476957:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2580 x1874669228300928/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828255.476964:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828255.476967:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828255.476970:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828255.476973:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4401 00000100:00080000:1.0:1787828255.476976:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828255.476983:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787828255.476987:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828255.476990:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228300800:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787828255.476996:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3840 x1874669228301056/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787828255.477065:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828255.477101:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828255/real 1787828255] req@ffff9856085a12c0 x1874669228300800/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828280 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828255.477115:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a12c0 x1874669228300800/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828280 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828255.477128:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828255.477132:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828255.477135:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228300800:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787828255.477140:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228300928:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787828255.477155:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828255.477164:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828255/real 1787828255] req@ffff9856085a2580 x1874669228300928/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828280 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828255.477172:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2580 x1874669228300928/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828280 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828255.477177:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828255.477179:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828255.477181:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228300928:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787828255.477201:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228301056:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787828255.477212:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828255.477218:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828255/real 1787828255] req@ffff9856085a3840 x1874669228301056/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828280 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828255.477225:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3840 x1874669228301056/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828280 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828255.477231:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828255.477232:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828255.477234:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228301056:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828255.540949:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828256.436779:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828256.564914:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828257.460851:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828257.588861:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828258.484814:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828258.612858:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828259.508753:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828259.636782:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828260.532794:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787828260.596814:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828260.596826:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828260.596830:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828260.596841:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4406 00000100:00080000:0.0:1787828260.596860:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828260.596867:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787828260.596912:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6940 x1874669228301184/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828260.596942:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828260.596945:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828260.596946:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828260.596949:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4406 00000100:00080000:0.0:1787828260.596951:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828260.596954:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828260.596962:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828260.596971:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4780 x1874669228301312/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828260.596979:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828260.596981:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828260.596982:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828260.596987:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4406 00000100:00080000:0.0:1787828260.596989:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828260.596991:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828260.596996:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828260.596998:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228301184:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787828260.597008:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4b40 x1874669228301440/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828260.597060:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828260.597095:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828260/real 1787828260] req@ffff985607df6940 x1874669228301184/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828285 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828260.597108:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6940 x1874669228301184/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828285 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828260.597117:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828260.597120:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828260.597124:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228301184:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787828260.597129:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228301312:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828260.597143:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828260.597151:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828260/real 1787828260] req@ffff985607df4780 x1874669228301312/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828285 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828260.597158:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4780 x1874669228301312/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828285 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828260.597164:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828260.597166:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828260.597168:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228301312:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787828260.597190:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228301440:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828260.597201:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828260.597208:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828260/real 1787828260] req@ffff985607df4b40 x1874669228301440/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828285 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828260.597215:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4b40 x1874669228301440/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828285 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828260.597221:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828260.597223:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828260.597238:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228301440:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828260.660905:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828261.556796:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828261.684806:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828262.580780:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828262.708820:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828263.604794:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828263.732869:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828264.628831:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828264.756867:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828265.652802:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787828265.716881:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828265.716891:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828265.716896:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828265.716908:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4411 00000100:00080000:2.0:1787828265.716921:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828265.716931:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787828265.716971:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db840 x1874669228301568/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828265.717004:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828265.717009:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828265.717011:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828265.717017:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4411 00000100:00080000:2.0:1787828265.717022:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828265.717028:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787828265.717039:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228301568:192.168.203.114@tcp:250:kworker.0 00000008:00100000:2.0:1787828265.717041:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828265.717063:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140da1c0 x1874669228301696/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828265.717077:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828265.717082:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828265.717084:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828265.717089:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4411 00000100:00080000:2.0:1787828265.717094:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828265.717099:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787828265.717105:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000800:00000100:0.0:1787828265.717111:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828265.717120:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140dbc00 x1874669228301824/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787828265.717149:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828265/real 1787828265] req@ffff9856140db840 x1874669228301568/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828290 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828265.717165:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db840 x1874669228301568/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828290 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828265.717175:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828265.717179:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828265.717182:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228301568:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787828265.717220:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228301696:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787828265.717244:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828265.717253:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828265/real 1787828265] req@ffff9856140da1c0 x1874669228301696/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828290 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828265.717267:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140da1c0 x1874669228301696/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828290 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828265.717274:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828265.717276:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828265.717279:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228301696:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787828265.717283:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228301824:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787828265.717295:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828265.717301:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828265/real 1787828265] req@ffff9856140dbc00 x1874669228301824/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828290 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828265.717311:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140dbc00 x1874669228301824/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828290 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828265.717317:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828265.717319:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828265.717322:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228301824:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828265.780851:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828266.676865:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828266.804913:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828267.700771:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828267.828911:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828268.724870:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828268.852833:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828269.748775:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828269.876898:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828270.772894:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787828270.836857:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828270.836874:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828270.836882:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828270.836900:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4416 00000100:00080000:0.0:1787828270.836920:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828270.836929:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787828270.836984:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7c00 x1874669228301952/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828270.837042:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828270.837047:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828270.837050:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828270.837054:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4416 00000100:00080000:0.0:1787828270.837059:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828270.837063:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828270.837077:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828270.837092:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4000 x1874669228302080/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828270.837105:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828270.837109:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828270.837111:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828270.837115:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4416 00000100:00080000:0.0:1787828270.837118:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828270.837122:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828270.837127:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828270.837142:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4b40 x1874669228302208/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828270.837196:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228301952:192.168.203.114@tcp:250:kworker.0 00000800:00000100:2.0:1787828270.837284:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828270.837347:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828270/real 1787828270] req@ffff985607df7c00 x1874669228301952/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828295 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828270.837370:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7c00 x1874669228301952/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828295 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828270.837384:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828270.837389:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828270.837395:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228301952:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787828270.837421:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228302080:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828270.837445:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828270.837458:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828270/real 1787828270] req@ffff985607df4000 x1874669228302080/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828295 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828270.837471:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4000 x1874669228302080/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828295 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828270.837480:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828270.837483:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828270.837486:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228302080:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787828270.837514:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228302208:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828270.837530:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828270.837540:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828270/real 1787828270] req@ffff985607df4b40 x1874669228302208/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828295 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828270.837552:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4b40 x1874669228302208/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828295 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828270.837560:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828270.837562:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828270.837565:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228302208:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828270.900942:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828271.796766:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828271.924861:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828272.820830:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828272.948810:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828273.844901:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828273.972827:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828274.868793:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828274.996919:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828275.892789:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787828275.956814:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828275.956825:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828275.956830:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828275.956842:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4421 00000100:00080000:1.0:1787828275.956853:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828275.956867:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787828275.956912:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a1680 x1874669228302336/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828275.956944:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828275.956949:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828275.956951:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828275.956955:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4421 00000100:00080000:1.0:1787828275.956960:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828275.956972:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787828275.956983:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828275.956992:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228302336:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787828275.957000:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a03c0 x1874669228302464/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828275.957015:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828275.957019:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828275.957021:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828275.957025:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4421 00000100:00080000:1.0:1787828275.957029:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828275.957032:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787828275.957036:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828275.957046:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0780 x1874669228302592/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828275.957087:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228302464:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787828275.957109:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228302592:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828276.020934:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828276.916831:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828277.044869:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828277.940761:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828278.068854:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828278.964821:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000400:00000100:3.0:1787828279.028977:0:2371:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.114:988 00000400:00000100:3.0:1787828279.029025:0:2371:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.114@tcp at host 192.168.203.114 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:3.0:1787828279.029040:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:3.0:1787828279.029071:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:3.0:1787828279.029078:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000100:00100000:2.0:1787828279.029145:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828275/real 1787828279] req@ffff9856085a03c0 x1874669228302464/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828300 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828279.029178:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a03c0 x1874669228302464/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828300 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828279.029191:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828279.029196:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828279.029201:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228302464:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787828279.029209:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828275/real 1787828279] req@ffff9856085a0780 x1874669228302592/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828300 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828279.029221:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0780 x1874669228302592/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828300 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828279.029229:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828279.029232:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828279.029236:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228302592:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787828279.029242:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828275/real 1787828279] req@ffff9856085a1680 x1874669228302336/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828300 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828279.029254:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a1680 x1874669228302336/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828300 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828279.029262:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828279.029266:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828279.029270:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228302336:192.168.203.114@tcp:250:kworker.0 00010000:00010000:0.0:1787828279.092856:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828279.988764:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828280.116857:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828281.012808:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787828281.076819:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828281.076829:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828281.076833:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828281.076844:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4426 00000100:00080000:1.0:1787828281.076855:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828281.076862:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787828281.076903:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3c00 x1874669228302720/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828281.076925:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828281.076927:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828281.076929:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828281.076932:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4426 00000100:00080000:1.0:1787828281.076935:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828281.076938:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787828281.076947:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828281.076957:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0b40 x1874669228302848/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828281.076964:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228302720:192.168.203.114@tcp:250:kworker.0 00000100:00080000:1.0:1787828281.076967:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828281.076973:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828281.076976:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828281.076980:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4426 00000100:00080000:1.0:1787828281.076982:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828281.076984:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787828281.076987:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828281.076996:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0000 x1874669228302976/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828281.077042:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828281.077078:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828281/real 1787828281] req@ffff9856085a3c00 x1874669228302720/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828306 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828281.077116:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3c00 x1874669228302720/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828306 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828281.077128:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828281.077133:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828281.077138:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228302720:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787828281.077170:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228302848:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828281.077198:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828281.077212:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828281/real 1787828281] req@ffff9856085a0b40 x1874669228302848/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828306 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828281.077227:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0b40 x1874669228302848/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828306 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828281.077236:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828281.077240:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828281.077244:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228302848:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787828281.077250:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228302976:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828281.077271:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828281.077284:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828281/real 1787828281] req@ffff9856085a0000 x1874669228302976/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828306 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828281.077305:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0000 x1874669228302976/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828306 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828281.077315:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828281.077318:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828281.077322:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228302976:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828281.140926:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828282.036766:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828282.164848:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828283.060785:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828283.188828:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828284.084782:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828284.212829:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828285.108778:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828285.236768:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828286.132805:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787828286.196833:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828286.196846:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828286.196851:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828286.196864:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4431 00000100:00080000:1.0:1787828286.196877:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828286.196887:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787828286.196931:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2d00 x1874669228303104/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828286.196954:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828286.196957:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828286.196958:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828286.196962:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4431 00000100:00080000:1.0:1787828286.196966:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828286.196969:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787828286.196978:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828286.196985:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228303104:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787828286.196992:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0000 x1874669228303232/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828286.197000:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828286.197003:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828286.197004:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828286.197007:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4431 00000100:00080000:1.0:1787828286.197009:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828286.197011:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787828286.197014:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828286.197023:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0b40 x1874669228303360/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828286.197042:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828286.197097:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828286/real 1787828286] req@ffff9856085a2d00 x1874669228303104/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828311 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828286.197141:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2d00 x1874669228303104/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828311 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828286.197152:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828286.197156:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828286.197160:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228303104:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787828286.197185:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228303232:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828286.197206:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828286.197217:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828286/real 1787828286] req@ffff9856085a0000 x1874669228303232/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828311 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828286.197227:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0000 x1874669228303232/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828311 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828286.197251:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828286.197256:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828286.197260:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228303232:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787828286.197268:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228303360:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828286.197292:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828286.197307:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828286/real 1787828286] req@ffff9856085a0b40 x1874669228303360/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828311 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828286.197322:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0b40 x1874669228303360/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828311 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828286.197345:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828286.197348:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828286.197351:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228303360:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828286.260892:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828287.156770:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787828287.284852:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828288.180776:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828288.308790:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828289.204793:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828289.332794:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828290.228829:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828290.356799:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828291.252784:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787828291.316800:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828291.316810:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828291.316814:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828291.316823:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4436 00000100:00080000:2.0:1787828291.316833:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828291.316840:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787828291.316878:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8f00 x1874669228303488/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828291.316903:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828291.316906:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828291.316907:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828291.316910:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4436 00000100:00080000:2.0:1787828291.316912:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828291.316915:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787828291.316923:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828291.316933:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140dad00 x1874669228303616/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828291.316941:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828291.316943:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828291.316945:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828291.316950:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4436 00000100:00100000:0.0:1787828291.316953:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228303488:192.168.203.114@tcp:250:kworker.0 00000100:00080000:2.0:1787828291.316953:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828291.316959:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787828291.316964:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828291.316970:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db0c0 x1874669228303744/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787828291.317032:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828291.317073:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828291/real 1787828291] req@ffff9856140d8f00 x1874669228303488/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828316 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828291.317087:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8f00 x1874669228303488/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828316 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828291.317095:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828291.317097:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828291.317100:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228303488:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787828291.317122:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228303616:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787828291.317138:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828291.317147:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828291/real 1787828291] req@ffff9856140dad00 x1874669228303616/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828316 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828291.317160:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140dad00 x1874669228303616/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828316 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828291.317165:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828291.317167:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828291.317169:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228303616:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787828291.317173:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228303744:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787828291.317183:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828291.317189:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828291/real 1787828291] req@ffff9856140db0c0 x1874669228303744/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828316 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828291.317196:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db0c0 x1874669228303744/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828316 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828291.317201:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828291.317203:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828291.317205:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228303744:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828291.380783:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828292.276792:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787828292.404816:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828293.300827:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828293.428894:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828294.324790:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828294.452917:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828295.348767:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828295.476830:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828296.372802:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787828296.436892:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828296.436904:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828296.436909:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828296.436921:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4442 00000100:00080000:0.0:1787828296.436935:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828296.436945:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787828296.437012:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4780 x1874669228303872/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828296.437047:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828296.437051:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828296.437054:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828296.437059:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4442 00000100:00080000:0.0:1787828296.437064:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828296.437069:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828296.437081:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828296.437084:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228303872:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787828296.437099:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6940 x1874669228304000/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828296.437113:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828296.437117:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828296.437120:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828296.437125:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4442 00000100:00080000:0.0:1787828296.437130:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828296.437135:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828296.437151:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000800:00000100:2.0:1787828296.437161:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828296.437173:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7480 x1874669228304128/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828296.437208:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828296/real 1787828296] req@ffff985607df4780 x1874669228303872/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828321 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828296.437230:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4780 x1874669228303872/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828321 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828296.437247:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828296.437251:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828296.437255:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228303872:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787828296.438428:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228304000:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828296.438491:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828296.438516:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828296/real 1787828296] req@ffff985607df6940 x1874669228304000/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828321 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828296.438529:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6940 x1874669228304000/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828321 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828296.438537:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828296.438541:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828296.438544:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228304000:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787828296.438549:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228304128:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828296.438561:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828296.438567:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828296/real 1787828296] req@ffff985607df7480 x1874669228304128/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828321 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828296.438575:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7480 x1874669228304128/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828321 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828296.438580:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828296.438582:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828296.438584:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228304128:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828296.500839:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828297.396765:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787828297.524893:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828298.420805:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828298.548815:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828299.444767:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828299.572839:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828300.468791:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828300.596825:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828301.492790:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787828301.556783:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828301.556796:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828301.556803:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828301.556820:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4447 00000100:00080000:0.0:1787828301.556833:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828301.556842:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787828301.556899:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df52c0 x1874669228304256/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828301.556924:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828301.556927:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828301.556929:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828301.556934:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4447 00000100:00080000:0.0:1787828301.556938:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00100000:2.0:1787828301.556955:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228304256:192.168.203.114@tcp:250:kworker.0 00000100:00080000:0.0:1787828301.556960:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828301.556970:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828301.556984:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7840 x1874669228304384/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828301.556996:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828301.557001:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828301.557004:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828301.557009:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4447 00000100:00080000:0.0:1787828301.557014:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000800:00000100:2.0:1787828301.557020:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00080000:0.0:1787828301.557021:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828301.557031:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828301.557056:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828301/real 1787828301] req@ffff985607df52c0 x1874669228304256/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828326 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828301.557057:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df61c0 x1874669228304512/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828301.557099:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df52c0 x1874669228304256/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828326 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828301.557110:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828301.557114:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828301.557117:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228304256:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787828301.557138:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228304384:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828301.557154:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828301.557163:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828301/real 1787828301] req@ffff985607df7840 x1874669228304384/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828326 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828301.557171:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7840 x1874669228304384/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828326 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828301.557177:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828301.557179:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828301.557182:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228304384:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787828301.557186:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228304512:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828301.557197:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828301.557203:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828301/real 1787828301] req@ffff985607df61c0 x1874669228304512/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828326 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828301.557211:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df61c0 x1874669228304512/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828326 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828301.557217:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828301.557219:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828301.557221:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228304512:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828301.620893:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828302.516778:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787828302.644874:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828303.540761:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828303.668848:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828304.564811:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828304.692815:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828305.588818:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828305.716827:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828306.612762:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787828306.676807:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828306.676817:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828306.676821:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828306.676831:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4452 00000100:00080000:1.0:1787828306.676841:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828306.676848:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787828306.676885:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3c00 x1874669228304640/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828306.676921:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828306.676924:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828306.676925:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828306.676928:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4452 00000100:00080000:1.0:1787828306.676931:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828306.676934:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787828306.676942:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828306.676953:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a1680 x1874669228304768/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828306.676961:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828306.676964:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828306.676965:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828306.676968:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4452 00000100:00080000:1.0:1787828306.676970:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00100000:2.0:1787828306.676975:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228304640:192.168.203.114@tcp:250:kworker.0 00000100:00080000:1.0:1787828306.676976:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787828306.676982:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828306.676989:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0780 x1874669228304896/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828306.677053:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828306.677086:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828306/real 1787828306] req@ffff9856085a3c00 x1874669228304640/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828331 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828306.677104:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3c00 x1874669228304640/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828331 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828306.677117:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828306.677122:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828306.677127:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228304640:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787828306.677151:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228304768:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828306.677177:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828306.677191:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828306/real 1787828306] req@ffff9856085a1680 x1874669228304768/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828331 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828306.677206:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a1680 x1874669228304768/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828331 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828306.677222:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828306.677227:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828306.677230:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228304768:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787828306.677237:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228304896:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828306.677258:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828306.677270:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828306/real 1787828306] req@ffff9856085a0780 x1874669228304896/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828331 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828306.677283:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0780 x1874669228304896/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828331 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828306.677292:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828306.677296:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828306.677308:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228304896:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828306.740864:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828307.636795:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787828307.764873:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828308.660779:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828308.788916:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828309.684780:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828309.812828:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828310.708840:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828310.836855:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828311.732809:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787828311.796808:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828311.796836:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828311.796841:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828311.796853:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4457 00000100:00080000:2.0:1787828311.796864:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828311.796886:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787828311.796943:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140da940 x1874669228305024/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828311.796974:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828311.796978:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828311.796980:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828311.796983:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4457 00000100:00080000:2.0:1787828311.796986:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828311.796990:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787828311.796998:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828311.797009:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8b40 x1874669228305152/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828311.797019:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828311.797022:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828311.797023:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828311.797027:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4457 00000100:00080000:2.0:1787828311.797029:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828311.797032:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787828311.797036:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828311.797044:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228305024:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787828311.797051:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d9680 x1874669228305280/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787828311.797132:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828311.797183:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828311/real 1787828311] req@ffff9856140da940 x1874669228305024/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828336 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828311.797212:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140da940 x1874669228305024/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828336 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828311.797229:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828311.797236:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828311.797242:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228305024:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787828311.797254:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228305152:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787828311.797293:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828311.797314:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828311/real 1787828311] req@ffff9856140d8b40 x1874669228305152/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828336 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828311.797329:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8b40 x1874669228305152/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828336 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828311.797346:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828311.797349:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828311.797353:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228305152:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787828311.797382:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228305280:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787828311.797405:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828311.797416:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828311/real 1787828311] req@ffff9856140d9680 x1874669228305280/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828336 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828311.797427:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d9680 x1874669228305280/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828336 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828311.797435:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828311.797439:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828311.797452:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228305280:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828311.860903:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828312.756825:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787828312.884983:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828313.780825:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828313.908841:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828314.804803:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828314.932887:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828315.828811:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828315.956933:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828316.852818:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787828316.916858:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828316.916889:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828316.916894:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828316.916906:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4462 00000100:00080000:2.0:1787828316.916920:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828316.916930:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787828316.916978:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d83c0 x1874669228305408/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828316.917014:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828316.917019:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828316.917021:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828316.917025:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4462 00000100:00080000:2.0:1787828316.917029:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828316.917033:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787828316.917045:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828316.917058:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d9a40 x1874669228305536/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828316.917071:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828316.917075:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828316.917077:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828316.917083:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4462 00000100:00080000:2.0:1787828316.917087:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828316.917091:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787828316.917095:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828316.917127:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db480 x1874669228305664/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787828316.917141:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228305408:192.168.203.114@tcp:250:kworker.0 00000800:00000100:0.0:1787828316.917260:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828316.917320:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828316/real 1787828316] req@ffff9856140d83c0 x1874669228305408/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828341 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828316.917339:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d83c0 x1874669228305408/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828341 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828316.917358:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828316.917364:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828316.917371:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228305408:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787828316.917381:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228305536:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787828316.917416:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828316.917435:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828316/real 1787828316] req@ffff9856140d9a40 x1874669228305536/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828341 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828316.917451:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d9a40 x1874669228305536/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828341 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828316.917460:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828316.917463:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828316.917466:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228305536:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787828316.917498:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228305664:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787828316.917520:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828316.917536:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828316/real 1787828316] req@ffff9856140db480 x1874669228305664/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828341 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828316.917546:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db480 x1874669228305664/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828341 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828316.917553:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828316.917556:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828316.917559:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228305664:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828316.980931:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828317.876868:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828318.004943:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828318.900816:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828319.028955:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828319.924810:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828320.052890:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828320.948769:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828321.076977:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828321.972874:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787828322.036918:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828322.036930:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828322.036935:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828322.036946:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4467 00000100:00080000:1.0:1787828322.036959:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828322.036967:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787828322.037022:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a03c0 x1874669228305792/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828322.037056:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828322.037060:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828322.037062:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828322.037067:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4467 00000100:00080000:1.0:1787828322.037072:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828322.037079:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787828322.037091:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828322.037109:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3840 x1874669228305920/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828322.037126:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828322.037131:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828322.037135:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828322.037140:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4467 00000100:00080000:1.0:1787828322.037144:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828322.037148:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787828322.037154:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828322.037173:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2580 x1874669228306048/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787828322.037174:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228305792:192.168.203.114@tcp:250:kworker.0 00000800:00000100:0.0:1787828322.037283:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828322.037346:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828322/real 1787828322] req@ffff9856085a03c0 x1874669228305792/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828347 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828322.037365:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a03c0 x1874669228305792/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828347 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828322.037378:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828322.037382:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828322.037386:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228305792:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787828322.037394:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228305920:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787828322.037414:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828322.037424:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828322/real 1787828322] req@ffff9856085a3840 x1874669228305920/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828347 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828322.037434:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3840 x1874669228305920/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828347 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828322.037442:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828322.037445:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828322.037461:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228305920:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787828322.037492:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228306048:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787828322.037510:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828322.037520:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828322/real 1787828322] req@ffff9856085a2580 x1874669228306048/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828347 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828322.037531:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2580 x1874669228306048/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828347 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828322.037538:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828322.037541:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828322.037544:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228306048:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828322.100921:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828322.996862:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828323.124844:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828324.020777:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828324.148850:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828325.044859:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828325.172871:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828326.068832:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828326.196852:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828327.092857:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787828327.156797:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828327.156819:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828327.156823:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828327.156832:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4472 00000100:00080000:2.0:1787828327.156844:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828327.156851:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787828327.156886:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8780 x1874669228306176/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828327.156910:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828327.156913:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828327.156915:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828327.156918:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4472 00000100:00080000:2.0:1787828327.156920:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828327.156922:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787828327.156930:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828327.156942:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db480 x1874669228306304/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828327.156950:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828327.156953:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828327.156956:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1787828327.156958:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228306176:192.168.203.114@tcp:250:kworker.0 00000100:00080000:2.0:1787828327.156961:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4472 00000100:00080000:2.0:1787828327.156967:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828327.156970:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787828327.156973:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828327.156981:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d9a40 x1874669228306432/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787828327.157041:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828327.157076:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828327/real 1787828327] req@ffff9856140d8780 x1874669228306176/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828352 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828327.157090:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8780 x1874669228306176/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828352 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828327.157098:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828327.157102:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828327.157105:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228306176:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787828327.157127:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228306304:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787828327.157144:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828327.157153:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828327/real 1787828327] req@ffff9856140db480 x1874669228306304/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828352 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828327.157174:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db480 x1874669228306304/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828352 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828327.157183:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828327.157186:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828327.157190:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228306304:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787828327.157196:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228306432:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787828327.157218:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828327.157230:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828327/real 1787828327] req@ffff9856140d9a40 x1874669228306432/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828352 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828327.157238:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d9a40 x1874669228306432/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828352 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828327.157244:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828327.157246:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828327.157248:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228306432:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828327.220858:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828328.116841:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828328.244961:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828329.140860:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828329.268843:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828330.164819:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828330.292923:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828331.188853:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828331.316963:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828332.212848:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787828332.276882:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828332.276894:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828332.276899:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828332.276910:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4477 00000100:00080000:3.0:1787828332.276938:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828332.276946:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828332.276986:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83840 x1874669228306560/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828332.277031:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828332.277037:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828332.277038:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828332.277042:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4477 00000100:00080000:3.0:1787828332.277044:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828332.277047:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828332.277055:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828332.277065:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81a40 x1874669228306688/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828332.277073:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828332.277077:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828332.277080:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828332.277085:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4477 00000100:00080000:3.0:1787828332.277089:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828332.277096:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787828332.277098:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228306560:192.168.203.114@tcp:250:kworker.0 00000008:00100000:3.0:1787828332.277104:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828332.277120:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f812c0 x1874669228306816/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787828332.277199:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828332.277260:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828332/real 1787828332] req@ffff985610f83840 x1874669228306560/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828357 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828332.277278:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83840 x1874669228306560/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828357 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828332.277306:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828332.277311:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828332.277315:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228306560:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787828332.277324:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228306688:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787828332.277351:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828332.277365:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828332/real 1787828332] req@ffff985610f81a40 x1874669228306688/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828357 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828332.277374:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81a40 x1874669228306688/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828357 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828332.277380:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828332.277383:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828332.277385:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228306688:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787828332.277408:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228306816:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787828332.277422:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828332.277429:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828332/real 1787828332] req@ffff985610f812c0 x1874669228306816/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828357 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828332.277442:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f812c0 x1874669228306816/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828357 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828332.277448:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828332.277449:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828332.277451:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228306816:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828332.340932:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828333.236818:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828333.364795:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828334.260803:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828334.388823:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828335.284804:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828335.412798:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828336.308855:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828336.436956:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828337.332835:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787828337.396818:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828337.396831:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828337.396839:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828337.396857:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4483 00000100:00080000:3.0:1787828337.396875:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828337.396889:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828337.396954:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80b40 x1874669228306944/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828337.396996:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828337.397001:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828337.397003:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828337.397008:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4483 00000100:00080000:3.0:1787828337.397014:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828337.397034:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828337.397057:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828337.397089:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228306944:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787828337.397090:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80000 x1874669228307072/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828337.397135:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828337.397142:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828337.397146:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828337.397153:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4483 00000100:00080000:3.0:1787828337.397162:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828337.397171:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828337.397183:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828337.397210:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f830c0 x1874669228307200/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787828337.397218:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828337.397279:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828337/real 1787828337] req@ffff985610f80b40 x1874669228306944/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828362 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828337.397317:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80b40 x1874669228306944/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828362 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828337.397333:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828337.397340:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828337.397346:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228306944:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787828337.397388:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228307072:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787828337.397429:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828337.397445:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828337/real 1787828337] req@ffff985610f80000 x1874669228307072/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828362 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828337.397475:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80000 x1874669228307072/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828362 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828337.397489:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828337.397494:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828337.397499:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228307072:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787828337.397510:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228307200:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787828337.397542:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828337.397560:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828337/real 1787828337] req@ffff985610f830c0 x1874669228307200/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828362 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828337.397580:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f830c0 x1874669228307200/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828362 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828337.397590:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828337.397594:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828337.397599:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228307200:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828337.460933:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828338.356774:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828338.486254:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828339.380761:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828339.508820:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828340.404800:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828340.532802:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828341.428796:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828341.556831:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828342.452802:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787828342.516847:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828342.516861:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828342.516868:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828342.516885:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4488 00000100:00080000:0.0:1787828342.516903:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828342.516916:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787828342.517012:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6d00 x1874669228307328/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828342.517054:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828342.517060:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828342.517062:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828342.517067:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4488 00000100:00080000:0.0:1787828342.517073:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828342.517078:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828342.517092:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828342.517110:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4f00 x1874669228307456/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828342.517127:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1787828342.517130:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228307328:192.168.203.114@tcp:250:kworker.0 00000100:00080000:0.0:1787828342.517135:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828342.517143:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828342.517151:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4488 00000100:00080000:0.0:1787828342.517155:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828342.517160:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828342.517166:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828342.517181:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5e00 x1874669228307584/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828342.517325:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228307456:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787828342.517395:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228307584:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828342.580971:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828343.476826:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828343.604815:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828344.500775:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828344.628896:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828345.524855:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000400:00000100:1.0:1787828345.588937:0:2372:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.114:988 00000400:00000100:1.0:1787828345.588984:0:2372:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.114@tcp at host 192.168.203.114 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1787828345.589001:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:1.0:1787828345.589043:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:1.0:1787828345.589053:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000100:00100000:2.0:1787828345.589122:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828342/real 1787828345] req@ffff985607df4f00 x1874669228307456/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828367 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828345.589145:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4f00 x1874669228307456/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828367 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828345.589159:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828345.589165:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828345.589169:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228307456:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787828345.589179:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828342/real 1787828345] req@ffff985607df5e00 x1874669228307584/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828367 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828345.589193:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5e00 x1874669228307584/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828367 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828345.589202:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828345.589206:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828345.589209:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228307584:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787828345.589217:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828342/real 1787828345] req@ffff985607df6d00 x1874669228307328/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828367 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828345.589231:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6d00 x1874669228307328/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828367 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828345.589247:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828345.589251:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828345.589256:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228307328:192.168.203.114@tcp:250:kworker.0 00010000:00010000:0.0:1787828345.652880:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828346.548844:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828346.676993:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828347.572770:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787828347.636815:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828347.636830:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828347.636836:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828347.636849:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4493 00000100:00080000:0.0:1787828347.636863:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828347.636873:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787828347.636916:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df43c0 x1874669228307712/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828347.636951:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828347.636955:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828347.636958:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828347.636962:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4493 00000100:00080000:0.0:1787828347.636966:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828347.636972:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828347.636985:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828347.637002:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5680 x1874669228307840/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828347.637013:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228307712:192.168.203.114@tcp:250:kworker.0 00000100:00080000:0.0:1787828347.637018:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828347.637030:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828347.637032:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828347.637038:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4493 00000100:00080000:0.0:1787828347.637050:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828347.637057:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828347.637064:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828347.637083:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df70c0 x1874669228307968/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828347.637104:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828347.637154:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828347/real 1787828347] req@ffff985607df43c0 x1874669228307712/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828372 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828347.637176:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df43c0 x1874669228307712/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828372 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828347.637190:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828347.637195:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828347.637200:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228307712:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787828347.637230:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228307840:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828347.637257:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828347.637272:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828347/real 1787828347] req@ffff985607df5680 x1874669228307840/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828372 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828347.637285:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5680 x1874669228307840/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828372 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828347.637292:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828347.637296:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828347.637299:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228307840:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787828347.637308:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228307968:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828347.637330:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828347.637342:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828347/real 1787828347] req@ffff985607df70c0 x1874669228307968/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828372 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828347.637354:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df70c0 x1874669228307968/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828372 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828347.637363:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828347.637367:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828347.637371:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228307968:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828347.700911:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828348.596811:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828348.724860:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828349.620823:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828349.748956:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828350.644840:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828350.772835:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828351.668804:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828351.796834:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828352.692817:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787828352.756818:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828352.756832:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828352.756837:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828352.756849:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4498 00000100:00080000:0.0:1787828352.756863:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828352.756872:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787828352.756920:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5a40 x1874669228308096/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828352.756954:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828352.756958:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828352.756960:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828352.756964:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4498 00000100:00080000:0.0:1787828352.756967:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828352.756970:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828352.756998:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828352.757010:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6580 x1874669228308224/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828352.757028:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228308096:192.168.203.114@tcp:250:kworker.0 00000100:00080000:0.0:1787828352.757033:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828352.757046:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828352.757049:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828352.757054:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4498 00000100:00080000:0.0:1787828352.757057:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828352.757063:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828352.757069:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828352.757087:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df70c0 x1874669228308352/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828352.757115:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828352.757153:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828352/real 1787828352] req@ffff985607df5a40 x1874669228308096/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828377 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828352.757174:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5a40 x1874669228308096/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828377 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828352.757207:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828352.757215:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828352.757224:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228308096:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787828352.757270:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228308224:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828352.757320:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828352.757358:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828352/real 1787828352] req@ffff985607df6580 x1874669228308224/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828377 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828352.757374:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6580 x1874669228308224/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828377 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828352.757381:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828352.757384:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828352.757387:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228308224:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787828352.757392:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228308352:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828352.757437:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828352.757458:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828352/real 1787828352] req@ffff985607df70c0 x1874669228308352/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828377 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828352.757483:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df70c0 x1874669228308352/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828377 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828352.757500:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828352.757507:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828352.757514:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228308352:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828352.820916:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828353.716821:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828353.844946:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828354.740764:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828354.868899:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828355.764838:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828355.892877:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828356.788858:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828356.916866:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828357.812842:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787828357.876825:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828357.876836:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828357.876841:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828357.876852:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4503 00000100:00080000:0.0:1787828357.876882:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828357.876890:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787828357.876954:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5680 x1874669228308480/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828357.876997:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828357.877004:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828357.877024:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828357.877031:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4503 00000100:00080000:0.0:1787828357.877037:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828357.877042:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828357.877055:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828357.877064:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228308480:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787828357.877073:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df43c0 x1874669228308608/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828357.877087:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828357.877090:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828357.877093:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828357.877096:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4503 00000100:00080000:0.0:1787828357.877101:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828357.877106:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828357.877114:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828357.877132:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6d00 x1874669228308736/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828357.877150:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828357.877207:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828357/real 1787828357] req@ffff985607df5680 x1874669228308480/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828382 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828357.877226:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5680 x1874669228308480/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828382 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828357.877245:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828357.877249:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828357.877254:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228308480:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787828357.877290:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228308608:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828357.877332:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828357.877350:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828357/real 1787828357] req@ffff985607df43c0 x1874669228308608/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828382 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828357.877369:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df43c0 x1874669228308608/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828382 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828357.877397:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828357.877403:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828357.877409:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228308608:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787828357.877422:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228308736:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828357.877474:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828357.877493:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828357/real 1787828357] req@ffff985607df6d00 x1874669228308736/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828382 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828357.877504:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6d00 x1874669228308736/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828382 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828357.877529:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828357.877533:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828357.877537:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228308736:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828357.940973:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828358.836822:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828358.964844:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828359.860748:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828359.988826:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828360.884757:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828361.012826:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828361.908813:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828362.036829:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828362.932816:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787828362.996920:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828362.996936:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828362.996942:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828362.996958:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4508 00000100:00080000:3.0:1787828362.996975:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828362.999052:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828362.999124:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83c00 x1874669228308864/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828362.999162:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828362.999166:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828362.999169:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828362.999173:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4508 00000100:00080000:3.0:1787828362.999178:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828362.999183:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828362.999196:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828362.999212:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80780 x1874669228308992/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828362.999244:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828362.999251:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828362.999253:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828362.999259:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4508 00000100:00080000:3.0:1787828362.999262:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00100000:2.0:1787828362.999264:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228308864:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787828362.999268:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828362.999281:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828362.999314:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f830c0 x1874669228309120/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828362.999401:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828362.999478:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828362/real 1787828362] req@ffff985610f83c00 x1874669228308864/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828387 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828362.999506:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83c00 x1874669228308864/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828387 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828362.999559:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828362.999563:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828362.999569:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228308864:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787828362.999629:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228308992:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828362.999667:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828362.999685:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828362/real 1787828362] req@ffff985610f80780 x1874669228308992/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828387 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828362.999701:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80780 x1874669228308992/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828387 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828362.999712:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828362.999754:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828362.999759:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228308992:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787828362.999768:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228309120:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828362.999795:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828362.999810:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828362/real 1787828362] req@ffff985610f830c0 x1874669228309120/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828387 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828362.999827:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f830c0 x1874669228309120/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828387 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828362.999838:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828362.999842:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828362.999846:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228309120:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828363.061012:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828363.956841:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828364.084899:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828364.980843:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828365.108890:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828366.004844:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828366.132840:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828367.028836:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828367.156933:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828368.052817:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787828368.116888:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828368.116901:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828368.116908:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828368.116925:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4513 00000100:00080000:1.0:1787828368.116941:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828368.116952:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787828368.117007:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a12c0 x1874669228309248/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828368.117052:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828368.117057:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828368.117060:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828368.117065:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4513 00000100:00080000:1.0:1787828368.117069:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828368.117074:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787828368.117088:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828368.117103:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a1a40 x1874669228309376/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828368.117116:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828368.117120:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828368.117123:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828368.117128:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4513 00000100:00080000:1.0:1787828368.117132:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828368.117152:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787828368.117158:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828368.117171:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2940 x1874669228309504/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828368.117197:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228309248:192.168.203.114@tcp:250:kworker.0 00000800:00000100:2.0:1787828368.117321:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828368.117376:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828368/real 1787828368] req@ffff9856085a12c0 x1874669228309248/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828393 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828368.117404:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a12c0 x1874669228309248/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828393 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828368.117420:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828368.117425:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828368.117431:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228309248:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787828368.117441:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228309376:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828368.117472:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828368.117520:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828368/real 1787828368] req@ffff9856085a1a40 x1874669228309376/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828393 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828368.117537:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a1a40 x1874669228309376/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828393 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828368.117563:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828368.117568:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828368.117572:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228309376:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787828368.117607:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228309504:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828368.117634:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828368.117646:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828368/real 1787828368] req@ffff9856085a2940 x1874669228309504/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828393 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828368.117660:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2940 x1874669228309504/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828393 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828368.117679:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828368.117683:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828368.117687:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228309504:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828368.180958:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828369.076915:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828369.204957:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828370.100848:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828370.228866:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828371.124830:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828371.252865:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828372.148849:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828372.276882:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828373.172809:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787828373.236879:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828373.236887:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828373.236891:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828373.236903:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4518 00000100:00080000:1.0:1787828373.236914:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828373.236922:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787828373.236970:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a30c0 x1874669228309632/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828373.237010:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828373.237014:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828373.237016:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828373.237020:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4518 00000100:00080000:1.0:1787828373.237023:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828373.237027:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787828373.237036:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828373.237044:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228309632:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787828373.237049:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a21c0 x1874669228309760/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828373.237060:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828373.237063:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828373.237064:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828373.237068:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4518 00000100:00080000:1.0:1787828373.237070:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828373.237073:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787828373.237078:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828373.237088:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3480 x1874669228309888/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828373.237130:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828373.237184:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828373/real 1787828373] req@ffff9856085a30c0 x1874669228309632/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828398 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828373.237264:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a30c0 x1874669228309632/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828398 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828373.237275:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828373.237278:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828373.237281:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228309632:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787828373.237315:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228309760:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787828373.237475:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828373.237524:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828373/real 1787828373] req@ffff9856085a21c0 x1874669228309760/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828398 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828373.237537:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a21c0 x1874669228309760/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828398 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828373.237547:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828373.237550:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828373.237554:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228309760:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787828373.237560:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228309888:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787828373.237579:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828373.237588:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828373/real 1787828373] req@ffff9856085a3480 x1874669228309888/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828398 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828373.237596:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3480 x1874669228309888/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828398 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828373.237602:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828373.237604:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828373.237607:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228309888:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828373.300824:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828374.196835:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828374.324977:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828375.220864:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828375.348811:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828376.244879:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828376.372883:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828377.268819:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828377.396787:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828378.292799:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787828378.356823:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828378.356840:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828378.356847:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828378.356862:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4523 00000100:00080000:1.0:1787828378.356880:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828378.356893:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787828378.356949:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0f00 x1874669228310016/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828378.356987:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828378.356992:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828378.356995:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828378.357000:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4523 00000100:00080000:1.0:1787828378.357005:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828378.357009:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787828378.357021:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828378.357038:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3480 x1874669228310144/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828378.357051:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828378.357056:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828378.357060:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828378.357065:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4523 00000100:00080000:1.0:1787828378.357069:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828378.357073:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787828378.357078:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828378.357089:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a21c0 x1874669228310272/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828378.357127:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228310016:192.168.203.114@tcp:250:kworker.0 00000800:00000100:3.0:1787828378.357246:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828378.357294:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828378/real 1787828378] req@ffff9856085a0f00 x1874669228310016/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828403 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828378.357323:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0f00 x1874669228310016/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828403 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828378.357333:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828378.357337:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828378.357340:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228310016:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787828378.357347:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228310144:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787828378.357387:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828378.357401:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828378/real 1787828378] req@ffff9856085a3480 x1874669228310144/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828403 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828378.357430:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3480 x1874669228310144/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828403 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828378.357440:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828378.357444:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828378.357463:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228310144:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787828378.357491:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228310272:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787828378.357508:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828378.357516:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828378/real 1787828378] req@ffff9856085a21c0 x1874669228310272/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828403 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828378.357524:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a21c0 x1874669228310272/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828403 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828378.357530:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828378.357532:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828378.357535:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228310272:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828378.420962:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828379.316807:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828379.444830:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828380.340798:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828380.468804:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828381.364833:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828381.492884:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828382.388803:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828382.516795:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828383.412832:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787828383.476825:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828383.476837:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828383.476841:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828383.476852:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4529 00000100:00080000:0.0:1787828383.476874:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828383.476881:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787828383.476964:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5e00 x1874669228310400/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828383.477012:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828383.477019:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828383.477022:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828383.477029:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4529 00000100:00080000:0.0:1787828383.477035:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828383.477044:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828383.477059:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828383.477076:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4f00 x1874669228310528/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828383.477089:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828383.477094:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828383.477096:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828383.477100:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4529 00000100:00080000:0.0:1787828383.477104:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828383.477108:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828383.477116:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228310400:192.168.203.114@tcp:250:kworker.0 00000008:00100000:0.0:1787828383.477117:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828383.477134:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df61c0 x1874669228310656/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828383.477243:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828383.477320:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828383/real 1787828383] req@ffff985607df5e00 x1874669228310400/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828408 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828383.477353:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5e00 x1874669228310400/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828408 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828383.477367:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828383.477408:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828383.477414:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228310400:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787828383.477444:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228310528:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787828383.477474:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828383.477484:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828383/real 1787828383] req@ffff985607df4f00 x1874669228310528/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828408 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828383.477493:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4f00 x1874669228310528/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828408 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828383.477499:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828383.477501:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828383.477503:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228310528:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787828383.477526:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228310656:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787828383.477537:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828383.477544:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828383/real 1787828383] req@ffff985607df61c0 x1874669228310656/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828408 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828383.477557:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df61c0 x1874669228310656/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828408 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828383.477562:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828383.477564:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828383.477566:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228310656:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828383.540928:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828384.436881:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828384.564829:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828385.460839:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828385.588890:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828386.484772:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828386.612823:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828387.508818:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828387.636906:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828388.532874:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787828388.596884:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828388.596913:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828388.596918:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828388.596930:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4534 00000100:00080000:3.0:1787828388.596944:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828388.596952:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828388.597020:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80000 x1874669228310784/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828388.597070:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828388.597076:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828388.597079:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828388.597085:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4534 00000100:00080000:3.0:1787828388.597096:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828388.597105:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787828388.597122:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228310784:192.168.203.114@tcp:250:kworker.0 00000008:00100000:3.0:1787828388.597123:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828388.597150:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80b40 x1874669228310912/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828388.597168:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828388.597173:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828388.597176:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828388.597180:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4534 00000100:00080000:3.0:1787828388.597200:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828388.597204:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828388.597210:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000800:00000100:2.0:1787828388.597214:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828388.597228:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f812c0 x1874669228311040/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828388.597262:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828388/real 1787828388] req@ffff985610f80000 x1874669228310784/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828413 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828388.597282:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80000 x1874669228310784/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828413 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828388.597292:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828388.597296:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828388.597301:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228310784:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787828388.597325:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228310912:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828388.597367:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828388.597380:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828388/real 1787828388] req@ffff985610f80b40 x1874669228310912/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828413 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828388.597392:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80b40 x1874669228310912/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828413 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828388.597400:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828388.597403:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828388.597406:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228310912:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787828388.597412:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228311040:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828388.597427:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828388.597435:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828388/real 1787828388] req@ffff985610f812c0 x1874669228311040/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828413 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828388.597444:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f812c0 x1874669228311040/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828413 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828388.597451:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828388.597454:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828388.597456:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228311040:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828388.660901:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828389.556889:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828389.684942:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828390.580804:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828390.708862:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828391.604830:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828391.732883:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828392.628859:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828392.756861:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828393.652860:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787828393.716782:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828393.716794:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828393.716799:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828393.716814:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4539 00000100:00080000:3.0:1787828393.716833:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828393.716846:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828393.716907:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81a40 x1874669228311168/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828393.716952:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828393.716957:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828393.716960:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828393.716965:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4539 00000100:00080000:3.0:1787828393.716971:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828393.716977:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828393.716989:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828393.717009:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83840 x1874669228311296/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828393.717036:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828393.717041:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828393.717043:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828393.717048:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4539 00000100:00080000:3.0:1787828393.717053:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828393.717058:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828393.717064:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828393.717080:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81680 x1874669228311424/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828393.717089:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228311168:192.168.203.114@tcp:250:kworker.0 00000800:00000100:2.0:1787828393.717214:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828393.717274:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828393/real 1787828393] req@ffff985610f81a40 x1874669228311168/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828418 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828393.717329:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81a40 x1874669228311168/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828418 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828393.717346:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828393.717352:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828393.717358:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228311168:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787828393.717370:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228311296:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828393.717406:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828393.717423:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828393/real 1787828393] req@ffff985610f83840 x1874669228311296/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828418 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828393.717439:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83840 x1874669228311296/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828418 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828393.717450:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828393.717454:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828393.717458:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228311296:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787828393.717495:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228311424:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828393.717517:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828393.717529:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828393/real 1787828393] req@ffff985610f81680 x1874669228311424/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828418 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828393.717543:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81680 x1874669228311424/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828418 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828393.717563:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828393.717567:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828393.717571:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228311424:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828393.780940:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828394.676877:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828394.804837:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828395.700908:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828395.828855:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828396.724800:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828396.852779:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828397.748878:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828397.876870:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828398.772827:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787828398.836767:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828398.836775:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828398.836778:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828398.836787:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4544 00000100:00080000:3.0:1787828398.836797:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828398.836803:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828398.836843:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f803c0 x1874669228311552/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828398.836870:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828398.836874:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828398.836876:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828398.836881:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4544 00000100:00080000:3.0:1787828398.836885:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828398.836890:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828398.836900:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828398.836911:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81e00 x1874669228311680/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828398.836920:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1787828398.836923:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228311552:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787828398.836923:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828398.836926:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828398.836933:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4544 00000100:00080000:3.0:1787828398.836936:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828398.836938:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828398.836942:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828398.836955:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80f00 x1874669228311808/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828398.836996:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828398.837030:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828398/real 1787828398] req@ffff985610f803c0 x1874669228311552/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828423 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828398.837047:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f803c0 x1874669228311552/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828423 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828398.837055:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828398.837058:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828398.837061:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228311552:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787828398.837081:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228311680:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828398.837098:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828398.837106:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828398/real 1787828398] req@ffff985610f81e00 x1874669228311680/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828423 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828398.837114:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81e00 x1874669228311680/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828423 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828398.837120:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828398.837122:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828398.837124:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228311680:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787828398.837128:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228311808:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828398.837138:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828398.837144:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828398/real 1787828398] req@ffff985610f80f00 x1874669228311808/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828423 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828398.837151:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80f00 x1874669228311808/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828423 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828398.837156:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828398.837158:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828398.837160:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228311808:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828398.900870:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828399.796823:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828399.924792:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828400.820807:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828400.948930:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828401.844843:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828401.972881:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828402.868827:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828402.996861:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828403.892842:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787828403.956817:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828403.956841:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828403.956846:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828403.956857:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4549 00000100:00080000:0.0:1787828403.956870:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828403.956880:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787828403.956934:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7840 x1874669228311936/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828403.956985:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828403.956991:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828403.956994:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828403.957000:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4549 00000100:00080000:0.0:1787828403.957005:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828403.957012:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828403.957029:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828403.957045:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228311936:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787828403.957048:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df52c0 x1874669228312064/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828403.957065:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828403.957068:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828403.957070:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828403.957074:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4549 00000100:00080000:0.0:1787828403.957077:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828403.957082:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828403.957086:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828403.957096:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7480 x1874669228312192/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828403.957142:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828403.957183:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828403/real 1787828403] req@ffff985607df7840 x1874669228311936/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828428 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828403.957201:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7840 x1874669228311936/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828428 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828403.957211:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828403.957215:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828403.957218:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228311936:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787828403.957273:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228312064:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828403.957308:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828403.957320:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828403/real 1787828403] req@ffff985607df52c0 x1874669228312064/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828428 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828403.957332:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df52c0 x1874669228312064/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828428 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828403.957351:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828403.957354:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828403.957357:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228312064:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787828403.957362:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228312192:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828403.957376:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828403.957384:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828403/real 1787828403] req@ffff985607df7480 x1874669228312192/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828428 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828403.957394:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7480 x1874669228312192/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828428 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828403.957400:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828403.957402:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828403.957405:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228312192:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828404.021093:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828404.916782:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828405.044951:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828405.940842:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828406.068885:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828406.964751:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828407.092780:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828407.988765:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828408.116836:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828409.012803:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787828409.076788:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828409.076801:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828409.076807:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828409.076820:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4554 00000100:00080000:0.0:1787828409.076847:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828409.076873:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787828409.076926:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6940 x1874669228312320/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828409.076971:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828409.076974:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828409.076976:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828409.076980:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4554 00000100:00080000:0.0:1787828409.076983:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828409.076987:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828409.076996:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828409.077028:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4780 x1874669228312448/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828409.077054:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828409.077060:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828409.077064:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828409.077071:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4554 00000100:00080000:0.0:1787828409.077077:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00100000:2.0:1787828409.077086:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228312320:192.168.203.114@tcp:250:kworker.0 00000100:00080000:0.0:1787828409.077089:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828409.077101:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828409.077119:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4b40 x1874669228312576/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828409.077315:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228312448:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787828409.077375:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228312576:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828409.141009:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828410.036761:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787828410.165066:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828411.060833:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828411.188812:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828412.084899:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000400:00000100:1.0:1787828412.149105:0:2373:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.114:988 00000400:00000100:1.0:1787828412.149160:0:2373:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.114@tcp at host 192.168.203.114 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1787828412.149202:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:1.0:1787828412.149273:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:1.0:1787828412.149286:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000100:00100000:2.0:1787828412.149343:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828409/real 1787828412] req@ffff985607df4b40 x1874669228312576/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828434 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828412.149380:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4b40 x1874669228312576/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828434 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828412.149397:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828412.149404:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828412.149411:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228312576:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787828412.149421:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828409/real 1787828412] req@ffff985607df6940 x1874669228312320/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828434 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828412.149438:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6940 x1874669228312320/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828434 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828412.149447:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828412.149452:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828412.149457:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228312320:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787828412.149466:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828409/real 1787828412] req@ffff985607df4780 x1874669228312448/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828434 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828412.149482:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4780 x1874669228312448/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828434 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828412.149493:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828412.149498:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828412.149502:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228312448:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787828412.212902:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828413.108876:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828413.236870:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828414.132827:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787828414.196905:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828414.196922:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828414.196929:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828414.196944:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4559 00000100:00080000:3.0:1787828414.196975:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828414.196987:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828414.197034:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82d00 x1874669228312704/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828414.197072:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828414.197079:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828414.197083:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828414.197090:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4559 00000100:00080000:3.0:1787828414.197097:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828414.197106:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828414.197122:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828414.197182:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82940 x1874669228312832/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828414.197207:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228312704:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787828414.197213:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828414.197232:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828414.197236:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828414.197245:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4559 00000100:00080000:3.0:1787828414.197252:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828414.197259:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828414.197267:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828414.197284:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82580 x1874669228312960/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828414.197346:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828414.197407:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828414/real 1787828414] req@ffff985610f82d00 x1874669228312704/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828439 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828414.197436:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82d00 x1874669228312704/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828439 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828414.197453:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828414.197459:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828414.197466:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228312704:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787828414.197509:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228312832:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828414.197545:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828414.197564:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828414/real 1787828414] req@ffff985610f82940 x1874669228312832/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828439 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828414.197582:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82940 x1874669228312832/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828439 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828414.197592:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828414.197596:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828414.197600:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228312832:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787828414.197610:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228312960:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828414.197637:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828414.197652:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828414/real 1787828414] req@ffff985610f82580 x1874669228312960/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828439 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828414.197677:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82580 x1874669228312960/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828439 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828414.197688:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828414.197734:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828414.197739:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228312960:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828414.260918:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828415.156783:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787828415.284854:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828416.180777:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828416.308822:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828417.204804:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828417.332797:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828418.228824:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828418.356938:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828419.252893:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787828419.316838:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828419.316858:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828419.316865:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828419.316884:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4564 00000100:00080000:3.0:1787828419.316904:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828419.316921:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828419.316996:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f821c0 x1874669228313088/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828419.317041:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828419.317054:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828419.317074:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828419.317082:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4564 00000100:00080000:3.0:1787828419.317090:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828419.317099:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828419.317116:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828419.317174:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f821c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228313088:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787828419.317184:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83480 x1874669228313216/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828419.317205:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828419.317210:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828419.317213:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828419.317220:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4564 00000100:00080000:3.0:1787828419.317255:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828419.317262:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000800:00000100:2.0:1787828419.317269:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000008:00100000:3.0:1787828419.317273:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828419.317297:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82580 x1874669228313344/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828419.317317:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828419/real 1787828419] req@ffff985610f821c0 x1874669228313088/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828444 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828419.317348:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f821c0 x1874669228313088/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828444 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828419.317360:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828419.317365:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828419.317372:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f821c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228313088:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787828419.317401:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228313216:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828419.317423:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828419.317435:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828419/real 1787828419] req@ffff985610f83480 x1874669228313216/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828444 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828419.317447:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83480 x1874669228313216/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828444 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828419.317462:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828419.317466:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828419.317469:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228313216:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787828419.317474:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228313344:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828419.317489:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828419.317498:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828419/real 1787828419] req@ffff985610f82580 x1874669228313344/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828444 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828419.317508:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82580 x1874669228313344/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828444 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828419.317516:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828419.317519:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828419.317521:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228313344:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828419.381041:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828420.276843:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787828420.405052:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828421.300816:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828421.428843:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828422.324854:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828422.452832:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828423.348871:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828423.476904:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828424.372789:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787828424.436816:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828424.436838:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828424.436843:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828424.436856:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4570 00000100:00080000:0.0:1787828424.436871:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828424.436881:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787828424.436921:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4000 x1874669228313472/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828424.436950:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828424.436953:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828424.436954:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828424.436958:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4570 00000100:00080000:0.0:1787828424.436961:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828424.436963:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828424.436972:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828424.436983:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7c00 x1874669228313600/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828424.436991:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828424.436994:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828424.436995:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828424.436998:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4570 00000100:00080000:0.0:1787828424.437001:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00100000:2.0:1787828424.437006:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228313472:192.168.203.114@tcp:250:kworker.0 00000100:00080000:0.0:1787828424.437007:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828424.437014:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828424.437024:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4780 x1874669228313728/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828424.437099:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828424.437163:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828424/real 1787828424] req@ffff985607df4000 x1874669228313472/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828449 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828424.437185:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4000 x1874669228313472/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828449 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828424.437198:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828424.437203:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828424.437208:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228313472:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787828424.437217:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228313600:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828424.437246:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828424.437259:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828424/real 1787828424] req@ffff985607df7c00 x1874669228313600/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828449 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828424.437272:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7c00 x1874669228313600/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828449 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828424.437281:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828424.437284:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828424.437299:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228313600:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787828424.437330:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228313728:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828424.437349:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828424.437365:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828424/real 1787828424] req@ffff985607df4780 x1874669228313728/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828449 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828424.437383:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4780 x1874669228313728/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828449 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828424.437412:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828424.437416:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828424.437420:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228313728:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828424.500892:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828425.396793:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787828425.524928:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828426.420807:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828426.548834:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828427.444814:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828427.572925:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828428.468802:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828428.596902:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828429.492834:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787828429.556970:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828429.556989:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828429.556996:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828429.557015:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4575 00000100:00080000:3.0:1787828429.557033:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828429.557047:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828429.557138:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82940 x1874669228313856/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828429.557189:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828429.557197:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828429.557200:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828429.557221:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4575 00000100:00080000:3.0:1787828429.557231:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828429.557241:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828429.557264:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828429.557285:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228313856:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787828429.557300:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82d00 x1874669228313984/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828429.557325:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828429.557332:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828429.557337:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828429.557346:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4575 00000100:00080000:3.0:1787828429.557373:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828429.557380:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000800:00000100:2.0:1787828429.557388:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000008:00100000:3.0:1787828429.557390:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828429.557414:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80f00 x1874669228314112/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828429.557465:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828429/real 1787828429] req@ffff985610f82940 x1874669228313856/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828454 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828429.557492:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82940 x1874669228313856/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828454 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828429.557508:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828429.557513:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828429.557518:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228313856:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787828429.557548:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228313984:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828429.557579:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828429.557595:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828429/real 1787828429] req@ffff985610f82d00 x1874669228313984/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828454 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828429.557611:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82d00 x1874669228313984/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828454 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828429.557645:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828429.557649:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828429.557653:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228313984:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787828429.557661:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228314112:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828429.557686:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828429.557762:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828429/real 1787828429] req@ffff985610f80f00 x1874669228314112/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828454 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828429.557779:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80f00 x1874669228314112/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828454 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828429.557798:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828429.557803:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828429.557807:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228314112:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828429.620904:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828430.516802:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787828430.644951:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828431.540814:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828431.668880:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828432.564803:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828432.692869:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828433.588812:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828433.716929:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828434.612837:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787828434.676809:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828434.676824:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828434.676830:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828434.676844:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4580 00000100:00080000:1.0:1787828434.676861:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828434.676870:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787828434.676915:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a30c0 x1874669228314240/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828434.676945:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828434.676950:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828434.676952:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828434.676958:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4580 00000100:00080000:1.0:1787828434.676963:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828434.676970:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787828434.676983:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828434.677007:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228314240:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787828434.677013:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2940 x1874669228314368/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828434.677037:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828434.677040:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828434.677042:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828434.677046:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4580 00000100:00080000:1.0:1787828434.677050:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828434.677053:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787828434.677058:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828434.677072:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a1a40 x1874669228314496/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828434.677115:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828434.677155:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828434/real 1787828434] req@ffff9856085a30c0 x1874669228314240/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828459 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828434.677172:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a30c0 x1874669228314240/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828459 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828434.677182:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828434.677185:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828434.677189:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228314240:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787828434.677213:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228314368:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828434.677233:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828434.677242:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828434/real 1787828434] req@ffff9856085a2940 x1874669228314368/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828459 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828434.677251:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2940 x1874669228314368/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828459 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828434.677258:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828434.677260:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828434.677263:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228314368:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787828434.677268:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228314496:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828434.677281:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828434.677288:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828434/real 1787828434] req@ffff9856085a1a40 x1874669228314496/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828459 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828434.677297:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a1a40 x1874669228314496/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828459 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828434.677303:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828434.677305:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828434.677307:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228314496:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828434.740866:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828435.636791:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787828435.764911:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828436.660875:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828436.788839:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828437.684805:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828437.812867:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828438.708817:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828438.836865:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828439.732839:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787828439.796865:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828439.796885:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828439.796892:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828439.796918:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4585 00000100:00080000:1.0:1787828439.796936:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828439.796975:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787828439.797035:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a12c0 x1874669228314624/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828439.797075:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828439.797080:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828439.797083:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828439.797088:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4585 00000100:00080000:1.0:1787828439.797094:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828439.797101:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787828439.797114:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828439.797133:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228314624:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787828439.797134:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2580 x1874669228314752/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828439.797154:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828439.797157:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828439.797160:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828439.797164:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4585 00000100:00080000:1.0:1787828439.797168:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828439.797171:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787828439.797176:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828439.797188:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3840 x1874669228314880/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828439.797233:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828439.797282:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828439/real 1787828439] req@ffff9856085a12c0 x1874669228314624/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828464 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828439.797310:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a12c0 x1874669228314624/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828464 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828439.797333:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828439.797337:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828439.797341:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228314624:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787828439.797368:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228314752:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828439.797393:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828439.797405:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828439/real 1787828439] req@ffff9856085a2580 x1874669228314752/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828464 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828439.797417:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2580 x1874669228314752/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828464 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828439.797425:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828439.797428:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828439.797430:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228314752:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787828439.797435:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228314880:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828439.797449:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828439.797456:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828439/real 1787828439] req@ffff9856085a3840 x1874669228314880/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828464 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828439.797466:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3840 x1874669228314880/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828464 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828439.797472:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828439.797475:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828439.797477:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228314880:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828439.860941:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828440.756813:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787828440.885030:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828441.780854:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828441.908850:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828442.804844:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828442.932878:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828443.828868:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828443.956849:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828444.852916:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787828444.916830:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828444.916842:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828444.916847:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828444.916859:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4590 00000100:00080000:1.0:1787828444.916891:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828444.916899:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787828444.916988:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a03c0 x1874669228315008/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828444.917016:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828444.917020:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828444.917022:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828444.917025:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4590 00000100:00080000:1.0:1787828444.917029:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828444.917032:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787828444.917060:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828444.917079:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0780 x1874669228315136/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828444.917093:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1787828444.917117:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228315008:192.168.203.114@tcp:250:kworker.0 00000100:00080000:1.0:1787828444.917120:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828444.917129:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828444.917145:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4590 00000100:00080000:1.0:1787828444.917150:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828444.917154:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787828444.917159:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828444.917195:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a1680 x1874669228315264/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828444.917263:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828444.917321:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828444/real 1787828444] req@ffff9856085a03c0 x1874669228315008/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828469 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828444.917349:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a03c0 x1874669228315008/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828469 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828444.917373:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828444.917379:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828444.917385:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228315008:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787828444.917423:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228315136:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828444.917457:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828444.917479:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828444/real 1787828444] req@ffff9856085a0780 x1874669228315136/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828469 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828444.917497:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0780 x1874669228315136/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828469 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828444.917510:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828444.917514:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828444.917519:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228315136:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787828444.917529:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228315264:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828444.917556:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828444.917578:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828444/real 1787828444] req@ffff9856085a1680 x1874669228315264/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828469 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828444.917605:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a1680 x1874669228315264/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828469 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828444.917614:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828444.917619:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828444.917624:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228315264:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828444.980988:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828445.876826:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787828446.004949:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828446.900803:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828447.028903:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828447.924805:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828448.052843:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828448.949115:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828449.076832:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828449.972779:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787828450.036824:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828450.036835:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828450.036839:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828450.036849:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4595 00000100:00080000:3.0:1787828450.036860:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828450.036868:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828450.036906:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81e00 x1874669228315392/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828450.036962:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828450.036964:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828450.036966:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828450.036969:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4595 00000100:00080000:3.0:1787828450.036972:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828450.036975:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828450.036982:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828450.036993:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f803c0 x1874669228315520/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828450.037005:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828450.037008:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828450.037010:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828450.037012:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4595 00000100:00080000:3.0:1787828450.037014:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828450.037017:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828450.037019:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828450.037028:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228315392:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787828450.037029:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81680 x1874669228315648/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828450.037110:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828450.037152:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828450/real 1787828450] req@ffff985610f81e00 x1874669228315392/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828475 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828450.037167:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81e00 x1874669228315392/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828475 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828450.037176:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828450.037179:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828450.037183:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228315392:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787828450.037188:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228315520:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828450.037205:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828450.037213:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828450/real 1787828450] req@ffff985610f803c0 x1874669228315520/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828475 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828450.037221:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f803c0 x1874669228315520/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828475 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828450.037226:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828450.037228:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828450.037230:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228315520:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787828450.037251:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228315648:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828450.037262:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828450.037268:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828450/real 1787828450] req@ffff985610f81680 x1874669228315648/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828475 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828450.037276:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81680 x1874669228315648/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828475 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828450.037282:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828450.037284:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828450.037286:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228315648:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828450.100834:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828450.996811:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787828451.124893:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828452.020781:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828452.148953:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828453.044865:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828453.172848:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828454.068908:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828454.196820:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828455.092772:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787828455.156860:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828455.156901:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828455.156909:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828455.156941:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4600 00000100:00080000:3.0:1787828455.156960:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828455.156973:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828455.157029:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83840 x1874669228315776/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828455.157088:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828455.157096:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828455.157098:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828455.157105:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4600 00000100:00080000:3.0:1787828455.157111:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828455.157118:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828455.157133:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828455.157166:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81a40 x1874669228315904/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828455.157169:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228315776:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787828455.157198:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828455.157203:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828455.157206:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828455.157213:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4600 00000100:00080000:3.0:1787828455.157219:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828455.157225:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828455.157233:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828455.157252:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f812c0 x1874669228316032/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828455.157286:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828455.157332:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828455/real 1787828455] req@ffff985610f83840 x1874669228315776/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828480 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828455.157372:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83840 x1874669228315776/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828480 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828455.157386:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828455.157391:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828455.157396:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228315776:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787828455.157429:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228315904:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828455.157490:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828455.157504:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828455/real 1787828455] req@ffff985610f81a40 x1874669228315904/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828480 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828455.157518:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81a40 x1874669228315904/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828480 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828455.157534:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828455.157538:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828455.157541:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228315904:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787828455.157547:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228316032:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828455.157564:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828455.157574:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828455/real 1787828455] req@ffff985610f812c0 x1874669228316032/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828480 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828455.157611:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f812c0 x1874669228316032/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828480 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828455.157620:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828455.157624:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828455.157627:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228316032:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828455.220992:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828456.116932:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787828456.244928:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828457.140816:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828457.268926:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828458.164826:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828458.292893:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828459.188800:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828459.316900:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828460.212854:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787828460.276774:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828460.276787:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828460.276793:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828460.276806:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4605 00000100:00080000:3.0:1787828460.276820:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828460.276830:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828460.276900:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80b40 x1874669228316160/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828460.276933:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828460.276936:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828460.276938:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828460.276941:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4605 00000100:00080000:3.0:1787828460.276944:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828460.276948:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828460.276959:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828460.276972:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80000 x1874669228316288/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828460.276983:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828460.276987:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828460.276989:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828460.276995:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4605 00000100:00100000:2.0:1787828460.276997:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228316160:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787828460.276999:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828460.277008:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828460.277014:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828460.277025:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f830c0 x1874669228316416/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828460.277095:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828460.277146:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828460/real 1787828460] req@ffff985610f80b40 x1874669228316160/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828485 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828460.277167:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80b40 x1874669228316160/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828485 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828460.277178:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828460.277182:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828460.277188:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228316160:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787828460.277217:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228316288:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828460.277240:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828460.277257:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828460/real 1787828460] req@ffff985610f80000 x1874669228316288/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828485 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828460.277277:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80000 x1874669228316288/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828485 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828460.277289:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828460.277294:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828460.277299:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228316288:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787828460.277309:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228316416:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828460.277347:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828460.277360:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828460/real 1787828460] req@ffff985610f830c0 x1874669228316416/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828485 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828460.277372:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f830c0 x1874669228316416/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828485 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828460.277380:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828460.277383:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828460.277386:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228316416:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828460.340974:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828461.236811:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787828461.364951:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828462.260770:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828462.388814:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828463.284792:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828463.412941:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828464.308836:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828464.436861:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828465.332851:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787828465.396797:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828465.396822:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828465.396827:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828465.396838:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4611 00000100:00080000:3.0:1787828465.396868:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828465.396876:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828465.396917:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80780 x1874669228316544/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828465.396943:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828465.396946:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828465.396948:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828465.396951:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4611 00000100:00080000:3.0:1787828465.396954:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828465.396956:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828465.396965:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828465.396975:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83c00 x1874669228316672/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828465.396983:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828465.396987:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828465.396989:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828465.396992:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4611 00000100:00080000:3.0:1787828465.396994:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828465.396998:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828465.397001:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828465.397012:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f830c0 x1874669228316800/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828465.397023:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228316544:192.168.203.114@tcp:250:kworker.0 00000800:00000100:2.0:1787828465.397119:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828465.397160:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828465/real 1787828465] req@ffff985610f80780 x1874669228316544/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828490 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828465.397179:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80780 x1874669228316544/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828490 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828465.397192:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828465.397197:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828465.397201:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228316544:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787828465.397209:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228316672:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828465.397230:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828465.397242:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828465/real 1787828465] req@ffff985610f83c00 x1874669228316672/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828490 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828465.397254:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83c00 x1874669228316672/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828490 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828465.397262:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828465.397265:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828465.397268:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228316672:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787828465.397308:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228316800:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828465.397324:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828465.397334:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828465/real 1787828465] req@ffff985610f830c0 x1874669228316800/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828490 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828465.397346:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f830c0 x1874669228316800/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828490 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828465.397370:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828465.397373:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828465.397376:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228316800:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828465.461016:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828466.356897:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787828466.484932:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828467.380776:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828467.508848:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828468.404845:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828468.532886:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828469.428795:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828469.556830:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828470.452884:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787828470.516877:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828470.516888:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828470.516893:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828470.516907:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4616 00000100:00080000:3.0:1787828470.516923:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828470.516934:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828470.516981:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80000 x1874669228316928/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828470.517021:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828470.517026:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828470.517029:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828470.517035:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4616 00000100:00080000:3.0:1787828470.517042:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828470.517051:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828470.517068:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828470.517095:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80b40 x1874669228317056/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828470.517134:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228316928:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787828470.517143:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828470.517152:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828470.517156:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828470.517168:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4616 00000100:00080000:3.0:1787828470.517179:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828470.517188:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828470.517199:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828470.517226:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f812c0 x1874669228317184/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828470.517247:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828470.517340:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828470/real 1787828470] req@ffff985610f80000 x1874669228316928/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828495 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828470.517364:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80000 x1874669228316928/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828495 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828470.517387:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828470.517393:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828470.517398:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228316928:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787828470.517444:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228317056:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828470.517466:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828470.517477:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828470/real 1787828470] req@ffff985610f80b40 x1874669228317056/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828495 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828470.517488:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80b40 x1874669228317056/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828495 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828470.517495:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828470.517498:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828470.517501:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228317056:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787828470.517507:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228317184:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828470.517521:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828470.517528:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828470/real 1787828470] req@ffff985610f812c0 x1874669228317184/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828495 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828470.517538:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f812c0 x1874669228317184/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828495 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828470.517544:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828470.517547:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828470.517549:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228317184:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828470.580973:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828471.476891:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787828471.605023:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828472.500815:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828472.628869:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828473.524846:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828473.652858:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828474.548800:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828474.676912:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828475.572868:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787828475.636799:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828475.636810:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828475.636815:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828475.636825:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4621 00000100:00080000:3.0:1787828475.636838:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828475.636846:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828475.636890:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81a40 x1874669228317312/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828475.636924:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828475.636928:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828475.636930:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828475.636933:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4621 00000100:00080000:3.0:1787828475.636936:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828475.636938:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828475.636948:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828475.636958:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83840 x1874669228317440/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828475.636969:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828475.636972:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828475.636974:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828475.636977:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4621 00000100:00080000:3.0:1787828475.636979:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828475.636982:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828475.636986:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828475.636993:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81680 x1874669228317568/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828475.637003:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228317312:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787828475.637123:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228317440:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787828475.637160:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228317568:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828475.700869:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828476.596839:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787828476.724928:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828477.620856:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828477.748915:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828478.644827:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000400:00000100:1.0:1787828478.709111:0:2370:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.114:988 00000400:00000100:1.0:1787828478.709171:0:2370:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.114@tcp at host 192.168.203.114 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1787828478.709191:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:1.0:1787828478.709240:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:1.0:1787828478.709250:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000100:00100000:2.0:1787828478.709294:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828475/real 1787828478] req@ffff985610f81680 x1874669228317568/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828500 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828478.709330:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81680 x1874669228317568/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828500 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828478.709346:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828478.709354:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828478.709361:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228317568:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787828478.709373:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828475/real 1787828478] req@ffff985610f81a40 x1874669228317312/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828500 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828478.709392:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81a40 x1874669228317312/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828500 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828478.709404:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828478.709408:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828478.709413:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228317312:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787828478.709447:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828475/real 1787828478] req@ffff985610f83840 x1874669228317440/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828500 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828478.709464:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83840 x1874669228317440/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828500 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828478.709475:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828478.709480:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828478.709484:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228317440:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787828478.772845:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828479.668844:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828479.796903:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828480.692803:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787828480.756979:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828480.756990:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828480.757022:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828480.757034:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4626 00000100:00080000:1.0:1787828480.757047:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828480.757056:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787828480.757143:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3c00 x1874669228317696/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828480.757181:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828480.757186:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828480.757188:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828480.757192:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4626 00000100:00080000:1.0:1787828480.757195:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828480.757198:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787828480.757208:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828480.757220:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0b40 x1874669228317824/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828480.757229:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828480.757233:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828480.757235:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828480.757238:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4626 00000100:00080000:1.0:1787828480.757240:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828480.757243:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787828480.757247:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828480.757260:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0000 x1874669228317952/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828480.757260:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228317696:192.168.203.114@tcp:250:kworker.0 00000800:00000100:2.0:1787828480.757402:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828480.757457:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828480/real 1787828480] req@ffff9856085a3c00 x1874669228317696/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828505 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828480.757488:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3c00 x1874669228317696/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828505 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828480.757506:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828480.757512:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828480.757520:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228317696:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787828480.757533:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228317824:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828480.757589:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828480.757609:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828480/real 1787828480] req@ffff9856085a0b40 x1874669228317824/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828505 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828480.757634:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0b40 x1874669228317824/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828505 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828480.757646:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828480.757649:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828480.757654:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228317824:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787828480.757744:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228317952:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828480.757777:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828480.757795:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828480/real 1787828480] req@ffff9856085a0000 x1874669228317952/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828505 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828480.757808:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0000 x1874669228317952/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828505 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828480.757822:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828480.757825:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828480.757829:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228317952:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828480.820886:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828481.716867:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787828481.844919:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828482.740800:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828482.868914:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828483.764870:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828483.892898:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828484.788835:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828484.916926:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828485.812804:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787828485.876844:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828485.876864:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828485.876872:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828485.876890:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4631 00000100:00080000:1.0:1787828485.876909:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828485.876924:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787828485.877001:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2d00 x1874669228318080/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828485.877048:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828485.877053:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828485.877055:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828485.877061:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4631 00000100:00080000:1.0:1787828485.877066:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828485.877072:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787828485.877086:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828485.877106:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0000 x1874669228318208/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828485.877121:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828485.877129:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828485.877134:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828485.877143:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4631 00000100:00080000:1.0:1787828485.877152:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828485.877160:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787828485.877171:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828485.877180:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228318080:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787828485.877204:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0b40 x1874669228318336/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828485.877316:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828485.877381:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828485/real 1787828485] req@ffff9856085a2d00 x1874669228318080/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828510 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828485.877409:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2d00 x1874669228318080/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828510 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828485.877423:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828485.877429:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828485.877434:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228318080:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787828485.877445:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228318208:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828485.877479:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828485.877495:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828485/real 1787828485] req@ffff9856085a0000 x1874669228318208/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828510 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828485.877510:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0000 x1874669228318208/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828510 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828485.877522:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828485.877526:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828485.877531:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228318208:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787828485.877563:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228318336:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828485.877584:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828485.877596:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828485/real 1787828485] req@ffff9856085a0b40 x1874669228318336/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828510 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828485.877610:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0b40 x1874669228318336/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828510 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828485.877620:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828485.877624:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828485.877628:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228318336:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828485.940920:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828486.836835:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787828486.964887:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828487.860841:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828487.988821:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828488.884857:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828489.012922:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828489.908802:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828490.036829:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828490.932858:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787828490.996797:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828490.996807:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828490.996813:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828490.996823:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4636 00000100:00080000:3.0:1787828490.996834:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828490.996841:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828490.996881:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f803c0 x1874669228318464/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828490.996908:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828490.996912:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828490.996913:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828490.996917:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4636 00000100:00080000:3.0:1787828490.996920:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828490.996923:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828490.996933:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828490.996944:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81e00 x1874669228318592/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828490.996953:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828490.996957:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828490.996959:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828490.996962:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4636 00000100:00080000:3.0:1787828490.996965:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828490.996969:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828490.996973:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828490.996984:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80f00 x1874669228318720/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828490.996995:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228318464:192.168.203.114@tcp:250:kworker.0 00000800:00000100:2.0:1787828490.997104:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828490.997150:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828490/real 1787828490] req@ffff985610f803c0 x1874669228318464/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828515 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828490.997167:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f803c0 x1874669228318464/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828515 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828490.997178:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828490.997182:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828490.997186:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228318464:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787828490.997192:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228318592:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828490.997210:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828490.997220:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828490/real 1787828490] req@ffff985610f81e00 x1874669228318592/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828515 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828490.997229:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81e00 x1874669228318592/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828515 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828490.997235:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828490.997238:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828490.997240:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228318592:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787828490.997265:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228318720:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828490.997279:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828490.997286:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828490/real 1787828490] req@ffff985610f80f00 x1874669228318720/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828515 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828490.997295:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80f00 x1874669228318720/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828515 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828490.997301:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828490.997303:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828490.997305:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228318720:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828491.060945:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828491.956758:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787828492.084898:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828492.980808:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828493.108852:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828494.004838:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828494.132887:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828495.028808:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828495.156820:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828496.052851:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787828496.116838:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828496.116847:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828496.116851:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828496.116862:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4641 00000100:00080000:0.0:1787828496.116873:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828496.116882:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787828496.116921:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6940 x1874669228318848/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828496.116945:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828496.116948:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828496.116950:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828496.116953:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4641 00000100:00080000:0.0:1787828496.116955:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828496.116957:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828496.116966:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828496.116976:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4b40 x1874669228318976/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828496.116983:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828496.116988:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:2.0:1787828496.116990:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228318848:192.168.203.114@tcp:250:kworker.0 00000100:00080000:0.0:1787828496.116991:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828496.117000:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4641 00000100:00080000:0.0:1787828496.117003:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828496.117006:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828496.117009:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828496.117017:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7480 x1874669228319104/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828496.117079:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828496.117115:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828496/real 1787828496] req@ffff985607df6940 x1874669228318848/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828521 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828496.117130:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6940 x1874669228318848/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828521 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828496.117139:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828496.117142:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828496.117145:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228318848:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787828496.117168:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228318976:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828496.117183:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828496.117191:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828496/real 1787828496] req@ffff985607df4b40 x1874669228318976/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828521 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828496.117200:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4b40 x1874669228318976/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828521 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828496.117205:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828496.117208:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828496.117210:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228318976:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787828496.117215:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228319104:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828496.117225:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828496.117242:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828496/real 1787828496] req@ffff985607df7480 x1874669228319104/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828521 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828496.117249:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7480 x1874669228319104/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828521 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828496.117254:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828496.117256:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828496.117258:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228319104:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828496.180980:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828497.076845:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787828497.204987:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828498.100817:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828498.228892:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828499.124779:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828499.252846:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828500.148788:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828500.276832:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828501.172807:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787828501.236760:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828501.236771:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828501.236774:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828501.236785:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4646 00000100:00080000:0.0:1787828501.236796:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828501.236803:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787828501.236839:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df52c0 x1874669228319232/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828501.236865:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828501.236867:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828501.236869:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828501.236871:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4646 00000100:00080000:0.0:1787828501.236874:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828501.236876:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828501.236884:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828501.236895:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7840 x1874669228319360/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828501.236905:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828501.236908:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828501.236909:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828501.236915:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4646 00000100:00100000:2.0:1787828501.236915:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228319232:192.168.203.114@tcp:250:kworker.0 00000100:00080000:0.0:1787828501.236919:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828501.236925:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828501.236928:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828501.236937:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df61c0 x1874669228319488/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828501.236994:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828501.237030:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828501/real 1787828501] req@ffff985607df52c0 x1874669228319232/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828526 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828501.237046:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df52c0 x1874669228319232/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828526 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828501.237055:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828501.237059:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828501.237062:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228319232:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787828501.237084:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228319360:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828501.237111:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828501.237121:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828501/real 1787828501] req@ffff985607df7840 x1874669228319360/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828526 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828501.237131:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7840 x1874669228319360/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828526 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828501.237137:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828501.237139:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828501.237142:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228319360:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787828501.237146:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228319488:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828501.237159:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828501.237165:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828501/real 1787828501] req@ffff985607df61c0 x1874669228319488/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828526 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828501.237173:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df61c0 x1874669228319488/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828526 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828501.237178:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828501.237180:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828501.237182:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228319488:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828501.300824:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828502.196872:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787828502.324923:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828503.220849:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828503.348944:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828504.244814:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828504.372864:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828505.268799:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828505.396853:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828506.292795:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787828506.356777:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828506.356791:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828506.356796:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828506.356817:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4651 00000100:00080000:3.0:1787828506.356833:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828506.356842:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828506.356888:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82d00 x1874669228319616/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828506.356917:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828506.356922:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828506.356924:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828506.356929:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4651 00000100:00080000:3.0:1787828506.356933:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828506.356938:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828506.356950:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828506.356953:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228319616:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787828506.356966:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82940 x1874669228319744/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828506.356979:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828506.356983:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828506.356985:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828506.356990:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4651 00000100:00080000:3.0:1787828506.356994:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828506.356999:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828506.357006:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828506.357019:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82580 x1874669228319872/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828506.357039:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828506.357075:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828506/real 1787828506] req@ffff985610f82d00 x1874669228319616/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828531 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828506.357101:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82d00 x1874669228319616/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828531 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828506.357110:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828506.357114:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828506.357117:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228319616:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787828506.357141:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228319744:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828506.357157:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828506.357166:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828506/real 1787828506] req@ffff985610f82940 x1874669228319744/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828531 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828506.357174:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82940 x1874669228319744/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828531 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828506.357191:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828506.357192:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828506.357195:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228319744:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787828506.357198:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228319872:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828506.357209:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828506.357215:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828506/real 1787828506] req@ffff985610f82580 x1874669228319872/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828531 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828506.357223:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82580 x1874669228319872/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828531 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828506.357228:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828506.357230:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828506.357232:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228319872:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828506.420855:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828507.316817:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787828507.444919:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828508.340843:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828508.468822:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828509.364833:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828509.492874:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828510.388893:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828510.516845:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828511.412856:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787828511.476862:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828511.476874:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828511.476879:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828511.476893:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4657 00000100:00080000:1.0:1787828511.476924:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828511.476933:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787828511.476984:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3c00 x1874669228320000/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828511.477017:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828511.477022:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828511.477024:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828511.477028:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4657 00000100:00080000:1.0:1787828511.477031:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828511.477035:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787828511.477047:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828511.477059:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a1680 x1874669228320128/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828511.477070:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828511.477073:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828511.477075:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828511.477081:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4657 00000100:00080000:1.0:1787828511.477084:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828511.477089:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787828511.477094:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828511.477103:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0780 x1874669228320256/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828511.477131:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228320000:192.168.203.114@tcp:250:kworker.0 00000800:00000100:2.0:1787828511.477242:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828511.477299:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828511/real 1787828511] req@ffff9856085a3c00 x1874669228320000/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828536 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828511.477334:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3c00 x1874669228320000/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828536 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828511.477349:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828511.477354:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828511.477359:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228320000:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787828511.477369:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228320128:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828511.477396:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828511.477410:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828511/real 1787828511] req@ffff9856085a1680 x1874669228320128/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828536 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828511.477431:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a1680 x1874669228320128/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828536 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828511.477442:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828511.477445:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828511.477449:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228320128:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787828511.477524:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228320256:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828511.477546:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828511.477558:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828511/real 1787828511] req@ffff9856085a0780 x1874669228320256/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828536 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828511.477569:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0780 x1874669228320256/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828536 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828511.477577:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828511.477581:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828511.477583:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228320256:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828511.540982:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828512.436851:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787828512.564938:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828513.460864:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828513.588906:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828514.484839:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828514.612813:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828515.508839:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828515.636871:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828516.532880:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787828516.596870:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828516.596887:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828516.596892:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828516.596907:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4662 00000100:00080000:3.0:1787828516.596921:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828516.596933:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828516.596984:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83480 x1874669228320384/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828516.597031:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828516.597036:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828516.597039:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828516.597045:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4662 00000100:00080000:3.0:1787828516.597049:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828516.597053:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828516.597065:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828516.597083:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f821c0 x1874669228320512/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828516.597096:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828516.597100:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828516.597103:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1787828516.597110:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228320384:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787828516.597112:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4662 00000100:00080000:3.0:1787828516.597124:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828516.597129:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828516.597134:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828516.597152:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82580 x1874669228320640/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787828516.597243:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828516.597306:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828516/real 1787828516] req@ffff985610f83480 x1874669228320384/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828541 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828516.597329:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83480 x1874669228320384/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828541 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828516.597342:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828516.597346:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828516.597351:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228320384:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787828516.597380:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f821c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228320512:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828516.597408:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828516.597421:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828516/real 1787828516] req@ffff985610f821c0 x1874669228320512/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828541 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828516.597436:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f821c0 x1874669228320512/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828541 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828516.597481:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828516.597512:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828516.597517:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f821c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228320512:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787828516.597523:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228320640:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828516.597561:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828516.597583:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828516/real 1787828516] req@ffff985610f82580 x1874669228320640/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828541 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828516.597603:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82580 x1874669228320640/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828541 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828516.597623:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828516.597628:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828516.597634:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228320640:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828516.660916:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828517.556864:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787828517.684912:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828518.580808:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828518.708882:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828519.604794:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828519.732837:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828520.628821:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828520.756848:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828521.652824:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787828521.716767:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828521.716780:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828521.716786:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828521.716799:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4667 00000100:00080000:3.0:1787828521.716812:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828521.716820:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828521.716863:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82940 x1874669228320768/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828521.716904:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828521.716907:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828521.716909:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828521.716912:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4667 00000100:00080000:3.0:1787828521.716916:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828521.716919:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828521.716929:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828521.716945:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82d00 x1874669228320896/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787828521.716955:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228320768:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787828521.716962:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828521.716968:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828521.716971:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828521.716976:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4667 00000100:00080000:3.0:1787828521.716981:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828521.716984:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828521.716989:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828521.717000:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80f00 x1874669228321024/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787828521.717032:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828521.717067:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828521/real 1787828521] req@ffff985610f82940 x1874669228320768/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828546 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828521.717081:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82940 x1874669228320768/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828546 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828521.717090:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828521.717094:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828521.717097:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228320768:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787828521.717118:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228320896:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828521.717134:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828521.717142:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828521/real 1787828521] req@ffff985610f82d00 x1874669228320896/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828546 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828521.717150:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82d00 x1874669228320896/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828546 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828521.717156:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828521.717157:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828521.717160:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228320896:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787828521.717164:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228321024:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828521.717191:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828521.717201:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828521/real 1787828521] req@ffff985610f80f00 x1874669228321024/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828546 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828521.717213:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80f00 x1874669228321024/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828546 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828521.717221:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828521.717224:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828521.717227:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228321024:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828521.780820:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828522.676878:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787828522.804950:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828523.700874:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828523.828934:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828524.724783:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828524.852863:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828525.748836:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828525.876854:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828526.772789:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787828526.836862:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828526.836885:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828526.836889:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828526.836899:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4672 00000100:00080000:1.0:1787828526.836909:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828526.836915:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787828526.836954:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a03c0 x1874669228321152/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828526.836978:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828526.836981:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828526.836982:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828526.836985:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4672 00000100:00080000:1.0:1787828526.836987:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828526.836990:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787828526.836999:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828526.837007:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3840 x1874669228321280/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828526.837015:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828526.837019:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828526.837021:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:2.0:1787828526.837021:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228321152:192.168.203.114@tcp:250:kworker.0 00000100:00080000:1.0:1787828526.837029:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4672 00000100:00080000:1.0:1787828526.837032:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828526.837034:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787828526.837038:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828526.837046:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2580 x1874669228321408/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828526.837099:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828526.837132:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828526/real 1787828526] req@ffff9856085a03c0 x1874669228321152/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828551 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828526.837145:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a03c0 x1874669228321152/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828551 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828526.837154:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828526.837175:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828526.837181:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228321152:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787828526.837214:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228321280:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828526.837247:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828526.837263:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828526/real 1787828526] req@ffff9856085a3840 x1874669228321280/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828551 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828526.837279:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3840 x1874669228321280/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828551 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828526.837290:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828526.837295:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828526.837299:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228321280:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787828526.837307:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228321408:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828526.837327:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828526.837339:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828526/real 1787828526] req@ffff9856085a2580 x1874669228321408/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828551 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828526.837353:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2580 x1874669228321408/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828551 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828526.837362:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828526.837365:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828526.837369:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228321408:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828526.900901:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828527.796832:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787828527.924963:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828528.820832:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828528.948876:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828529.844907:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828529.972939:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828530.868795:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828530.996907:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828531.892875:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787828531.956853:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828531.956867:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828531.956873:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828531.956889:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4677 00000100:00080000:1.0:1787828531.956905:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828531.956917:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787828531.957000:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a12c0 x1874669228321536/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828531.957030:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828531.957033:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828531.957035:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828531.957039:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4677 00000100:00080000:1.0:1787828531.957042:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828531.957045:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787828531.957056:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828531.957078:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a1a40 x1874669228321664/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828531.957081:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228321536:192.168.203.114@tcp:250:kworker.0 00000100:00080000:1.0:1787828531.957120:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828531.957129:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828531.957135:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828531.957145:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4677 00000100:00080000:1.0:1787828531.957155:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828531.957164:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787828531.957174:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828531.957193:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2940 x1874669228321792/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828531.957226:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828531.957292:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828531/real 1787828531] req@ffff9856085a12c0 x1874669228321536/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828556 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828531.957329:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a12c0 x1874669228321536/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828556 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828531.957389:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828531.957396:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828531.957402:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228321536:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787828531.957456:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228321664:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828531.957498:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828531.957518:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828531/real 1787828531] req@ffff9856085a1a40 x1874669228321664/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828556 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828531.957535:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a1a40 x1874669228321664/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828556 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828531.957546:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828531.957551:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828531.957556:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228321664:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787828531.957564:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228321792:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828531.957585:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828531.957597:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828531/real 1787828531] req@ffff9856085a2940 x1874669228321792/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828556 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828531.957617:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2940 x1874669228321792/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828556 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828531.957633:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828531.957640:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828531.957647:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228321792:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828532.020931:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828532.916802:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787828533.044958:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828533.940772:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828534.068900:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828534.964830:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828535.092982:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828535.988790:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828536.116827:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828537.012840:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787828537.076811:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828537.076824:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828537.076829:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828537.076844:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4682 00000100:00080000:1.0:1787828537.076859:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828537.076869:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787828537.076943:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a30c0 x1874669228321920/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828537.076997:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828537.077002:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828537.077005:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828537.077010:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4682 00000100:00080000:1.0:1787828537.077014:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828537.077019:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787828537.077045:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828537.077069:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a21c0 x1874669228322048/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828537.077072:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228321920:192.168.203.114@tcp:250:kworker.0 00000100:00080000:1.0:1787828537.077100:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828537.077108:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828537.077112:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828537.077121:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4682 00000100:00080000:1.0:1787828537.077127:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828537.077135:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787828537.077146:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828537.077170:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3480 x1874669228322176/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828537.077210:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828537.077269:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828537/real 1787828537] req@ffff9856085a30c0 x1874669228321920/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828562 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828537.077295:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a30c0 x1874669228321920/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828562 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828537.077336:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828537.077342:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828537.077347:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228321920:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787828537.077381:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228322048:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828537.077412:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828537.077428:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828537/real 1787828537] req@ffff9856085a21c0 x1874669228322048/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828562 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828537.077449:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a21c0 x1874669228322048/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828562 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828537.077461:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828537.077465:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828537.077469:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228322048:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787828537.077476:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228322176:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828537.077496:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828537.077507:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828537/real 1787828537] req@ffff9856085a3480 x1874669228322176/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828562 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828537.077522:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3480 x1874669228322176/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828562 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828537.077532:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828537.077536:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828537.077539:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228322176:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828537.140952:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828538.036819:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787828538.164913:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828539.060781:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828539.188864:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828540.084776:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828540.212922:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828541.108829:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828541.236927:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828542.132770:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787828542.196893:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828542.196908:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828542.196914:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828542.196928:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4687 00000100:00080000:0.0:1787828542.196945:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828542.196956:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787828542.197024:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4f00 x1874669228322304/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828542.197062:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828542.197068:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828542.197070:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828542.197076:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4687 00000100:00080000:0.0:1787828542.197081:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828542.197086:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828542.197098:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828542.197116:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5e00 x1874669228322432/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828542.197131:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828542.197137:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828542.197140:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828542.197145:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4687 00000100:00080000:0.0:1787828542.197150:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828542.197154:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828542.197163:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828542.197165:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228322304:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787828542.197183:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6d00 x1874669228322560/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828542.197339:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228322432:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787828542.197396:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228322560:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828542.260908:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828543.156843:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787828543.284950:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828544.180787:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828544.308829:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828545.204914:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000400:00000100:1.0:1787828545.269230:0:2371:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.114:988 00000400:00000100:1.0:1787828545.269330:0:2371:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.114@tcp at host 192.168.203.114 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1787828545.269348:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:1.0:1787828545.269403:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:1.0:1787828545.269418:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000100:00100000:3.0:1787828545.269452:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828542/real 1787828545] req@ffff985607df6d00 x1874669228322560/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828567 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828545.269489:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6d00 x1874669228322560/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828567 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828545.269505:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828545.269513:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828545.269519:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228322560:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787828545.269530:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828542/real 1787828545] req@ffff985607df4f00 x1874669228322304/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828567 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828545.269547:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4f00 x1874669228322304/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828567 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828545.269556:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828545.269560:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828545.269564:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228322304:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787828545.269574:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828542/real 1787828545] req@ffff985607df5e00 x1874669228322432/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828567 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828545.269589:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5e00 x1874669228322432/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828567 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828545.269597:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828545.269601:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828545.269604:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228322432:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787828545.332891:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828546.228798:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828546.356937:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828547.252888:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787828547.316832:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828547.316846:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828547.316853:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828547.316870:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4692 00000100:00080000:0.0:1787828547.316888:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828547.316902:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787828547.316958:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df43c0 x1874669228322688/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828547.317025:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828547.317032:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828547.317036:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828547.317044:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4692 00000100:00080000:0.0:1787828547.317052:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828547.317061:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828547.317076:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828547.317095:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228322688:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787828547.317100:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5680 x1874669228322816/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828547.317120:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828547.317124:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828547.317126:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828547.317132:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4692 00000100:00080000:0.0:1787828547.317136:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828547.317151:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828547.317156:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828547.317170:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df70c0 x1874669228322944/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828547.317257:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828547.317342:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828547/real 1787828547] req@ffff985607df43c0 x1874669228322688/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828572 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828547.317378:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df43c0 x1874669228322688/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828572 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828547.317409:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828547.317416:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828547.317423:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228322688:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787828547.317486:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228322816:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787828547.317554:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828547.317580:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828547/real 1787828547] req@ffff985607df5680 x1874669228322816/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828572 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828547.317613:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5680 x1874669228322816/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828572 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828547.317632:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828547.317638:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828547.317645:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228322816:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787828547.317656:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228322944:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787828547.317731:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828547.317756:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828547/real 1787828547] req@ffff985607df70c0 x1874669228322944/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828572 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828547.317779:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df70c0 x1874669228322944/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828572 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828547.317794:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828547.317801:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828547.317807:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228322944:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828547.381089:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828548.276829:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787828548.404995:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828549.300836:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828549.428864:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828550.324800:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828550.452876:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828551.348845:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828551.476905:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828552.372873:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787828552.436812:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828552.436824:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828552.436830:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828552.436843:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4698 00000100:00080000:0.0:1787828552.436858:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828552.436868:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787828552.436933:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6580 x1874669228323072/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828552.436991:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828552.436998:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828552.437000:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828552.437005:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4698 00000100:00080000:0.0:1787828552.437009:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828552.437013:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828552.437026:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828552.437041:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5a40 x1874669228323200/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828552.437056:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828552.437060:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828552.437063:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1787828552.437072:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228323072:192.168.203.114@tcp:250:kworker.0 00000100:00080000:0.0:1787828552.437073:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4698 00000100:00080000:0.0:1787828552.437082:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828552.437092:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828552.437097:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828552.437113:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df70c0 x1874669228323328/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828552.437218:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828552.437310:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828552/real 1787828552] req@ffff985607df6580 x1874669228323072/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828577 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828552.437346:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6580 x1874669228323072/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828577 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828552.437368:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828552.437376:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828552.437385:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228323072:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787828552.437433:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228323200:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787828552.437488:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828552.437519:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828552/real 1787828552] req@ffff985607df5a40 x1874669228323200/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828577 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828552.437545:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5a40 x1874669228323200/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828577 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828552.437564:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828552.437572:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828552.437580:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228323200:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787828552.437596:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228323328:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787828552.437659:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828552.438071:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828552/real 1787828552] req@ffff985607df70c0 x1874669228323328/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828577 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828552.438178:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df70c0 x1874669228323328/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828577 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828552.438223:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828552.438231:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828552.438239:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228323328:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828552.500941:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828553.396884:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787828553.524858:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828554.420867:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828554.548969:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828555.444937:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828555.572923:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828556.468806:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828556.596895:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828557.492834:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787828557.556886:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828557.556897:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828557.556902:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828557.556913:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4703 00000100:00080000:0.0:1787828557.556926:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828557.556935:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787828557.556986:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5680 x1874669228323456/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828557.557016:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828557.557020:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828557.557022:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828557.557026:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4703 00000100:00080000:0.0:1787828557.557029:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828557.557032:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828557.557044:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828557.557079:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df43c0 x1874669228323584/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828557.557095:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1787828557.557102:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228323456:192.168.203.114@tcp:250:kworker.0 00000100:00080000:0.0:1787828557.557104:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828557.557110:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828557.557120:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4703 00000100:00080000:0.0:1787828557.557125:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828557.557131:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828557.557137:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828557.557150:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5e00 x1874669228323712/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828557.557214:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828557.557258:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828557/real 1787828557] req@ffff985607df5680 x1874669228323456/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828582 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828557.557278:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5680 x1874669228323456/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828582 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828557.557289:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828557.557293:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828557.557297:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228323456:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787828557.557321:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228323584:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787828557.557341:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828557.557363:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828557/real 1787828557] req@ffff985607df43c0 x1874669228323584/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828582 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828557.557375:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df43c0 x1874669228323584/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828582 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828557.557382:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828557.557385:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828557.557388:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228323584:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787828557.557392:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228323712:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787828557.557407:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828557.557415:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828557/real 1787828557] req@ffff985607df5e00 x1874669228323712/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828582 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828557.557425:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5e00 x1874669228323712/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828582 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828557.557444:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828557.557449:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828557.557453:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228323712:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828557.620903:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828558.516838:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787828558.644889:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828559.540860:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828559.668927:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828560.564853:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828560.692935:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828561.588888:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828561.716847:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828562.612813:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787828562.676878:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828562.676889:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828562.676893:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828562.676904:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4708 00000100:00080000:3.0:1787828562.676923:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828562.676931:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828562.676993:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81e00 x1874669228323840/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828562.677020:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828562.677022:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828562.677024:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828562.677027:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4708 00000100:00080000:3.0:1787828562.677030:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828562.677033:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828562.677041:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828562.677049:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f803c0 x1874669228323968/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828562.677059:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828562.677061:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828562.677063:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828562.677065:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4708 00000100:00080000:3.0:1787828562.677068:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828562.677070:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828562.677083:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828562.677089:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83840 x1874669228324096/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828562.677181:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228323840:192.168.203.114@tcp:250:kworker.0 00000800:00000100:2.0:1787828562.677309:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828562.677378:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828562/real 1787828562] req@ffff985610f81e00 x1874669228323840/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828587 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828562.677397:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81e00 x1874669228323840/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828587 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828562.677409:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828562.677414:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828562.677418:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228323840:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787828562.677426:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228323968:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828562.677447:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828562.677457:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828562/real 1787828562] req@ffff985610f803c0 x1874669228323968/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828587 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828562.677466:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f803c0 x1874669228323968/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828587 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828562.677472:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828562.677476:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828562.677480:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228323968:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787828562.677486:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228324096:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828562.677524:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828562.677534:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828562/real 1787828562] req@ffff985610f83840 x1874669228324096/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828587 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828562.677547:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83840 x1874669228324096/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828587 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828562.677556:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828562.677559:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828562.677563:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228324096:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828562.740891:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828563.636820:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787828563.764824:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828564.660848:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828564.788771:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828565.684869:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828565.812908:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828566.708867:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828566.836806:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828567.732793:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787828567.796810:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828567.796821:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828567.796826:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828567.796837:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4713 00000100:00080000:0.0:1787828567.796850:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828567.796857:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787828567.796905:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4f00 x1874669228324224/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828567.796959:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828567.796963:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828567.796966:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828567.796969:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4713 00000100:00080000:0.0:1787828567.796973:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828567.796976:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828567.796985:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828567.796998:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6d00 x1874669228324352/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828567.797029:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828567.797032:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828567.797034:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828567.797043:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4713 00000100:00100000:2.0:1787828567.797045:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228324224:192.168.203.114@tcp:250:kworker.0 00000100:00080000:0.0:1787828567.797047:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828567.797056:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828567.797065:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828567.797077:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df61c0 x1874669228324480/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828567.797173:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828567.797212:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828567/real 1787828567] req@ffff985607df4f00 x1874669228324224/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828592 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828567.797245:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4f00 x1874669228324224/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828592 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828567.797258:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828567.797263:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828567.797270:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228324224:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787828567.797297:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228324352:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828567.797346:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828567.797368:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828567/real 1787828567] req@ffff985607df6d00 x1874669228324352/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828592 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828567.797389:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6d00 x1874669228324352/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828592 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828567.797403:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828567.797408:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828567.797413:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228324352:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787828567.797424:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228324480:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828567.797457:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828567.797476:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828567/real 1787828567] req@ffff985607df61c0 x1874669228324480/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828592 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828567.797495:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df61c0 x1874669228324480/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828592 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828567.797508:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828567.797514:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828567.797535:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228324480:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828567.860858:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828568.756794:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787828568.884828:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828569.780828:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828569.908848:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828570.804772:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828570.932786:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828571.828840:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828571.956822:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828572.852840:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787828572.916880:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828572.916888:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828572.916892:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828572.916901:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4718 00000100:00080000:2.0:1787828572.916922:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828572.916929:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787828572.916970:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d9680 x1874669228324608/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828572.916999:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828572.917002:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828572.917003:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828572.917007:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4718 00000100:00080000:2.0:1787828572.917009:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828572.917012:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787828572.917021:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828572.917030:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8b40 x1874669228324736/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828572.917040:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828572.917043:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828572.917044:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828572.917047:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4718 00000100:00080000:2.0:1787828572.917049:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828572.917052:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787828572.917072:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828572.917092:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140da940 x1874669228324864/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828572.917123:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228324608:192.168.203.114@tcp:250:kworker.0 00000800:00000100:3.0:1787828572.917249:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828572.917314:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828572/real 1787828572] req@ffff9856140d9680 x1874669228324608/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828597 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828572.917337:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d9680 x1874669228324608/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828597 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828572.917352:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828572.917358:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828572.917364:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228324608:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787828572.917374:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228324736:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787828572.917436:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828572.917449:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828572/real 1787828572] req@ffff9856140d8b40 x1874669228324736/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828597 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828572.917459:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8b40 x1874669228324736/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828597 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828572.917472:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828572.917475:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828572.917478:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228324736:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787828572.917517:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228324864:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787828572.917550:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828572.917559:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828572/real 1787828572] req@ffff9856140da940 x1874669228324864/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828597 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828572.917567:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140da940 x1874669228324864/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828597 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828572.917572:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828572.917575:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828572.917577:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228324864:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828572.980894:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828573.876874:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787828574.005020:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828574.900852:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828575.028879:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828575.924832:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828576.052856:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828576.948790:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828577.076850:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828577.972947:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787828578.036851:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828578.036863:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828578.036869:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828578.036882:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4723 00000100:00080000:3.0:1787828578.036895:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828578.036904:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828578.036951:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81a40 x1874669228324992/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828578.036991:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828578.036996:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828578.036998:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828578.037002:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4723 00000100:00080000:3.0:1787828578.037006:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828578.037010:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828578.037041:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828578.037055:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228324992:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787828578.037058:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81680 x1874669228325120/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828578.037083:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828578.037089:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828578.037092:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828578.037098:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4723 00000100:00080000:3.0:1787828578.037101:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828578.037106:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828578.037112:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828578.037127:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f812c0 x1874669228325248/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828578.037147:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828578.037188:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828578/real 1787828578] req@ffff985610f81a40 x1874669228324992/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828603 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828578.037202:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81a40 x1874669228324992/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828603 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828578.037211:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828578.037215:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828578.037218:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228324992:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787828578.037241:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228325120:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828578.037256:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828578.037265:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828578/real 1787828578] req@ffff985610f81680 x1874669228325120/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828603 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828578.037273:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81680 x1874669228325120/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828603 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828578.037283:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828578.037286:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828578.037288:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228325120:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787828578.037292:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228325248:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828578.037304:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828578.037310:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828578/real 1787828578] req@ffff985610f812c0 x1874669228325248/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828603 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828578.037317:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f812c0 x1874669228325248/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828603 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828578.037322:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828578.037324:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828578.037326:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228325248:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828578.100872:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828578.996851:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787828579.124968:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828580.020903:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828580.148850:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828581.044798:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828581.172818:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828582.068869:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828582.196952:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828583.092794:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787828583.156819:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828583.156831:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828583.156835:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828583.156850:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4728 00000100:00080000:3.0:1787828583.156865:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828583.156888:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828583.156941:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80b40 x1874669228325376/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828583.156971:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828583.156975:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828583.156976:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828583.156980:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4728 00000100:00080000:3.0:1787828583.157035:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00100000:2.0:1787828583.157036:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228325376:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787828583.157043:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828583.157054:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828583.157068:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80000 x1874669228325504/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828583.157077:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828583.157080:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828583.157082:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828583.157084:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4728 00000100:00080000:3.0:1787828583.157087:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828583.157089:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828583.157093:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828583.157100:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f830c0 x1874669228325632/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828583.157138:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828583.157180:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828583/real 1787828583] req@ffff985610f80b40 x1874669228325376/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828608 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828583.157197:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80b40 x1874669228325376/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828608 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828583.157213:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828583.157217:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828583.157220:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228325376:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787828583.157246:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228325504:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828583.157263:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828583.157273:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828583/real 1787828583] req@ffff985610f80000 x1874669228325504/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828608 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828583.157281:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80000 x1874669228325504/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828608 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828583.157287:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828583.157290:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828583.157292:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228325504:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787828583.157295:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228325632:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828583.157306:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828583.157312:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828583/real 1787828583] req@ffff985610f830c0 x1874669228325632/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828608 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828583.157329:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f830c0 x1874669228325632/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828608 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828583.157335:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828583.157337:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828583.157338:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228325632:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828583.220951:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828584.116861:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787828584.244892:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828585.140805:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828585.268809:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828586.164799:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828586.292848:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828587.188764:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828587.316864:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828588.212817:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787828588.276783:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828588.276796:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828588.276800:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828588.276813:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4733 00000100:00080000:3.0:1787828588.276828:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828588.276837:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828588.276879:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83c00 x1874669228325760/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828588.276910:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828588.276915:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828588.276917:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828588.276922:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4733 00000100:00080000:3.0:1787828588.276926:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828588.276931:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828588.276943:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828588.276962:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228325760:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787828588.276974:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80780 x1874669228325888/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828588.276988:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828588.276992:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828588.276994:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828588.276999:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4733 00000100:00080000:3.0:1787828588.277004:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828588.277009:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828588.277015:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828588.277028:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f830c0 x1874669228326016/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787828588.277045:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828588.277085:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828588/real 1787828588] req@ffff985610f83c00 x1874669228325760/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828613 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828588.277105:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83c00 x1874669228325760/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828613 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828588.277117:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828588.277121:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828588.277126:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228325760:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787828588.277154:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228325888:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787828588.277180:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828588.277193:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828588/real 1787828588] req@ffff985610f80780 x1874669228325888/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828613 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828588.277206:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80780 x1874669228325888/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828613 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828588.277215:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828588.277219:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828588.277222:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228325888:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787828588.277229:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228326016:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787828588.277251:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828588.277263:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828588/real 1787828588] req@ffff985610f830c0 x1874669228326016/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828613 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828588.277275:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f830c0 x1874669228326016/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828613 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828588.277284:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828588.277287:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828588.277291:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228326016:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828588.340855:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828589.236791:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787828589.364935:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828590.260793:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828590.388860:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828591.284785:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828591.412828:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828592.308771:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828592.436783:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828593.332843:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787828593.396796:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828593.396806:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828593.396810:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828593.396820:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4739 00000100:00080000:3.0:1787828593.396831:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828593.396839:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828593.396878:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80000 x1874669228326144/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828593.396906:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828593.396909:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828593.396921:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828593.396925:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4739 00000100:00080000:3.0:1787828593.396928:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828593.396932:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828593.396943:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828593.396955:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80b40 x1874669228326272/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787828593.396963:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228326144:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787828593.396969:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828593.396976:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828593.396977:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828593.396981:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4739 00000100:00080000:3.0:1787828593.396984:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828593.396986:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828593.396990:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828593.397003:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f812c0 x1874669228326400/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787828593.397043:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828593.397074:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828593/real 1787828593] req@ffff985610f80000 x1874669228326144/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828618 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828593.397087:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80000 x1874669228326144/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828618 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828593.397104:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828593.397108:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828593.397111:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228326144:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787828593.397133:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228326272:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787828593.397150:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828593.397158:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828593/real 1787828593] req@ffff985610f80b40 x1874669228326272/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828618 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828593.397166:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80b40 x1874669228326272/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828618 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828593.397172:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828593.397175:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828593.397177:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228326272:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787828593.397181:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228326400:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787828593.397192:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828593.397198:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828593/real 1787828593] req@ffff985610f812c0 x1874669228326400/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828618 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828593.397206:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f812c0 x1874669228326400/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828618 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828593.397211:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828593.397213:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828593.397215:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228326400:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828593.460851:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828594.356869:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787828594.484842:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828595.380785:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828595.508853:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828596.404819:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828596.532817:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828597.428789:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828597.556853:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828598.452775:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787828598.516902:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828598.516913:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828598.516918:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828598.516930:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4744 00000100:00080000:0.0:1787828598.516943:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828598.516950:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787828598.516996:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7840 x1874669228326528/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828598.517034:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828598.517038:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828598.517040:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828598.517044:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4744 00000100:00080000:0.0:1787828598.517049:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828598.517053:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828598.517064:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828598.517080:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df52c0 x1874669228326656/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828598.517092:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828598.517098:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828598.517104:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:2.0:1787828598.517105:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228326528:192.168.203.114@tcp:250:kworker.0 00000100:00080000:0.0:1787828598.517118:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4744 00000100:00080000:0.0:1787828598.517123:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828598.517128:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828598.517135:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828598.517151:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7480 x1874669228326784/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828598.517194:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828598.517234:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828598/real 1787828598] req@ffff985607df7840 x1874669228326528/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828623 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828598.517250:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7840 x1874669228326528/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828623 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828598.517259:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828598.517262:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828598.517266:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228326528:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787828598.517289:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228326656:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828598.517305:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828598.517314:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828598/real 1787828598] req@ffff985607df52c0 x1874669228326656/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828623 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828598.517322:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df52c0 x1874669228326656/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828623 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828598.517328:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828598.517331:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828598.517333:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228326656:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787828598.517337:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228326784:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828598.517348:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828598.517354:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828598/real 1787828598] req@ffff985607df7480 x1874669228326784/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828623 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828598.517361:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7480 x1874669228326784/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828623 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828598.517366:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828598.517368:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828598.517370:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228326784:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828598.580857:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828599.476823:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787828599.604924:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828600.500776:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828600.628816:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828601.524796:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828601.652834:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828602.548776:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828602.676881:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828603.572790:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787828603.636805:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828603.636815:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828603.636820:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828603.636830:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4749 00000100:00080000:3.0:1787828603.636852:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828603.636861:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828603.636900:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81680 x1874669228326912/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828603.636925:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828603.636928:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828603.636930:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828603.636933:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4749 00000100:00080000:3.0:1787828603.636935:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828603.636938:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828603.636947:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828603.636956:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81a40 x1874669228327040/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828603.636964:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828603.636967:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828603.636968:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828603.636971:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4749 00000100:00080000:3.0:1787828603.636973:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828603.636975:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828603.636980:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828603.636983:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228326912:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787828603.636990:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83840 x1874669228327168/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828603.637055:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828603.637091:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828603/real 1787828603] req@ffff985610f81680 x1874669228326912/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828628 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828603.637106:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81680 x1874669228326912/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828628 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828603.637115:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828603.637118:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828603.637121:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228326912:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787828603.637127:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228327040:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828603.637142:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828603.637150:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828603/real 1787828603] req@ffff985610f81a40 x1874669228327040/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828628 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828603.637158:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81a40 x1874669228327040/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828628 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828603.637164:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828603.637165:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828603.637167:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228327040:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787828603.637189:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228327168:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828603.637212:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828603.637228:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828603/real 1787828603] req@ffff985610f83840 x1874669228327168/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828628 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828603.637240:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83840 x1874669228327168/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828628 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828603.637255:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828603.637258:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828603.637262:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228327168:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828603.700893:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828604.596778:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787828604.724913:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828605.620801:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828605.748852:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828606.644781:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828606.772809:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828607.668799:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828607.796840:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828608.692788:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787828608.756794:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828608.756819:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828608.756824:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828608.756835:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4754 00000100:00080000:3.0:1787828608.756846:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828608.756853:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828608.756892:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f803c0 x1874669228327296/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828608.756919:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828608.756922:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828608.756924:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828608.756926:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4754 00000100:00080000:3.0:1787828608.756929:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828608.756931:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828608.756940:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828608.756948:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81e00 x1874669228327424/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828608.756956:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828608.756959:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828608.756961:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828608.756965:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4754 00000100:00080000:3.0:1787828608.756967:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828608.756970:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828608.756974:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828608.756981:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228327296:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787828608.756982:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80f00 x1874669228327552/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828608.757084:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228327424:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787828608.757114:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228327552:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828608.820919:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828609.716838:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787828609.844885:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828610.740777:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828610.868871:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828611.764824:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000400:00000100:1.0:1787828611.828986:0:2372:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.114:988 00000400:00000100:1.0:1787828611.829063:0:2372:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.114@tcp at host 192.168.203.114 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1787828611.829085:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:1.0:1787828611.829135:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:1.0:1787828611.829148:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000100:00100000:2.0:1787828611.829187:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828608/real 1787828611] req@ffff985610f80f00 x1874669228327552/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828633 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828611.829218:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80f00 x1874669228327552/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828633 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828611.829232:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828611.829238:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828611.829243:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228327552:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787828611.829252:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828608/real 1787828611] req@ffff985610f803c0 x1874669228327296/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828633 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828611.829269:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f803c0 x1874669228327296/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828633 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828611.829286:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828611.829290:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828611.829304:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228327296:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787828611.829312:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828608/real 1787828611] req@ffff985610f81e00 x1874669228327424/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828633 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828611.829328:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81e00 x1874669228327424/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828633 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828611.829338:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828611.829342:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828611.829352:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228327424:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787828611.892885:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828612.788826:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828612.916921:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828613.812847:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787828613.876935:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828613.876945:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828613.876952:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828613.876963:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4759 00000100:00080000:3.0:1787828613.876974:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828613.876982:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828613.877028:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82d00 x1874669228327680/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828613.877065:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828613.877070:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828613.877072:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828613.877077:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4759 00000100:00080000:3.0:1787828613.877081:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828613.877086:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828613.877099:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828613.877116:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82940 x1874669228327808/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828613.877129:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828613.877133:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828613.877136:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:2.0:1787828613.877141:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228327680:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787828613.877143:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4759 00000100:00080000:3.0:1787828613.877150:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828613.877156:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828613.877162:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828613.877175:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82580 x1874669228327936/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828613.877240:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828613.877294:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828613/real 1787828613] req@ffff985610f82d00 x1874669228327680/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828638 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828613.877317:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82d00 x1874669228327680/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828638 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828613.877329:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828613.877333:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828613.877338:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228327680:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787828613.877367:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228327808:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828613.877397:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828613.877411:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828613/real 1787828613] req@ffff985610f82940 x1874669228327808/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828638 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828613.877425:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82940 x1874669228327808/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828638 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828613.877433:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828613.877437:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828613.877441:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228327808:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787828613.877447:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228327936:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828613.877467:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828613.877479:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828613/real 1787828613] req@ffff985610f82580 x1874669228327936/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828638 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828613.877491:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82580 x1874669228327936/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828638 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828613.877499:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828613.877502:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828613.877505:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228327936:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828613.940857:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828614.836782:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787828614.964892:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828615.860810:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828615.988810:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828616.884838:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828617.012825:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828617.908807:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828618.036922:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828618.932852:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787828618.996870:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828618.996884:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828618.996889:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828618.996903:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4764 00000100:00080000:1.0:1787828618.996917:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828618.996927:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787828618.996982:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0f00 x1874669228328064/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828618.997013:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828618.997017:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828618.997019:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828618.997025:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4764 00000100:00080000:1.0:1787828618.997030:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828618.997035:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787828618.997047:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828618.997065:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3480 x1874669228328192/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828618.997077:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828618.997081:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828618.997084:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828618.997089:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4764 00000100:00080000:1.0:1787828618.997094:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828618.997100:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787828618.997108:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228328064:192.168.203.114@tcp:250:kworker.0 00000008:00100000:1.0:1787828618.997111:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828618.997142:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a21c0 x1874669228328320/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828618.997178:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828618.997223:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828618/real 1787828618] req@ffff9856085a0f00 x1874669228328064/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828643 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828618.997257:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0f00 x1874669228328064/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828643 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828618.997267:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828618.997270:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828618.997273:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228328064:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787828618.997280:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228328192:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828618.997299:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828618.997309:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828618/real 1787828618] req@ffff9856085a3480 x1874669228328192/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828643 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828618.997317:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3480 x1874669228328192/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828643 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828618.997322:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828618.997325:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828618.997327:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228328192:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787828618.997348:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228328320:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787828618.997359:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787828618.997366:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828618/real 1787828618] req@ffff9856085a21c0 x1874669228328320/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828643 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828618.997373:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a21c0 x1874669228328320/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828643 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828618.997382:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828618.997385:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787828618.997387:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228328320:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828619.060910:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828619.956832:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787828620.084859:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828620.980769:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828621.108818:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828622.004776:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828622.132769:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828623.028780:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828623.156826:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828624.052780:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787828624.116768:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828624.116777:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828624.116781:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828624.116792:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4769 00000100:00080000:1.0:1787828624.116803:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828624.116810:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787828624.116849:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a30c0 x1874669228328448/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828624.116878:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828624.116882:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828624.116884:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828624.116887:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4769 00000100:00080000:1.0:1787828624.116890:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828624.116894:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787828624.116903:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828624.116917:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2940 x1874669228328576/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828624.116928:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828624.116932:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828624.116936:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1787828624.116936:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228328448:192.168.203.114@tcp:250:kworker.0 00000100:00080000:1.0:1787828624.116941:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4769 00000100:00080000:1.0:1787828624.116947:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828624.116950:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787828624.116953:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828624.116964:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a1a40 x1874669228328704/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828624.117029:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828624.117163:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828624/real 1787828624] req@ffff9856085a30c0 x1874669228328448/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828649 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828624.117320:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a30c0 x1874669228328448/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828649 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828624.117335:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828624.117379:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828624.117386:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228328448:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787828624.117418:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228328576:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787828624.117445:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828624.117460:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828624/real 1787828624] req@ffff9856085a2940 x1874669228328576/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828649 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828624.117476:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2940 x1874669228328576/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828649 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828624.117484:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828624.117489:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828624.117493:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228328576:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787828624.117499:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228328704:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787828624.117519:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828624.117532:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828624/real 1787828624] req@ffff9856085a1a40 x1874669228328704/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828649 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828624.117546:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a1a40 x1874669228328704/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828649 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828624.117562:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828624.117565:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828624.117569:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228328704:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828624.180812:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828625.076812:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787828625.204870:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828626.100773:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828626.228875:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828627.124793:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828627.252866:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828628.148764:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828628.276849:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828629.172773:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787828629.236821:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828629.236834:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828629.236839:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828629.236849:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4774 00000100:00080000:0.0:1787828629.236860:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828629.236869:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787828629.236917:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4b40 x1874669228328832/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828629.236947:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828629.236952:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828629.236954:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828629.236958:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4774 00000100:00080000:0.0:1787828629.236962:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828629.236967:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828629.236978:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828629.236994:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6940 x1874669228328960/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828629.237004:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828629.237007:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828629.237009:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828629.237012:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4774 00000100:00080000:0.0:1787828629.237015:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828629.237018:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828629.237023:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828629.237027:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228328832:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787828629.237034:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4780 x1874669228329088/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828629.237098:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828629.237137:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828629/real 1787828629] req@ffff985607df4b40 x1874669228328832/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828654 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828629.237154:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4b40 x1874669228328832/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828654 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828629.237170:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828629.237173:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828629.237176:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228328832:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787828629.237182:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228328960:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787828629.237197:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828629.237206:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828629/real 1787828629] req@ffff985607df6940 x1874669228328960/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828654 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828629.237214:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6940 x1874669228328960/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828654 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828629.237221:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828629.237223:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828629.237226:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228328960:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787828629.237247:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228329088:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787828629.237258:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828629.237264:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828629/real 1787828629] req@ffff985607df4780 x1874669228329088/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828654 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828629.237271:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4780 x1874669228329088/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828654 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828629.237277:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828629.237279:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828629.237281:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228329088:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828629.300831:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828630.196763:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787828630.324898:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828631.220839:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828631.348829:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828632.244811:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828632.372872:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828633.268804:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828633.396852:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828634.292767:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787828634.356758:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828634.356768:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828634.356773:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828634.356784:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4779 00000100:00080000:0.0:1787828634.356795:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828634.356804:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787828634.356842:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7c00 x1874669228329216/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828634.356864:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828634.356866:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828634.356868:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828634.356871:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4779 00000100:00080000:0.0:1787828634.356873:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828634.356876:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828634.356884:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828634.356893:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4000 x1874669228329344/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828634.356899:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828634.356903:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828634.356904:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828634.356907:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4779 00000100:00080000:0.0:1787828634.356910:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828634.356912:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828634.356915:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828634.356926:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4780 x1874669228329472/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828634.356942:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228329216:192.168.203.114@tcp:250:kworker.0 00000800:00000100:3.0:1787828634.357012:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828634.357047:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828634/real 1787828634] req@ffff985607df7c00 x1874669228329216/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828659 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828634.357063:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7c00 x1874669228329216/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828659 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828634.357083:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828634.357087:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828634.357090:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228329216:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787828634.357096:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228329344:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787828634.357112:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828634.357121:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828634/real 1787828634] req@ffff985607df4000 x1874669228329344/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828659 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828634.357129:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4000 x1874669228329344/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828659 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828634.357134:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828634.357137:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828634.357139:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228329344:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787828634.357161:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228329472:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787828634.357173:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828634.357184:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828634/real 1787828634] req@ffff985607df4780 x1874669228329472/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828659 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828634.357197:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4780 x1874669228329472/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828659 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828634.357206:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828634.357209:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828634.357213:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228329472:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828634.420850:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828635.316825:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787828635.444864:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828636.340744:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828636.468850:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828637.364861:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828637.492806:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828638.388848:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828638.516949:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828639.412792:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787828639.476783:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828639.476792:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828639.476796:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828639.476805:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4785 00000100:00080000:1.0:1787828639.476816:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828639.476822:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787828639.476857:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a12c0 x1874669228329600/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828639.476881:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828639.476884:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828639.476886:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828639.476889:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4785 00000100:00080000:1.0:1787828639.476892:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828639.476895:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787828639.476905:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828639.476915:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2580 x1874669228329728/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828639.476924:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828639.476928:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828639.476931:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1787828639.476934:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228329600:192.168.203.114@tcp:250:kworker.0 00000100:00080000:1.0:1787828639.476936:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4785 00000100:00080000:1.0:1787828639.476950:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828639.476953:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787828639.476958:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828639.476968:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3840 x1874669228329856/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828639.477001:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828639.477062:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828639/real 1787828639] req@ffff9856085a12c0 x1874669228329600/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828664 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828639.477080:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a12c0 x1874669228329600/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828664 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828639.477090:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828639.477094:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828639.477097:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228329600:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787828639.477120:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228329728:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787828639.477140:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828639.477151:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828639/real 1787828639] req@ffff9856085a2580 x1874669228329728/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828664 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828639.477161:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2580 x1874669228329728/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828664 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828639.477168:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828639.477171:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828639.477173:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228329728:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787828639.477179:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228329856:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787828639.477192:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828639.477200:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828639/real 1787828639] req@ffff9856085a3840 x1874669228329856/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828664 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828639.477210:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3840 x1874669228329856/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828664 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828639.477216:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828639.477218:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828639.477222:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228329856:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828639.540872:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828640.436792:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787828640.564865:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828641.460781:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828641.588843:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828642.484788:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828642.612872:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828643.508751:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828643.636813:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828644.532838:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787828644.596773:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828644.596784:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828644.596789:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828644.596798:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4790 00000100:00080000:1.0:1787828644.596810:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828644.596817:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787828644.596857:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a03c0 x1874669228329984/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828644.596884:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828644.596887:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828644.596889:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828644.596892:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4790 00000100:00080000:1.0:1787828644.596895:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828644.596898:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787828644.596914:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828644.596923:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228329984:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787828644.596925:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0780 x1874669228330112/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828644.596938:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828644.596940:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828644.596942:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828644.596945:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4790 00000100:00080000:1.0:1787828644.596947:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828644.596950:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787828644.596953:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828644.596960:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a1680 x1874669228330240/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828644.597000:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828644.597039:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828644/real 1787828644] req@ffff9856085a03c0 x1874669228329984/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828669 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828644.597055:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a03c0 x1874669228329984/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828669 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828644.597064:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828644.597068:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828644.597071:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228329984:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787828644.597099:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228330112:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787828644.597124:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828644.597139:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828644/real 1787828644] req@ffff9856085a0780 x1874669228330112/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828669 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828644.597148:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0780 x1874669228330112/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828669 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828644.597154:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828644.597156:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828644.597159:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228330112:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787828644.597163:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228330240:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787828644.597177:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828644.597193:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828644/real 1787828644] req@ffff9856085a1680 x1874669228330240/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828669 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828644.597201:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a1680 x1874669228330240/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828669 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828644.597207:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828644.597209:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828644.597212:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228330240:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828644.660873:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828645.556836:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787828645.684965:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828646.580897:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828646.708876:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828647.604783:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828647.732870:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828648.628798:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828648.756803:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828649.652787:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787828649.716872:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828649.716884:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828649.716888:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828649.716897:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4795 00000100:00080000:2.0:1787828649.716908:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828649.716914:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787828649.716978:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db0c0 x1874669228330368/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828649.717004:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828649.717006:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828649.717008:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828649.717011:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4795 00000100:00080000:2.0:1787828649.717014:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828649.717016:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787828649.717024:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828649.717035:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140dad00 x1874669228330496/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828649.717042:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828649.717045:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828649.717047:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828649.717050:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4795 00000100:00080000:2.0:1787828649.717052:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828649.717057:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787828649.717062:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828649.717066:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228330368:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787828649.717073:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8f00 x1874669228330624/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828649.717142:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828649.717184:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828649/real 1787828649] req@ffff9856140db0c0 x1874669228330368/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828674 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828649.717208:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db0c0 x1874669228330368/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828674 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828649.717217:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828649.717220:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828649.717223:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228330368:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787828649.717230:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228330496:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787828649.717247:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828649.717261:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828649/real 1787828649] req@ffff9856140dad00 x1874669228330496/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828674 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828649.717270:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140dad00 x1874669228330496/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828674 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828649.717276:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828649.717278:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828649.717281:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228330496:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787828649.717305:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228330624:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787828649.717318:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828649.717325:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828649/real 1787828649] req@ffff9856140d8f00 x1874669228330624/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828674 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828649.717334:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8f00 x1874669228330624/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828674 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828649.717339:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828649.717341:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828649.717343:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228330624:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828649.780874:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828650.676879:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787828650.804863:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828651.700810:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828651.828894:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828652.724779:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828652.852799:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828653.748800:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828653.876864:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828654.772788:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787828654.836859:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828654.836867:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828654.836871:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828654.836879:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4800 00000100:00080000:0.0:1787828654.836889:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828654.836896:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787828654.836930:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6940 x1874669228330752/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828654.836954:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828654.836957:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828654.836958:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828654.836961:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4800 00000100:00080000:0.0:1787828654.836963:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828654.836966:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828654.836974:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828654.836983:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4b40 x1874669228330880/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828654.837009:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228330752:192.168.203.114@tcp:250:kworker.0 00000100:00080000:0.0:1787828654.837010:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828654.837018:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828654.837021:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828654.837024:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4800 00000100:00080000:0.0:1787828654.837026:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828654.837029:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828654.837033:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828654.837042:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7480 x1874669228331008/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828654.837078:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828654.837111:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828654/real 1787828654] req@ffff985607df6940 x1874669228330752/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828679 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828654.837125:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6940 x1874669228330752/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828679 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828654.837134:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828654.837138:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828654.837141:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228330752:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787828654.837160:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228330880:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787828654.837174:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828654.837183:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828654/real 1787828654] req@ffff985607df4b40 x1874669228330880/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828679 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828654.837192:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4b40 x1874669228330880/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828679 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828654.837197:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828654.837199:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828654.837201:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228330880:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787828654.837205:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228331008:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787828654.837216:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828654.837222:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828654/real 1787828654] req@ffff985607df7480 x1874669228331008/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828679 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828654.837230:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7480 x1874669228331008/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828679 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828654.837235:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828654.837237:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828654.837239:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228331008:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828654.900943:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828655.796832:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787828655.924920:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828656.820791:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787828656.948880:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828657.844827:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828657.972893:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828658.868863:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828658.996840:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828659.892813:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787828659.956839:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828659.956852:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828659.956857:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828659.956872:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4805 00000100:00080000:0.0:1787828659.956885:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828659.956894:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787828659.956943:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df52c0 x1874669228331136/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828659.956988:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828659.956992:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828659.956994:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828659.956998:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4805 00000100:00080000:0.0:1787828659.957002:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828659.957007:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828659.957017:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828659.957033:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7840 x1874669228331264/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828659.957044:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828659.957048:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828659.957051:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828659.957064:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4805 00000100:00080000:0.0:1787828659.957070:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00100000:1.0:1787828659.957074:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228331136:192.168.203.114@tcp:250:kworker.0 00000100:00080000:0.0:1787828659.957080:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828659.957092:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828659.957107:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df61c0 x1874669228331392/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787828659.957175:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828659.957230:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828659/real 1787828659] req@ffff985607df52c0 x1874669228331136/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828684 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828659.957250:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df52c0 x1874669228331136/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828684 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828659.957260:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828659.957264:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828659.957268:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228331136:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787828659.957275:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228331264:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828659.957297:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828659.957311:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828659/real 1787828659] req@ffff985607df7840 x1874669228331264/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828684 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828659.957323:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7840 x1874669228331264/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828684 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828659.957331:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828659.957334:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828659.957337:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228331264:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787828659.957363:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228331392:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828659.957380:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828659.957389:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828659/real 1787828659] req@ffff985607df61c0 x1874669228331392/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828684 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828659.957399:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df61c0 x1874669228331392/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828684 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828659.957412:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828659.957416:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828659.957419:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228331392:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828660.020964:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828660.916809:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787828661.044924:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828661.940822:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787828662.068975:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828662.964848:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828663.092950:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828663.988791:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828664.116795:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828665.012800:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787828665.076863:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828665.076874:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828665.076878:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828665.076889:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4810 00000100:00080000:0.0:1787828665.076900:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828665.076907:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787828665.076958:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6d00 x1874669228331520/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828665.076987:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828665.076990:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828665.076992:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828665.076994:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4810 00000100:00080000:0.0:1787828665.077000:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828665.077003:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828665.077011:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828665.077021:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4f00 x1874669228331648/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828665.077029:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828665.077031:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828665.077033:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828665.077035:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4810 00000100:00080000:0.0:1787828665.077040:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828665.077043:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828665.077047:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828665.077053:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228331520:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787828665.077061:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5e00 x1874669228331776/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787828665.077130:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828665.077168:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828665/real 1787828665] req@ffff985607df6d00 x1874669228331520/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828690 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828665.077182:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6d00 x1874669228331520/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828690 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828665.077191:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828665.077194:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828665.077198:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228331520:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787828665.077203:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228331648:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828665.077218:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828665.077226:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828665/real 1787828665] req@ffff985607df4f00 x1874669228331648/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828690 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828665.077234:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4f00 x1874669228331648/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828690 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828665.077240:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828665.077242:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828665.077244:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228331648:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787828665.077267:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228331776:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828665.077279:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828665.077285:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828665/real 1787828665] req@ffff985607df5e00 x1874669228331776/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828690 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828665.077293:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5e00 x1874669228331776/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828690 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828665.077298:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828665.077300:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828665.077303:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228331776:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828665.140969:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828666.036851:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787828666.164886:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828667.060813:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787828667.188892:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828668.084848:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828668.212895:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828669.108774:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828669.236892:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828670.132762:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787828670.196858:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828670.196869:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828670.196873:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828670.196884:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4815 00000100:00080000:0.0:1787828670.196897:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828670.196912:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787828670.196954:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df43c0 x1874669228331904/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828670.196980:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828670.196983:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828670.196985:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828670.196987:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4815 00000100:00080000:0.0:1787828670.196990:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828670.196993:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828670.197001:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828670.197016:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5680 x1874669228332032/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828670.197029:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828670.197032:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828670.197033:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828670.197037:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4815 00000100:00080000:0.0:1787828670.197039:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828670.197042:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828670.197045:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828670.197050:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228331904:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787828670.197056:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df70c0 x1874669228332160/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787828670.197132:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828670.197170:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828670/real 1787828670] req@ffff985607df43c0 x1874669228331904/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828695 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828670.197185:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df43c0 x1874669228331904/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828695 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828670.197194:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828670.197198:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828670.197201:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228331904:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787828670.197207:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228332032:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828670.197223:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828670.197232:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828670/real 1787828670] req@ffff985607df5680 x1874669228332032/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828695 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828670.197240:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5680 x1874669228332032/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828695 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828670.197245:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828670.197247:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828670.197249:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228332032:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787828670.197269:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228332160:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828670.197280:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828670.197286:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828670/real 1787828670] req@ffff985607df70c0 x1874669228332160/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828695 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828670.197294:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df70c0 x1874669228332160/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828695 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828670.197299:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828670.197301:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828670.197303:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228332160:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828670.260865:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828671.156803:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787828671.284869:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828672.180766:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787828672.308836:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828673.204776:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828673.332878:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828674.228774:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828674.356809:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828675.252795:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787828675.316890:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828675.316900:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828675.316905:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828675.316915:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4820 00000100:00080000:2.0:1787828675.316927:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828675.316935:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787828675.316976:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140dbc00 x1874669228332288/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828675.317000:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828675.317003:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828675.317004:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828675.317007:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4820 00000100:00080000:2.0:1787828675.317009:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828675.317013:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787828675.317022:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828675.317030:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140da1c0 x1874669228332416/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828675.317038:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828675.317041:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828675.317042:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828675.317045:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4820 00000100:00080000:2.0:1787828675.317047:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828675.317049:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787828675.317053:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828675.317059:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228332288:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787828675.317064:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db840 x1874669228332544/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787828675.317161:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228332416:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787828675.317190:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228332544:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828675.380844:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828676.276857:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787828676.404903:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828677.300779:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787828677.428909:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828678.324797:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000400:00000100:3.0:1787828678.389063:0:2373:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.114:988 00000400:00000100:3.0:1787828678.389115:0:2373:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.114@tcp at host 192.168.203.114 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:3.0:1787828678.389134:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:3.0:1787828678.389184:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:3.0:1787828678.389194:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000100:00100000:1.0:1787828678.389245:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828675/real 1787828678] req@ffff9856140db840 x1874669228332544/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828700 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828678.389277:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db840 x1874669228332544/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828700 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828678.389297:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828678.389303:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828678.389307:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228332544:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787828678.389316:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828675/real 1787828678] req@ffff9856140dbc00 x1874669228332288/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828700 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828678.389344:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140dbc00 x1874669228332288/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828700 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828678.389354:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828678.389360:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828678.389363:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228332288:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787828678.389371:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828675/real 1787828678] req@ffff9856140da1c0 x1874669228332416/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828700 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828678.389384:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140da1c0 x1874669228332416/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828700 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828678.389393:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828678.389397:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828678.389400:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228332416:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787828678.452912:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828679.348848:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828679.476872:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828680.372779:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787828680.436900:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828680.436910:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828680.436914:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828680.436924:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4826 00000100:00080000:1.0:1787828680.436936:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828680.436944:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787828680.436985:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3c00 x1874669228332672/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828680.437019:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828680.437022:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828680.437024:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828680.437027:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4826 00000100:00080000:1.0:1787828680.437030:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828680.437034:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787828680.437044:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828680.437060:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228332672:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787828680.437062:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0b40 x1874669228332800/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828680.437079:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828680.437082:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828680.437086:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828680.437090:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4826 00000100:00080000:1.0:1787828680.437104:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828680.437109:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787828680.437125:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828680.437138:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0000 x1874669228332928/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828680.437157:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828680.437207:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828680/real 1787828680] req@ffff9856085a3c00 x1874669228332672/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828705 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828680.437242:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3c00 x1874669228332672/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828705 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828680.437254:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828680.437259:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828680.437263:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228332672:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787828680.437303:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228332800:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787828680.437348:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828680.437363:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828680/real 1787828680] req@ffff9856085a0b40 x1874669228332800/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828705 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828680.437377:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0b40 x1874669228332800/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828705 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828680.437387:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828680.437390:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828680.437394:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228332800:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787828680.437401:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228332928:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787828680.437423:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828680.437436:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828680/real 1787828680] req@ffff9856085a0000 x1874669228332928/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828705 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828680.437448:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0000 x1874669228332928/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828705 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828680.437457:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828680.437461:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828680.437465:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228332928:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828680.500903:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828681.396841:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787828681.524882:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828682.420806:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787828682.548917:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828683.444824:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828683.572873:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828684.468827:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828684.596889:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828685.492811:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787828685.556913:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828685.556923:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828685.556927:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828685.556940:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4831 00000100:00080000:3.0:1787828685.556955:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828685.556965:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828685.557015:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f821c0 x1874669228333056/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828685.557053:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828685.557057:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828685.557060:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828685.557066:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4831 00000100:00080000:3.0:1787828685.557071:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828685.557075:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787828685.557087:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f821c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228333056:192.168.203.114@tcp:250:kworker.0 00000008:00100000:3.0:1787828685.557088:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828685.557107:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83480 x1874669228333184/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828685.557119:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828685.557123:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828685.557125:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828685.557130:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4831 00000100:00080000:3.0:1787828685.557145:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828685.557150:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828685.557156:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828685.557170:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82580 x1874669228333312/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787828685.557189:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828685.557230:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828685/real 1787828685] req@ffff985610f821c0 x1874669228333056/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828710 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828685.557246:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f821c0 x1874669228333056/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828710 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828685.557254:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828685.557257:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828685.557261:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f821c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228333056:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787828685.557340:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228333184:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828685.557374:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828685.557388:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828685/real 1787828685] req@ffff985610f83480 x1874669228333184/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828710 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828685.557399:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83480 x1874669228333184/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828710 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828685.557408:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828685.557411:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828685.557414:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228333184:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787828685.557419:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228333312:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828685.557436:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828685.557444:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828685/real 1787828685] req@ffff985610f82580 x1874669228333312/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828710 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828685.557453:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82580 x1874669228333312/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828710 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828685.557460:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828685.557462:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828685.557464:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228333312:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828685.620887:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828686.516785:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787828686.644883:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828687.540813:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787828687.668869:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828688.564825:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828688.692863:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828689.588831:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828689.716807:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828690.612808:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787828690.676795:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828690.676806:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828690.676810:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828690.676821:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4836 00000100:00080000:3.0:1787828690.676834:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828690.676853:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828690.676928:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82940 x1874669228333440/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828690.676976:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828690.676979:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828690.676981:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828690.676984:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4836 00000100:00080000:3.0:1787828690.676987:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828690.676990:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828690.677000:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828690.677011:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82d00 x1874669228333568/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828690.677019:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828690.677022:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828690.677023:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828690.677027:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4836 00000100:00080000:3.0:1787828690.677047:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828690.677052:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828690.677058:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828690.677073:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81e00 x1874669228333696/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787828690.677093:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228333440:192.168.203.114@tcp:250:kworker.0 00000800:00000100:1.0:1787828690.677200:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828690.677269:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828690/real 1787828690] req@ffff985610f82940 x1874669228333440/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828715 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828690.677304:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82940 x1874669228333440/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828715 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828690.677314:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828690.677318:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828690.677333:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228333440:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787828690.677339:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228333568:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828690.677358:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828690.677368:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828690/real 1787828690] req@ffff985610f82d00 x1874669228333568/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828715 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828690.677376:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82d00 x1874669228333568/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828715 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828690.677381:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828690.677383:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828690.677385:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228333568:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787828690.677418:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228333696:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828690.677429:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828690.677436:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828690/real 1787828690] req@ffff985610f81e00 x1874669228333696/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828715 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828690.677444:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81e00 x1874669228333696/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828715 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828690.677449:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828690.677451:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828690.677454:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228333696:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828690.740906:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828691.636829:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787828691.764867:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828692.660811:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787828692.788963:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828693.684850:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828693.812913:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828694.708828:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828694.836842:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828695.732793:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787828695.796921:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828695.796941:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828695.796949:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828695.796967:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4841 00000100:00080000:3.0:1787828695.796987:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828695.797003:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828695.797083:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f803c0 x1874669228333824/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828695.797147:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828695.797157:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828695.797161:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828695.797170:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4841 00000100:00080000:3.0:1787828695.797180:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828695.797190:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828695.797251:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828695.797253:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228333824:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787828695.797292:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80f00 x1874669228333952/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828695.797319:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828695.797329:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828695.797333:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828695.797341:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4841 00000100:00080000:3.0:1787828695.797352:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828695.797361:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000800:00000100:1.0:1787828695.797369:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000008:00100000:3.0:1787828695.797378:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828695.797412:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83840 x1874669228334080/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787828695.797467:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828695/real 1787828695] req@ffff985610f803c0 x1874669228333824/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828720 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828695.797511:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f803c0 x1874669228333824/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828720 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828695.797532:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828695.797541:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828695.797554:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228333824:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787828695.797602:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228333952:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828695.797666:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828695.797750:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828695/real 1787828695] req@ffff985610f80f00 x1874669228333952/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828720 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828695.797774:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80f00 x1874669228333952/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828720 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828695.797787:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828695.797795:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828695.797800:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228333952:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787828695.797813:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228334080:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828695.797859:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828695.797882:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828695/real 1787828695] req@ffff985610f83840 x1874669228334080/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828720 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828695.797902:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83840 x1874669228334080/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828720 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828695.797914:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828695.797921:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828695.797926:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228334080:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828695.861039:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828696.756818:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787828696.884833:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828697.780809:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787828697.908962:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828698.804856:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828698.932907:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828699.828863:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828699.956930:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828700.852803:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787828700.916786:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828700.916799:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828700.916805:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828700.916817:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4846 00000100:00080000:0.0:1787828700.916831:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828700.916841:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787828700.916892:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5a40 x1874669228334208/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828700.916930:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828700.916934:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828700.916936:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828700.916965:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4846 00000100:00080000:0.0:1787828700.916969:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828700.916973:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828700.916984:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828700.916999:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6580 x1874669228334336/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828700.917011:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828700.917034:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828700.917036:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828700.917044:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4846 00000100:00080000:0.0:1787828700.917047:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00100000:1.0:1787828700.917047:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228334208:192.168.203.114@tcp:250:kworker.0 00000100:00080000:0.0:1787828700.917055:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828700.917066:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828700.917079:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df70c0 x1874669228334464/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787828700.917209:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828700.917254:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828700/real 1787828700] req@ffff985607df5a40 x1874669228334208/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828725 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828700.917296:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5a40 x1874669228334208/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828725 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828700.917309:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828700.917314:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828700.917319:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228334208:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787828700.917363:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228334336:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828700.917397:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828700.917416:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828700/real 1787828700] req@ffff985607df6580 x1874669228334336/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828725 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828700.917435:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6580 x1874669228334336/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828725 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828700.917449:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828700.917452:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828700.917456:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228334336:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787828700.917462:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228334464:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828700.917485:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828700.917496:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828700/real 1787828700] req@ffff985607df70c0 x1874669228334464/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828725 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828700.917506:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df70c0 x1874669228334464/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828725 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828700.917513:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828700.917517:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828700.917519:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228334464:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828700.980909:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828701.876776:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787828702.004911:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828702.900912:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787828703.028878:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828703.924831:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828704.052847:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828704.948946:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828705.076893:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828705.972798:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787828706.036767:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828706.036778:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828706.036782:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828706.036792:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4851 00000100:00080000:0.0:1787828706.036803:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828706.036811:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787828706.036846:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5680 x1874669228334592/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828706.036870:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828706.036872:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828706.036874:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828706.036877:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4851 00000100:00080000:0.0:1787828706.036879:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828706.036881:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828706.036890:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828706.036900:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df43c0 x1874669228334720/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828706.036908:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:1.0:1787828706.036913:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228334592:192.168.203.114@tcp:250:kworker.0 00000100:00080000:0.0:1787828706.036914:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828706.036916:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828706.036924:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4851 00000100:00080000:0.0:1787828706.036927:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828706.036929:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828706.036932:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828706.036942:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5e00 x1874669228334848/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787828706.036995:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828706.037034:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828706/real 1787828706] req@ffff985607df5680 x1874669228334592/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828731 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828706.037048:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5680 x1874669228334592/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828731 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828706.037057:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828706.037061:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828706.037064:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228334592:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787828706.037086:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228334720:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828706.037102:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828706.037111:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828706/real 1787828706] req@ffff985607df43c0 x1874669228334720/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828731 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828706.037119:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df43c0 x1874669228334720/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828731 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828706.037125:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828706.037127:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828706.037142:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228334720:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787828706.037148:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228334848:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828706.037164:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828706.037172:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828706/real 1787828706] req@ffff985607df5e00 x1874669228334848/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828731 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828706.037180:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5e00 x1874669228334848/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828731 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828706.037186:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828706.037188:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828706.037190:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228334848:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828706.100828:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828706.996831:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787828707.124950:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828708.020770:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787828708.148980:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828709.044794:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828709.172821:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828710.068814:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828710.196938:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828711.092822:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787828711.156865:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828711.156881:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828711.156887:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828711.156919:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4856 00000100:00080000:3.0:1787828711.156935:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828711.156944:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828711.156996:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81a40 x1874669228334976/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828711.157032:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828711.157036:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828711.157038:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828711.157042:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4856 00000100:00080000:3.0:1787828711.157046:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828711.157050:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828711.157062:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828711.157075:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81680 x1874669228335104/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828711.157118:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828711.157121:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828711.157123:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828711.157128:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4856 00000100:00080000:3.0:1787828711.157131:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828711.157135:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828711.157140:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828711.157163:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228334976:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787828711.157174:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f812c0 x1874669228335232/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787828711.157286:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828711.157331:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828711/real 1787828711] req@ffff985610f81a40 x1874669228334976/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828736 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828711.157349:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81a40 x1874669228334976/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828736 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828711.157360:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828711.157364:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828711.157367:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228334976:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787828711.157375:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228335104:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828711.157394:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828711.157405:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828711/real 1787828711] req@ffff985610f81680 x1874669228335104/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828736 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828711.157415:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81680 x1874669228335104/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828736 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828711.157421:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828711.157424:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828711.157426:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228335104:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787828711.157450:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228335232:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828711.157464:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828711.157472:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828711/real 1787828711] req@ffff985610f812c0 x1874669228335232/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828736 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828711.157482:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f812c0 x1874669228335232/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828736 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828711.157489:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828711.157491:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828711.157493:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228335232:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828711.220903:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828712.116837:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787828712.244935:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828713.140782:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787828713.268878:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828714.164764:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828714.292844:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828715.188804:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828715.316866:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828716.212826:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787828716.276803:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828716.276814:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828716.276818:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828716.276828:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4861 00000100:00080000:3.0:1787828716.276839:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828716.276847:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828716.276885:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80b40 x1874669228335360/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828716.276915:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828716.276919:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828716.276921:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828716.276926:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4861 00000100:00080000:3.0:1787828716.276930:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828716.276934:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828716.276946:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828716.276960:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80000 x1874669228335488/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828716.276971:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828716.276974:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828716.276976:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828716.276979:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4861 00000100:00080000:3.0:1787828716.276984:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00100000:1.0:1787828716.276985:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228335360:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787828716.276988:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828716.276992:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828716.277002:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f830c0 x1874669228335616/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787828716.277076:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828716.277116:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828716/real 1787828716] req@ffff985610f80b40 x1874669228335360/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828741 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828716.277132:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80b40 x1874669228335360/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828741 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828716.277140:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828716.277143:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828716.277147:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228335360:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787828716.277152:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228335488:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828716.277173:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828716.277182:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828716/real 1787828716] req@ffff985610f80000 x1874669228335488/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828741 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828716.277190:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80000 x1874669228335488/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828741 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828716.277196:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828716.277198:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828716.277201:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228335488:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787828716.277221:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228335616:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828716.277232:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828716.277238:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828716/real 1787828716] req@ffff985610f830c0 x1874669228335616/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828741 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828716.277246:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f830c0 x1874669228335616/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828741 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828716.277251:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828716.277253:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828716.277255:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228335616:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828716.340884:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828717.236794:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787828717.364858:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828718.260804:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787828718.388939:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828719.284772:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828719.412821:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828720.308838:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828720.436837:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828721.332804:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787828721.396807:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828721.396818:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828721.396821:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828721.396831:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4867 00000100:00080000:3.0:1787828721.396843:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828721.396851:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828721.396891:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80780 x1874669228335744/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828721.396918:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828721.396922:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828721.396925:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828721.396930:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4867 00000100:00080000:3.0:1787828721.396938:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828721.396942:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828721.396954:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828721.396969:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83c00 x1874669228335872/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828721.396979:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828721.396983:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:1.0:1787828721.396985:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228335744:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787828721.396985:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828721.396989:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4867 00000100:00080000:3.0:1787828721.396992:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828721.396995:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828721.397000:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828721.397011:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f830c0 x1874669228336000/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787828721.397080:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828721.397120:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828721/real 1787828721] req@ffff985610f80780 x1874669228335744/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828746 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828721.397134:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80780 x1874669228335744/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828746 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828721.397142:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828721.397146:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828721.397149:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228335744:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787828721.397171:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228335872:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828721.397189:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828721.397197:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828721/real 1787828721] req@ffff985610f83c00 x1874669228335872/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828746 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828721.397205:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83c00 x1874669228335872/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828746 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828721.397211:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828721.397213:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828721.397215:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228335872:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787828721.397218:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228336000:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828721.397229:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828721.397235:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828721/real 1787828721] req@ffff985610f830c0 x1874669228336000/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828746 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828721.397242:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f830c0 x1874669228336000/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828746 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828721.397247:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828721.397249:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828721.397251:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228336000:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828721.460922:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828722.356843:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787828722.484871:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828723.380783:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787828723.509044:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828724.404799:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828724.532849:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828725.428798:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828725.556932:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828726.452791:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787828726.516871:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828726.516882:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828726.516897:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828726.516910:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4872 00000100:00080000:1.0:1787828726.516923:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828726.516932:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787828726.516979:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2d00 x1874669228336128/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828726.517041:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828726.517045:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828726.517047:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828726.517051:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4872 00000100:00080000:1.0:1787828726.517054:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828726.517059:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787828726.517069:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828726.517097:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0000 x1874669228336256/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828726.517108:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228336128:192.168.203.114@tcp:250:kworker.0 00000100:00080000:1.0:1787828726.517112:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828726.517118:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828726.517120:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828726.517124:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4872 00000100:00080000:1.0:1787828726.517127:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828726.517130:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787828726.517135:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828726.517147:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0b40 x1874669228336384/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828726.517197:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828726.517246:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828726/real 1787828726] req@ffff9856085a2d00 x1874669228336128/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828751 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828726.517272:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2d00 x1874669228336128/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828751 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828726.517314:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828726.517319:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828726.517326:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228336128:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787828726.517355:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228336256:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787828726.517394:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828726.517409:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828726/real 1787828726] req@ffff9856085a0000 x1874669228336256/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828751 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828726.517437:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0000 x1874669228336256/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828751 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828726.517445:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828726.517448:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828726.517451:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228336256:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787828726.517456:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228336384:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787828726.517473:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828726.517494:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828726/real 1787828726] req@ffff9856085a0b40 x1874669228336384/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828751 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828726.517509:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0b40 x1874669228336384/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828751 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828726.517519:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828726.517524:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828726.517528:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228336384:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828726.580870:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828727.476791:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787828727.605034:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828728.500814:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787828728.628902:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828729.524797:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828729.652806:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828730.548804:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828730.676927:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828731.572860:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787828731.636842:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828731.636870:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828731.636875:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828731.636888:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4877 00000100:00080000:3.0:1787828731.636901:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828731.636911:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828731.636991:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80000 x1874669228336512/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828731.637020:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828731.637023:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828731.637025:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828731.637028:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4877 00000100:00080000:3.0:1787828731.637043:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828731.637048:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828731.637059:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828731.637070:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228336512:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787828731.637081:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80b40 x1874669228336640/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828731.637097:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828731.637102:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828731.637104:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828731.637109:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4877 00000100:00080000:3.0:1787828731.637115:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828731.637120:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828731.637128:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828731.637152:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f812c0 x1874669228336768/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787828731.637160:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828731.637205:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828731/real 1787828731] req@ffff985610f80000 x1874669228336512/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828756 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828731.637228:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80000 x1874669228336512/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828756 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828731.637237:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828731.637241:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828731.637247:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228336512:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787828731.637285:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228336640:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828731.637311:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828731.637338:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828731/real 1787828731] req@ffff985610f80b40 x1874669228336640/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828756 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828731.637353:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80b40 x1874669228336640/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828756 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828731.637363:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828731.637365:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828731.637369:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228336640:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787828731.637374:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228336768:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828731.637397:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828731.637412:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828731/real 1787828731] req@ffff985610f812c0 x1874669228336768/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828756 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828731.637427:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f812c0 x1874669228336768/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828756 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828731.637447:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828731.637450:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828731.637455:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228336768:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828731.700895:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828732.596794:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787828732.725015:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828733.620828:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787828733.748904:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828734.644799:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828734.772853:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828735.668823:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828735.796839:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828736.692798:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787828736.756821:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828736.756831:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828736.756834:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828736.756845:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4882 00000100:00080000:3.0:1787828736.756856:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828736.756863:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828736.756903:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81680 x1874669228336896/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828736.756945:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828736.756948:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828736.756949:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828736.756952:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4882 00000100:00080000:3.0:1787828736.756955:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828736.756962:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828736.756971:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828736.756985:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81a40 x1874669228337024/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828736.756993:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828736.756996:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828736.756999:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828736.757002:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4882 00000100:00080000:3.0:1787828736.757004:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828736.757007:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828736.757010:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828736.757019:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83840 x1874669228337152/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787828736.757026:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228336896:192.168.203.114@tcp:250:kworker.0 00000800:00000100:1.0:1787828736.757112:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828736.757158:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828736/real 1787828736] req@ffff985610f81680 x1874669228336896/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828761 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828736.757181:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81680 x1874669228336896/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828761 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828736.757194:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828736.757198:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828736.757204:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228336896:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787828736.757213:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228337024:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828736.757241:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828736.757256:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828736/real 1787828736] req@ffff985610f81a40 x1874669228337024/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828761 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828736.757271:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81a40 x1874669228337024/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828761 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828736.757281:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828736.757285:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828736.757289:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228337024:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787828736.757319:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228337152:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828736.757344:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828736.757358:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828736/real 1787828736] req@ffff985610f83840 x1874669228337152/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828761 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828736.757372:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83840 x1874669228337152/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828761 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828736.757379:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828736.757382:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828736.757384:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228337152:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828736.820932:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828737.716752:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787828737.844942:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828738.740752:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787828738.868974:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828739.764822:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828739.892957:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828740.788797:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828740.916805:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828741.812816:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787828741.876950:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828741.876980:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828741.876986:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828741.877000:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4887 00000100:00080000:0.0:1787828741.877016:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828741.877027:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787828741.877083:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4f00 x1874669228337280/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828741.877144:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828741.877149:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828741.877151:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828741.877157:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4887 00000100:00080000:0.0:1787828741.877162:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828741.877168:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828741.877181:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828741.877200:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6d00 x1874669228337408/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828741.877210:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828741.877215:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828741.877217:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828741.877223:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4887 00000100:00080000:0.0:1787828741.877226:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828741.877229:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787828741.877231:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228337280:192.168.203.114@tcp:250:kworker.0 00000008:00100000:0.0:1787828741.877234:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828741.877246:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df61c0 x1874669228337536/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787828741.877389:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228337408:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787828741.877439:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228337536:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828741.940865:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828742.836801:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787828742.965030:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828743.860896:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787828743.988985:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828744.884877:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000400:00000100:1.0:1787828744.949269:0:2370:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.114:988 00000400:00000100:1.0:1787828744.949389:0:2370:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.114@tcp at host 192.168.203.114 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1787828744.949415:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:1.0:1787828744.949489:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:1.0:1787828744.949502:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000100:00100000:3.0:1787828744.949604:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828741/real 1787828744] req@ffff985607df6d00 x1874669228337408/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828766 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828744.949636:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6d00 x1874669228337408/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828766 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828744.949652:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828744.949659:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828744.949664:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228337408:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787828744.949676:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828741/real 1787828744] req@ffff985607df61c0 x1874669228337536/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828766 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828744.950428:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df61c0 x1874669228337536/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828766 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828744.950444:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828744.950450:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828744.950455:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228337536:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787828744.950481:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828741/real 1787828744] req@ffff985607df4f00 x1874669228337280/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828766 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828744.950501:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4f00 x1874669228337280/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828766 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828744.950510:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828744.950514:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828744.950518:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228337280:192.168.203.114@tcp:250:kworker.0 00010000:00010000:0.0:1787828745.012846:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828745.908790:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828746.036824:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828746.932912:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787828746.996926:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828746.996939:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828746.996945:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828746.996961:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4892 00000100:00080000:3.0:1787828746.996977:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828746.996989:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828746.997041:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80f00 x1874669228337664/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828746.997103:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828746.997111:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828746.997115:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828746.997124:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4892 00000100:00080000:3.0:1787828746.997131:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828746.997139:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828746.997156:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828746.997185:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f803c0 x1874669228337792/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828746.997211:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828746.997222:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:1.0:1787828746.997223:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228337664:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787828746.997227:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828746.997245:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4892 00000100:00080000:3.0:1787828746.997255:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828746.997263:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828746.997275:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828746.997304:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81e00 x1874669228337920/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787828746.997368:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828746.997445:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828746/real 1787828746] req@ffff985610f80f00 x1874669228337664/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828771 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828746.997485:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80f00 x1874669228337664/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828771 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828746.997507:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828746.997516:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828746.997525:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228337664:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787828746.997572:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228337792:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828746.997626:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828746.997656:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828746/real 1787828746] req@ffff985610f803c0 x1874669228337792/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828771 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828746.997684:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f803c0 x1874669228337792/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828771 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828746.997759:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828746.997767:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828746.997774:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228337792:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787828746.997786:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228337920:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828746.997833:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828746.997893:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828746/real 1787828746] req@ffff985610f81e00 x1874669228337920/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828771 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828746.997916:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81e00 x1874669228337920/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828771 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828746.997931:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828746.997938:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828746.997944:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228337920:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828747.060905:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828747.956865:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787828748.084904:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828748.980824:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787828749.108927:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828750.004847:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828750.133036:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828751.028892:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828751.156902:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828752.052884:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787828752.116853:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828752.116861:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828752.116865:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828752.116886:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4897 00000100:00080000:3.0:1787828752.116895:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828752.116901:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828752.116936:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82d00 x1874669228338048/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828752.116978:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828752.116981:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828752.116982:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828752.116985:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4897 00000100:00080000:3.0:1787828752.116987:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828752.116991:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828752.116999:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828752.117014:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82940 x1874669228338176/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828752.117026:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828752.117030:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828752.117034:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1787828752.117036:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228338048:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787828752.117042:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4897 00000100:00080000:3.0:1787828752.117050:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828752.117054:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828752.117069:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828752.117081:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82580 x1874669228338304/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787828752.117123:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828752.117163:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828752/real 1787828752] req@ffff985610f82d00 x1874669228338048/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828777 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828752.117177:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82d00 x1874669228338048/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828777 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828752.117186:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828752.117190:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828752.117193:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228338048:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787828752.117214:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228338176:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828752.117229:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828752.117237:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828752/real 1787828752] req@ffff985610f82940 x1874669228338176/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828777 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828752.117245:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82940 x1874669228338176/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828777 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828752.117251:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828752.117253:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828752.117255:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228338176:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787828752.117259:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228338304:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828752.117269:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828752.117275:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828752/real 1787828752] req@ffff985610f82580 x1874669228338304/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828777 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828752.117282:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82580 x1874669228338304/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828777 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828752.117287:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828752.117289:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828752.117290:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228338304:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828752.180889:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828753.076856:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787828753.205024:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828754.100770:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787828754.228899:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828755.124853:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828755.252843:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828756.148777:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828756.276848:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828757.172900:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787828757.236793:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828757.236813:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828757.236817:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828757.236826:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4902 00000100:00080000:3.0:1787828757.236836:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828757.236843:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828757.236891:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83480 x1874669228338432/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828757.236915:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828757.236918:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828757.236920:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828757.236923:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4902 00000100:00080000:3.0:1787828757.236925:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828757.236928:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828757.236936:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828757.236946:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f821c0 x1874669228338560/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828757.236956:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:1.0:1787828757.236961:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228338432:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787828757.236961:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828757.236963:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828757.236969:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4902 00000100:00080000:3.0:1787828757.236972:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828757.236975:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828757.236981:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828757.236998:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82580 x1874669228338688/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787828757.237047:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828757.237076:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828757/real 1787828757] req@ffff985610f83480 x1874669228338432/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828782 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828757.237090:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83480 x1874669228338432/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828782 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828757.237099:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828757.237102:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828757.237105:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228338432:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787828757.237125:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f821c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228338560:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828757.237140:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828757.237148:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828757/real 1787828757] req@ffff985610f821c0 x1874669228338560/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828782 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828757.237157:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f821c0 x1874669228338560/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828782 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828757.237163:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828757.237165:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828757.237168:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f821c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228338560:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787828757.237172:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228338688:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828757.237183:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828757.237189:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828757/real 1787828757] req@ffff985610f82580 x1874669228338688/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828782 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828757.237197:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82580 x1874669228338688/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828782 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828757.237203:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828757.237204:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828757.237207:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228338688:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828757.300820:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828758.196754:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787828758.324888:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828759.220844:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787828759.348875:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828760.244823:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828760.372898:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828761.268893:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828761.396866:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828762.292819:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787828762.356898:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828762.356911:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828762.356916:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828762.356929:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4907 00000100:00080000:3.0:1787828762.356954:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828762.356965:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828762.357021:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82940 x1874669228338816/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828762.357077:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828762.357082:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828762.357085:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828762.357090:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4907 00000100:00080000:3.0:1787828762.357095:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828762.357101:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828762.357113:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828762.357133:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82d00 x1874669228338944/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828762.357148:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828762.357153:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828762.357156:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828762.357162:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4907 00000100:00080000:3.0:1787828762.357167:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828762.357176:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787828762.357186:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228338816:192.168.203.114@tcp:250:kworker.0 00000008:00100000:3.0:1787828762.357190:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828762.357221:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81e00 x1874669228339072/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787828762.357297:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828762.357350:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828762/real 1787828762] req@ffff985610f82940 x1874669228338816/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828787 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828762.357380:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82940 x1874669228338816/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828787 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828762.357396:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828762.357402:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828762.357408:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228338816:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787828762.357421:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228338944:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828762.357469:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828762.357490:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828762/real 1787828762] req@ffff985610f82d00 x1874669228338944/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828787 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828762.357517:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82d00 x1874669228338944/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828787 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828762.357548:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828762.357554:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828762.357560:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228338944:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787828762.357617:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228339072:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828762.357650:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828762.357666:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828762/real 1787828762] req@ffff985610f81e00 x1874669228339072/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828787 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828762.357681:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81e00 x1874669228339072/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828787 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828762.357692:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828762.357697:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828762.357700:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228339072:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828762.421061:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828763.316807:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787828763.444946:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828764.340852:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787828764.468858:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828765.364794:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828765.492809:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828766.388830:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828766.516834:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828767.412836:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787828767.476795:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828767.476806:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828767.476815:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828767.476826:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4913 00000100:00080000:0.0:1787828767.476839:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828767.476849:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787828767.476885:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7840 x1874669228339200/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828767.476913:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828767.476916:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828767.476935:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828767.476939:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4913 00000100:00080000:0.0:1787828767.476944:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828767.476949:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828767.476961:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828767.476971:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228339200:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787828767.476981:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df52c0 x1874669228339328/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828767.477021:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828767.477025:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828767.477027:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828767.477032:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4913 00000100:00080000:0.0:1787828767.477037:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828767.477041:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828767.477046:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828767.477072:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7480 x1874669228339456/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828767.477084:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828767.477127:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828767/real 1787828767] req@ffff985607df7840 x1874669228339200/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828792 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828767.477205:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7840 x1874669228339200/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828792 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828767.477217:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828767.477222:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828767.477226:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228339200:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787828767.477254:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228339328:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787828767.477296:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828767.477307:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828767/real 1787828767] req@ffff985607df52c0 x1874669228339328/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828792 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828767.477317:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df52c0 x1874669228339328/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828792 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828767.477323:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828767.477444:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828767.477448:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228339328:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787828767.477456:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228339456:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787828767.477535:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828767.477561:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828767/real 1787828767] req@ffff985607df7480 x1874669228339456/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828792 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828767.477582:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7480 x1874669228339456/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828792 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828767.477591:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828767.477595:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828767.477597:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228339456:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828767.540952:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828768.436824:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787828768.564979:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828769.460774:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787828769.588886:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828770.484796:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828770.612809:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828771.508809:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828771.636884:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828772.532830:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787828772.596799:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828772.596816:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828772.596821:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828772.596850:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4918 00000100:00080000:3.0:1787828772.596865:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828772.596875:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828772.596924:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f803c0 x1874669228339584/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828772.596991:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828772.597002:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828772.597004:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828772.597010:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4918 00000100:00080000:3.0:1787828772.597015:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828772.597021:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828772.597049:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828772.597056:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228339584:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787828772.597069:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80f00 x1874669228339712/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828772.597083:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828772.597088:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828772.597090:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828772.597095:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4918 00000100:00080000:3.0:1787828772.597116:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828772.597121:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828772.597127:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828772.597142:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83840 x1874669228339840/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787828772.597152:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828772.597198:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828772/real 1787828772] req@ffff985610f803c0 x1874669228339584/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828797 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828772.597214:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f803c0 x1874669228339584/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828797 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828772.597227:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828772.597231:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828772.597235:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228339584:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787828772.597257:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228339712:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828772.597274:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828772.597283:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828772/real 1787828772] req@ffff985610f80f00 x1874669228339712/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828797 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828772.597292:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80f00 x1874669228339712/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828797 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828772.597298:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828772.597300:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828772.597303:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228339712:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787828772.597306:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228339840:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828772.597324:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828772.597331:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828772/real 1787828772] req@ffff985610f83840 x1874669228339840/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828797 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828772.597339:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83840 x1874669228339840/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828797 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828772.597344:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828772.597347:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828772.597349:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228339840:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828772.660909:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828773.556779:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787828773.684913:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828774.580924:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787828774.708880:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828775.604797:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828775.732862:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828776.628835:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828776.756808:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828777.652771:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787828777.716801:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828777.716812:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828777.716816:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828777.716827:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4923 00000100:00080000:3.0:1787828777.716839:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828777.716846:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828777.716905:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81a40 x1874669228339968/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828777.716938:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828777.716942:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828777.716945:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828777.716959:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4923 00000100:00080000:3.0:1787828777.716964:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828777.716969:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828777.716980:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828777.716993:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81680 x1874669228340096/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828777.717002:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828777.717005:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828777.717006:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828777.717009:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4923 00000100:00080000:3.0:1787828777.717012:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828777.717017:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787828777.717018:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228339968:192.168.203.114@tcp:250:kworker.0 00000008:00100000:3.0:1787828777.717036:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828777.717051:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f812c0 x1874669228340224/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787828777.717110:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828777.717152:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828777/real 1787828777] req@ffff985610f81a40 x1874669228339968/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828802 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828777.717167:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81a40 x1874669228339968/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828802 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828777.717176:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828777.717179:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828777.717182:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228339968:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787828777.717188:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228340096:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828777.717228:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828777.717237:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828777/real 1787828777] req@ffff985610f81680 x1874669228340096/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828802 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828777.717249:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81680 x1874669228340096/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828802 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828777.717258:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828777.717261:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828777.717265:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228340096:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787828777.717293:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228340224:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828777.717314:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828777.717322:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828777/real 1787828777] req@ffff985610f812c0 x1874669228340224/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828802 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828777.717331:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f812c0 x1874669228340224/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828802 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828777.717336:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828777.717339:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828777.717341:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228340224:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828777.780948:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828778.676820:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787828778.804837:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828779.700763:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787828779.828890:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828780.724819:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828780.852803:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828781.748793:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828781.876822:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828782.772833:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787828782.836854:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828782.836868:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828782.836874:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828782.836887:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4928 00000100:00080000:3.0:1787828782.836900:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828782.836910:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828782.836953:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80b40 x1874669228340352/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828782.836989:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828782.836993:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828782.836995:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828782.837000:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4928 00000100:00080000:3.0:1787828782.837003:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828782.837008:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828782.837019:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828782.837031:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80000 x1874669228340480/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828782.837044:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828782.837048:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828782.837050:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828782.837055:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4928 00000100:00080000:3.0:1787828782.837059:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828782.837064:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828782.837070:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828782.837073:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228340352:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787828782.837083:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f830c0 x1874669228340608/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787828782.837159:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828782.837201:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828782/real 1787828782] req@ffff985610f80b40 x1874669228340352/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828807 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828782.837216:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80b40 x1874669228340352/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828807 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828782.837225:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828782.837228:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828782.837231:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228340352:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787828782.837236:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228340480:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828782.837255:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828782.837263:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828782/real 1787828782] req@ffff985610f80000 x1874669228340480/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828807 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828782.837272:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80000 x1874669228340480/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828807 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828782.837277:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828782.837280:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828782.837282:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228340480:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787828782.837302:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228340608:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828782.837313:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828782.837319:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828782/real 1787828782] req@ffff985610f830c0 x1874669228340608/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828807 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828782.837327:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f830c0 x1874669228340608/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828807 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828782.837332:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828782.837333:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828782.837335:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228340608:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828782.900870:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828783.796818:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787828783.924899:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828784.820809:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828784.948896:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828785.844828:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828785.972877:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828786.868847:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828786.996897:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828787.892825:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787828787.956763:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828787.956778:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828787.956782:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828787.956795:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4933 00000100:00080000:3.0:1787828787.956810:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828787.956819:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828787.956861:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83c00 x1874669228340736/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828787.956889:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828787.956891:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828787.956894:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828787.956897:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4933 00000100:00080000:3.0:1787828787.956900:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828787.956902:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828787.956911:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828787.956919:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80780 x1874669228340864/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828787.956926:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828787.956929:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828787.956930:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828787.956933:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4933 00000100:00080000:3.0:1787828787.956935:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828787.956938:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828787.956941:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828787.956950:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f830c0 x1874669228340992/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787828787.956966:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228340736:192.168.203.114@tcp:250:kworker.0 00000800:00000100:1.0:1787828787.957046:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828787.957090:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828787/real 1787828787] req@ffff985610f83c00 x1874669228340736/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828812 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828787.957105:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83c00 x1874669228340736/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828812 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828787.957114:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828787.957117:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828787.957121:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228340736:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787828787.957126:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228340864:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828787.957145:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828787.957153:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828787/real 1787828787] req@ffff985610f80780 x1874669228340864/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828812 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828787.957161:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80780 x1874669228340864/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828812 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828787.957167:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828787.957169:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828787.957171:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228340864:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787828787.957194:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228340992:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828787.957205:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828787.957212:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828787/real 1787828787] req@ffff985610f830c0 x1874669228340992/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828812 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828787.957219:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f830c0 x1874669228340992/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828812 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828787.957225:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828787.957226:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828787.957228:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228340992:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828788.020915:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828788.916809:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787828789.044858:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828789.940870:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828790.068835:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828790.964806:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828791.092925:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828791.988847:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828792.116862:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828793.012801:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787828793.076837:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828793.076850:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828793.076854:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828793.076864:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4938 00000100:00080000:3.0:1787828793.076876:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828793.076884:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828793.076921:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80000 x1874669228341120/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828793.076951:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828793.076954:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828793.076956:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828793.076959:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4938 00000100:00080000:3.0:1787828793.076962:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828793.076964:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828793.076972:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828793.076981:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80b40 x1874669228341248/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828793.076989:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828793.076992:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828793.076994:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828793.076997:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4938 00000100:00080000:3.0:1787828793.076999:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828793.077001:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828793.077005:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828793.077010:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228341120:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787828793.077016:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f812c0 x1874669228341376/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787828793.077088:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828793.077123:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828793/real 1787828793] req@ffff985610f80000 x1874669228341120/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828818 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828793.077138:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80000 x1874669228341120/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828818 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828793.077146:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828793.077149:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828793.077153:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228341120:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787828793.077158:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228341248:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828793.077175:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828793.077182:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828793/real 1787828793] req@ffff985610f80b40 x1874669228341248/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828818 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828793.077190:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80b40 x1874669228341248/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828818 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828793.077196:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828793.077198:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828793.077200:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228341248:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787828793.077222:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228341376:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828793.077236:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828793.077243:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828793/real 1787828793] req@ffff985610f812c0 x1874669228341376/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828818 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828793.077251:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f812c0 x1874669228341376/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828818 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828793.077256:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828793.077257:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828793.077259:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228341376:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828793.140994:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828794.036765:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787828794.164886:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828795.060783:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828795.188831:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828796.084817:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828796.212836:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828797.108779:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828797.236838:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828798.132786:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787828798.196754:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828798.196763:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828798.196766:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828798.196776:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4943 00000100:00080000:3.0:1787828798.196786:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828798.196792:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828798.196828:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81680 x1874669228341504/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828798.196852:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828798.196855:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828798.196857:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828798.196859:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4943 00000100:00080000:3.0:1787828798.196862:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828798.196864:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828798.196872:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828798.196882:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81a40 x1874669228341632/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828798.196897:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828798.196900:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828798.196902:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828798.196905:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4943 00000100:00100000:1.0:1787828798.196911:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228341504:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787828798.196911:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828798.196919:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828798.196923:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828798.196932:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83840 x1874669228341760/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787828798.196974:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828798.197003:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828798/real 1787828798] req@ffff985610f81680 x1874669228341504/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828823 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828798.197017:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81680 x1874669228341504/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828823 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828798.197026:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828798.197029:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828798.197033:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228341504:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787828798.197038:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228341632:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828798.197054:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828798.197063:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828798/real 1787828798] req@ffff985610f81a40 x1874669228341632/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828823 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828798.197072:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81a40 x1874669228341632/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828823 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828798.197078:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828798.197080:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828798.197083:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228341632:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787828798.197103:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228341760:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828798.197115:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828798.197122:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828798/real 1787828798] req@ffff985610f83840 x1874669228341760/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828823 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828798.197131:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83840 x1874669228341760/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828823 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828798.197137:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828798.197140:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828798.197144:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228341760:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828798.260975:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828799.156794:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828799.284962:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828800.180793:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828800.308815:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828801.204772:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828801.332849:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828802.228754:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828802.356869:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828803.252736:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787828803.316798:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828803.316806:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828803.316810:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828803.316819:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4948 00000100:00080000:0.0:1787828803.316829:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828803.316836:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787828803.316869:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4b40 x1874669228341888/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828803.316892:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828803.316896:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828803.316897:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828803.316900:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4948 00000100:00080000:0.0:1787828803.316909:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828803.316915:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828803.316923:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828803.316937:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228341888:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787828803.316940:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6940 x1874669228342016/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828803.316955:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828803.316958:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828803.316960:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828803.316965:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4948 00000100:00080000:0.0:1787828803.316970:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828803.316975:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828803.316981:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828803.316996:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4780 x1874669228342144/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828803.316999:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828803.317039:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828803/real 1787828803] req@ffff985607df4b40 x1874669228341888/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828828 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828803.317060:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4b40 x1874669228341888/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828828 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828803.317068:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828803.317072:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828803.317136:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228341888:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787828803.317158:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228342016:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787828803.317188:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828803.317215:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828803/real 1787828803] req@ffff985607df6940 x1874669228342016/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828828 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828803.317229:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6940 x1874669228342016/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828828 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828803.317235:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828803.317237:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828803.317239:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228342016:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787828803.317243:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228342144:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787828803.317256:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828803.317262:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828803/real 1787828803] req@ffff985607df4780 x1874669228342144/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828828 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828803.317270:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4780 x1874669228342144/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828828 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828803.317275:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828803.317277:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828803.317279:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228342144:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828803.380865:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828804.276816:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828804.404790:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828805.300771:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828805.428935:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828806.324838:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828806.452901:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828807.348854:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828807.476950:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828808.372775:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787828808.436843:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828808.436852:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828808.436856:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828808.436868:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4954 00000100:00080000:3.0:1787828808.436879:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828808.436886:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828808.436927:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80f00 x1874669228342272/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828808.436951:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828808.436954:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828808.436955:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828808.436959:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4954 00000100:00080000:3.0:1787828808.436961:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828808.436964:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787828808.436973:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228342272:192.168.203.114@tcp:250:kworker.0 00000008:00100000:3.0:1787828808.436973:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828808.436986:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f803c0 x1874669228342400/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828808.436994:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828808.436996:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828808.436998:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828808.437000:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4954 00000100:00080000:3.0:1787828808.437002:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828808.437005:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828808.437008:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828808.437017:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81e00 x1874669228342528/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787828808.437078:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228342400:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787828808.437102:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228342528:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828808.500867:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828809.396839:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828809.524845:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828810.420849:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828810.548906:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828811.444841:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000400:00000100:3.0:1787828811.509145:0:2371:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.114:988 00000400:00000100:3.0:1787828811.509209:0:2371:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.114@tcp at host 192.168.203.114 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:3.0:1787828811.509233:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:3.0:1787828811.509290:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:3.0:1787828811.509302:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000100:00100000:1.0:1787828811.509442:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828808/real 1787828811] req@ffff985610f803c0 x1874669228342400/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828833 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828811.509498:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f803c0 x1874669228342400/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828833 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828811.509531:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828811.509551:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828811.509559:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228342400:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787828811.509575:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828808/real 1787828811] req@ffff985610f81e00 x1874669228342528/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828833 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828811.509625:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81e00 x1874669228342528/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828833 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828811.509642:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828811.509650:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828811.509657:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228342528:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787828811.509686:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828808/real 1787828811] req@ffff985610f80f00 x1874669228342272/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828833 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828811.509770:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80f00 x1874669228342272/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828833 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828811.509787:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828811.509794:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828811.509801:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228342272:192.168.203.114@tcp:250:kworker.0 00010000:00010000:0.0:1787828811.572931:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828812.468785:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828812.597018:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828813.492814:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787828813.556835:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828813.556843:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828813.556847:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828813.556857:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4959 00000100:00080000:1.0:1787828813.556867:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828813.556872:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787828813.556905:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3c00 x1874669228342656/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828813.556929:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828813.556932:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828813.556933:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828813.556936:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4959 00000100:00080000:1.0:1787828813.556938:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828813.556941:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787828813.556949:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828813.556957:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a1680 x1874669228342784/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828813.556963:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828813.556966:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828813.556968:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828813.556971:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4959 00000100:00080000:1.0:1787828813.556973:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828813.556976:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787828813.556978:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828813.556984:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0780 x1874669228342912/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828813.556992:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228342656:192.168.203.114@tcp:250:kworker.0 00000800:00000100:3.0:1787828813.557088:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828813.557151:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828813/real 1787828813] req@ffff9856085a3c00 x1874669228342656/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828838 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828813.557171:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3c00 x1874669228342656/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828838 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828813.557183:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828813.557188:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828813.557192:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228342656:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787828813.557201:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228342784:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787828813.557227:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828813.557243:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828813/real 1787828813] req@ffff9856085a1680 x1874669228342784/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828838 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828813.557276:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a1680 x1874669228342784/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828838 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828813.557287:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828813.557290:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828813.557294:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228342784:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787828813.557352:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228342912:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787828813.557375:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828813.557403:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828813/real 1787828813] req@ffff9856085a0780 x1874669228342912/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828838 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828813.557417:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0780 x1874669228342912/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828838 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828813.557426:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828813.557429:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828813.557433:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228342912:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828813.620834:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828814.516816:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828814.644806:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828815.540795:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828815.668802:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828816.564757:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828816.692835:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828817.588813:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828817.716851:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828818.612772:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787828818.676824:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828818.676833:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828818.676838:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828818.676847:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4964 00000100:00080000:0.0:1787828818.676857:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828818.676864:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787828818.676915:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4000 x1874669228343040/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828818.676939:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828818.676942:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828818.676943:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828818.676946:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4964 00000100:00080000:0.0:1787828818.676949:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828818.676952:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828818.676960:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828818.676970:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7c00 x1874669228343168/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828818.676979:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828818.676982:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828818.676987:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1787828818.676988:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228343040:192.168.203.114@tcp:250:kworker.0 00000100:00080000:0.0:1787828818.676992:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4964 00000100:00080000:0.0:1787828818.676997:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828818.676999:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828818.677002:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828818.677013:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4780 x1874669228343296/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828818.677068:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828818.677103:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828818/real 1787828818] req@ffff985607df4000 x1874669228343040/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828843 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828818.677120:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4000 x1874669228343040/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828843 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828818.677148:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828818.677152:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828818.677156:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228343040:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787828818.677181:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228343168:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787828818.677217:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828818.677228:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828818/real 1787828818] req@ffff985607df7c00 x1874669228343168/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828843 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828818.677238:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7c00 x1874669228343168/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828843 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828818.677245:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828818.677248:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828818.677261:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228343168:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787828818.677265:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228343296:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787828818.677298:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828818.677308:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828818/real 1787828818] req@ffff985607df4780 x1874669228343296/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828843 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828818.677320:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4780 x1874669228343296/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828843 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828818.677335:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828818.677338:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828818.677341:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228343296:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828818.740905:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828819.636857:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828819.764799:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828820.660765:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828820.788893:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828821.684849:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828821.812842:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828822.708788:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828822.836818:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828823.732818:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787828823.796803:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828823.796811:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828823.796815:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828823.796823:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4969 00000100:00080000:3.0:1787828823.796834:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828823.796840:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828823.796875:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82d00 x1874669228343424/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828823.796900:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828823.796902:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828823.796904:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828823.796906:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4969 00000100:00080000:3.0:1787828823.796909:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828823.796911:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828823.796918:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828823.796927:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82940 x1874669228343552/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828823.796935:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828823.796937:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828823.796939:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828823.796941:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4969 00000100:00080000:3.0:1787828823.796943:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828823.796945:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828823.796948:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828823.796954:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82580 x1874669228343680/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787828823.796964:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228343424:192.168.203.114@tcp:250:kworker.0 00000800:00000100:1.0:1787828823.797036:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828823.797071:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828823/real 1787828823] req@ffff985610f82d00 x1874669228343424/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828848 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828823.797091:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82d00 x1874669228343424/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828848 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828823.797103:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828823.797108:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828823.797112:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228343424:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787828823.797120:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228343552:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828823.797144:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828823.797158:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828823/real 1787828823] req@ffff985610f82940 x1874669228343552/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828848 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828823.797171:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82940 x1874669228343552/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828848 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828823.797180:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828823.797184:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828823.797188:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228343552:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787828823.797213:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228343680:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828823.797236:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828823.797248:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828823/real 1787828823] req@ffff985610f82580 x1874669228343680/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828848 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828823.797261:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82580 x1874669228343680/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828848 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828823.797270:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828823.797274:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828823.797277:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228343680:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828823.860888:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828824.756811:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828824.884826:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828825.780761:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828825.908863:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828826.804834:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828826.932834:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828827.828798:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828827.956921:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828828.852806:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787828828.916786:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828828.916800:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828828.916805:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828828.916846:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4974 00000100:00080000:3.0:1787828828.916859:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828828.916867:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828828.916940:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f821c0 x1874669228343808/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828828.916981:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828828.916984:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828828.916987:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828828.916990:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4974 00000100:00080000:3.0:1787828828.916993:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828828.916997:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828828.917006:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828828.917017:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83480 x1874669228343936/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828828.917040:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828828.917044:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828828.917047:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828828.917051:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4974 00000100:00080000:3.0:1787828828.917055:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00100000:1.0:1787828828.917066:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f821c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228343808:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787828828.917074:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828828.917085:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828828.917115:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82580 x1874669228344064/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787828828.917165:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828828.917196:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828828/real 1787828828] req@ffff985610f821c0 x1874669228343808/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828853 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828828.917211:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f821c0 x1874669228343808/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828853 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828828.917223:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828828.917226:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828828.917229:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f821c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228343808:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787828828.917264:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228343936:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828828.917303:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828828.917313:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828828/real 1787828828] req@ffff985610f83480 x1874669228343936/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828853 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828828.917339:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83480 x1874669228343936/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828853 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828828.917345:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828828.917348:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828828.917350:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228343936:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787828828.917371:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228344064:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828828.917390:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828828.917402:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828828/real 1787828828] req@ffff985610f82580 x1874669228344064/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828853 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828828.917416:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82580 x1874669228344064/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828853 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828828.917424:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828828.917428:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828828.917431:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228344064:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828828.980875:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828829.876756:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828830.004848:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828830.900816:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828831.028862:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828831.924768:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828832.052878:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828832.948854:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828833.076906:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828833.972755:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787828834.036739:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828834.036749:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828834.036754:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828834.036764:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4979 00000100:00080000:3.0:1787828834.036775:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828834.036783:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828834.036820:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82940 x1874669228344192/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828834.036848:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828834.036851:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828834.036853:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828834.036856:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4979 00000100:00080000:3.0:1787828834.036858:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828834.036860:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828834.036869:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828834.036878:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82d00 x1874669228344320/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828834.036885:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828834.036887:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828834.036889:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828834.036891:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4979 00000100:00080000:3.0:1787828834.036893:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828834.036895:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828834.036897:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828834.036903:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80f00 x1874669228344448/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787828834.036947:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228344192:192.168.203.114@tcp:250:kworker.0 00000800:00000100:1.0:1787828834.037031:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828834.037082:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828834/real 1787828834] req@ffff985610f82940 x1874669228344192/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828859 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828834.037100:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82940 x1874669228344192/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828859 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828834.037112:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828834.037115:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828834.037119:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228344192:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787828834.037125:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228344320:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828834.037145:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828834.037155:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828834/real 1787828834] req@ffff985610f82d00 x1874669228344320/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828859 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828834.037165:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82d00 x1874669228344320/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828859 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828834.037172:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828834.037174:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828834.037176:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228344320:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787828834.037181:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228344448:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828834.037194:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828834.037201:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828834/real 1787828834] req@ffff985610f80f00 x1874669228344448/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828859 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828834.037210:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80f00 x1874669228344448/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828859 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828834.037217:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828834.037219:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828834.037221:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228344448:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828834.100921:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828834.996826:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828835.124826:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828836.020778:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828836.148918:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828837.044840:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828837.172807:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828838.068796:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828838.196889:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828839.092828:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787828839.156755:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828839.156766:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828839.156769:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828839.156780:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4984 00000100:00080000:3.0:1787828839.156791:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828839.156799:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828839.156835:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81e00 x1874669228344576/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828839.156875:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828839.156879:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828839.156882:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828839.156888:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4984 00000100:00080000:3.0:1787828839.156891:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828839.156895:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828839.156905:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828839.156908:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228344576:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787828839.156922:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f803c0 x1874669228344704/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828839.156937:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828839.156941:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828839.156945:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828839.156952:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4984 00000100:00080000:3.0:1787828839.156959:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828839.156966:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828839.156972:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000800:00000100:0.0:1787828839.156989:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828839.156990:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83840 x1874669228344832/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787828839.157028:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828839/real 1787828839] req@ffff985610f81e00 x1874669228344576/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828864 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828839.157044:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81e00 x1874669228344576/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828864 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828839.157057:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828839.157060:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828839.157064:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228344576:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787828839.157096:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228344704:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787828839.157112:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828839.157122:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828839/real 1787828839] req@ffff985610f803c0 x1874669228344704/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828864 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828839.157131:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f803c0 x1874669228344704/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828864 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828839.157137:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828839.157140:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828839.157142:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228344704:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787828839.157145:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228344832:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787828839.157157:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828839.157163:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828839/real 1787828839] req@ffff985610f83840 x1874669228344832/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828864 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828839.157170:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83840 x1874669228344832/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828864 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828839.157176:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828839.157178:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828839.157179:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228344832:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828839.220807:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828840.116760:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828840.244830:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828841.140784:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828841.268859:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828842.164834:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828842.292838:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828843.188809:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828843.317051:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828844.212797:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787828844.276781:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828844.276791:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828844.276796:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828844.276806:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4989 00000100:00080000:3.0:1787828844.276817:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828844.276823:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828844.276863:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81a40 x1874669228344960/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828844.276887:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828844.276890:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828844.276891:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828844.276894:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4989 00000100:00080000:3.0:1787828844.276898:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828844.276902:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828844.276916:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828844.276929:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81680 x1874669228345088/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787828844.276935:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228344960:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787828844.276944:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828844.276948:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828844.276950:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828844.276954:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4989 00000100:00080000:3.0:1787828844.276956:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828844.276959:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828844.276965:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828844.276976:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f812c0 x1874669228345216/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787828844.277022:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828844.277061:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828844/real 1787828844] req@ffff985610f81a40 x1874669228344960/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828869 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828844.277076:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81a40 x1874669228344960/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828869 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828844.277085:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828844.277089:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828844.277092:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228344960:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787828844.277115:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228345088:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787828844.277132:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828844.277140:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828844/real 1787828844] req@ffff985610f81680 x1874669228345088/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828869 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828844.277148:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81680 x1874669228345088/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828869 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828844.277154:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828844.277157:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828844.277159:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228345088:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787828844.277165:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228345216:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787828844.277197:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787828844.277205:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828844/real 1787828844] req@ffff985610f812c0 x1874669228345216/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828869 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828844.277213:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f812c0 x1874669228345216/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828869 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828844.277219:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828844.277221:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787828844.277223:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228345216:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828844.340901:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828845.236861:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828845.364894:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828846.260776:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828846.388944:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828847.284802:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828847.412881:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828848.308817:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828848.436874:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828849.332798:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787828849.396872:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828849.396886:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828849.396892:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828849.396904:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 4995 00000100:00080000:0.0:1787828849.396918:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828849.396936:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787828849.396982:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6940 x1874669228345344/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828849.397018:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828849.397023:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828849.397025:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828849.397030:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4995 00000100:00080000:0.0:1787828849.397034:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828849.397039:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828849.397051:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828849.397067:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4b40 x1874669228345472/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787828849.397074:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228345344:192.168.203.114@tcp:250:kworker.0 00000100:00080000:0.0:1787828849.397086:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828849.397090:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828849.397092:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828849.397097:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 4995 00000100:00080000:0.0:1787828849.397101:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828849.397105:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828849.397111:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828849.397124:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7480 x1874669228345600/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787828849.397182:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828849.397224:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828849/real 1787828849] req@ffff985607df6940 x1874669228345344/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828874 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828849.397241:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6940 x1874669228345344/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828874 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828849.397250:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828849.397254:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828849.397273:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228345344:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787828849.397295:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228345472:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828849.397312:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828849.397321:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828849/real 1787828849] req@ffff985607df4b40 x1874669228345472/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828874 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828849.397328:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4b40 x1874669228345472/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828874 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828849.397335:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828849.397337:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828849.397339:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228345472:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787828849.397343:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228345600:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828849.397354:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828849.397360:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828849/real 1787828849] req@ffff985607df7480 x1874669228345600/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828874 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828849.397368:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7480 x1874669228345600/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828874 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828849.397373:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828849.397374:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828849.397376:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228345600:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828849.460933:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828850.356819:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828850.484802:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828851.380767:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828851.508850:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828852.404836:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828852.532909:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828853.428787:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828853.556980:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828854.452819:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787828854.516836:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828854.516845:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828854.516848:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828854.516857:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5000 00000100:00080000:3.0:1787828854.516867:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828854.516872:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828854.516913:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80b40 x1874669228345728/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828854.516935:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828854.516939:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828854.516941:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828854.516945:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5000 00000100:00080000:3.0:1787828854.516949:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828854.516954:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828854.516975:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828854.516988:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80000 x1874669228345856/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787828854.516995:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228345728:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787828854.516999:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828854.517002:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828854.517003:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828854.517007:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5000 00000100:00080000:3.0:1787828854.517011:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828854.517014:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828854.517018:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828854.517027:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f830c0 x1874669228345984/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787828854.517057:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828854.517090:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828854/real 1787828854] req@ffff985610f80b40 x1874669228345728/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828879 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828854.517117:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80b40 x1874669228345728/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828879 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828854.517125:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828854.517128:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828854.517131:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228345728:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787828854.517161:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228345856:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828854.517178:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828854.517187:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828854/real 1787828854] req@ffff985610f80000 x1874669228345856/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828879 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828854.517200:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80000 x1874669228345856/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828879 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828854.517205:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828854.517207:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828854.517210:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228345856:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787828854.517213:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228345984:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828854.517223:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828854.517232:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828854/real 1787828854] req@ffff985610f830c0 x1874669228345984/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828879 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828854.517245:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f830c0 x1874669228345984/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828879 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828854.517253:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828854.517256:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828854.517260:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228345984:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828854.580817:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828855.476820:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828855.604852:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828856.500829:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828856.628810:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828857.524773:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828857.652811:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828858.548792:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828858.676815:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828859.572882:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787828859.636830:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828859.636845:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828859.636851:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828859.636879:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5005 00000100:00080000:3.0:1787828859.636896:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828859.636906:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828859.636987:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80780 x1874669228346112/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828859.637020:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828859.637024:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828859.637025:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828859.637041:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5005 00000100:00080000:3.0:1787828859.637045:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828859.637048:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828859.637059:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828859.637091:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83c00 x1874669228346240/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828859.637121:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:1.0:1787828859.637128:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228346112:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787828859.637128:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828859.637132:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828859.637144:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5005 00000100:00080000:3.0:1787828859.637149:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828859.637153:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828859.637159:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828859.637174:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f830c0 x1874669228346368/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787828859.637236:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828859.637279:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828859/real 1787828859] req@ffff985610f80780 x1874669228346112/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828884 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828859.637298:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80780 x1874669228346112/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828884 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828859.637309:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828859.637313:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828859.637317:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228346112:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787828859.637373:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228346240:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828859.637412:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828859.637429:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828859/real 1787828859] req@ffff985610f83c00 x1874669228346240/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828884 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828859.637446:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83c00 x1874669228346240/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828884 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828859.637477:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828859.637482:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828859.637488:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228346240:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787828859.637497:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228346368:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828859.637522:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828859.637534:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828859/real 1787828859] req@ffff985610f830c0 x1874669228346368/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828884 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828859.637545:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f830c0 x1874669228346368/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828884 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828859.637553:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828859.637557:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828859.637560:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228346368:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828859.700968:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828860.596832:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828860.724916:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828861.620807:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828861.748976:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828862.644847:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828862.772893:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828863.668823:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828863.796804:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828864.692783:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787828864.756801:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828864.756810:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828864.756813:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828864.756822:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5010 00000100:00080000:3.0:1787828864.756831:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828864.756837:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828864.756872:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80000 x1874669228346496/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828864.756897:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828864.756900:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828864.756902:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828864.756905:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5010 00000100:00080000:3.0:1787828864.756907:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828864.756910:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828864.756918:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828864.756928:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80b40 x1874669228346624/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828864.756938:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828864.756941:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828864.756945:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1787828864.756947:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228346496:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787828864.756951:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5010 00000100:00080000:3.0:1787828864.756958:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828864.756961:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828864.756964:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828864.756987:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f812c0 x1874669228346752/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787828864.757072:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828864.757099:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828864/real 1787828864] req@ffff985610f80000 x1874669228346496/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828889 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828864.757112:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80000 x1874669228346496/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828889 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828864.757120:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828864.757124:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828864.757129:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228346496:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787828864.758209:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228346624:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828864.758232:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828864.758243:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828864/real 1787828864] req@ffff985610f80b40 x1874669228346624/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828889 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828864.758252:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80b40 x1874669228346624/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828889 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828864.758269:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828864.758271:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828864.758274:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228346624:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787828864.758278:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228346752:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828864.758289:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828864.758295:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828864/real 1787828864] req@ffff985610f812c0 x1874669228346752/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828889 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828864.758303:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f812c0 x1874669228346752/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828889 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828864.758308:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828864.758310:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828864.758312:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228346752:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828864.820885:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828865.716883:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828865.844958:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828866.740777:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828866.868823:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828867.764815:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828867.892864:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828868.788763:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828868.916874:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828869.812759:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787828869.876886:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828869.876896:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828869.876901:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828869.876911:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5015 00000100:00080000:0.0:1787828869.876923:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828869.876930:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787828869.876974:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df52c0 x1874669228346880/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828869.877022:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828869.877042:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828869.877045:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828869.877050:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5015 00000100:00080000:0.0:1787828869.877055:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828869.877060:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828869.877073:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828869.877106:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7840 x1874669228347008/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828869.877121:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828869.877126:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:1.0:1787828869.877126:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228346880:192.168.203.114@tcp:250:kworker.0 00000100:00080000:0.0:1787828869.877132:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828869.877143:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5015 00000100:00080000:0.0:1787828869.877148:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828869.877152:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828869.877185:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828869.877201:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4f00 x1874669228347136/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787828869.877240:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828869.877287:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828869/real 1787828869] req@ffff985607df52c0 x1874669228346880/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828894 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828869.877309:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df52c0 x1874669228346880/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828894 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828869.877321:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828869.877338:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828869.877343:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228346880:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787828869.877372:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228347008:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828869.877399:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828869.877419:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828869/real 1787828869] req@ffff985607df7840 x1874669228347008/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828894 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828869.877434:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7840 x1874669228347008/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828894 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828869.877444:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828869.877447:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828869.877451:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228347008:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787828869.877458:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228347136:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828869.877481:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828869.877494:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828869/real 1787828869] req@ffff985607df4f00 x1874669228347136/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828894 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828869.877507:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4f00 x1874669228347136/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828894 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828869.877516:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828869.877519:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828869.877523:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228347136:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828869.940855:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828870.836768:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828870.964844:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828871.860816:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828871.988893:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828872.884769:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828873.012903:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828873.908812:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828874.036808:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828874.932789:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787828874.996860:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828874.996872:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828874.996878:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828874.996894:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5020 00000100:00080000:0.0:1787828874.996911:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828874.996922:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787828874.996988:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df61c0 x1874669228347264/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828874.997023:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828874.997028:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828874.997030:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828874.997037:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5020 00000100:00080000:0.0:1787828874.997043:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828874.997049:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828874.997063:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828874.997084:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6d00 x1874669228347392/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828874.997084:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228347264:192.168.203.114@tcp:250:kworker.0 00000100:00080000:0.0:1787828874.997111:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828874.997125:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828874.997127:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828874.997134:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5020 00000100:00080000:0.0:1787828874.997138:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828874.997142:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828874.997146:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828874.997157:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5e00 x1874669228347520/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828874.997464:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228347392:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787828874.997502:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228347520:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828875.060901:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828875.956801:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828876.084801:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828876.980846:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828877.108819:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828878.004805:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000400:00000100:1.0:1787828878.069090:0:2372:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.114:988 00000400:00000100:1.0:1787828878.069147:0:2372:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.114@tcp at host 192.168.203.114 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1787828878.069169:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:1.0:1787828878.069225:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:1.0:1787828878.069240:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000100:00100000:3.0:1787828878.069296:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828874/real 1787828878] req@ffff985607df6d00 x1874669228347392/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828899 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828878.069351:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6d00 x1874669228347392/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828899 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828878.069376:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828878.069384:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828878.069391:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228347392:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787828878.069405:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828874/real 1787828878] req@ffff985607df5e00 x1874669228347520/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828899 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828878.069424:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5e00 x1874669228347520/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828899 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828878.069436:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828878.069441:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828878.069446:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228347520:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787828878.069456:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828874/real 1787828878] req@ffff985607df61c0 x1874669228347264/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828899 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828878.069474:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df61c0 x1874669228347264/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828899 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828878.069485:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828878.069491:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828878.069496:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228347264:192.168.203.114@tcp:250:kworker.0 00010000:00010000:0.0:1787828878.132872:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828879.028828:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828879.156808:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828880.052861:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787828880.116802:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828880.116810:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828880.116814:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828880.116823:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5025 00000100:00080000:0.0:1787828880.116833:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828880.116839:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787828880.116870:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df43c0 x1874669228347648/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828880.116895:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828880.116899:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828880.116900:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828880.116903:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5025 00000100:00080000:0.0:1787828880.116906:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828880.116909:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828880.116929:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828880.116943:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5680 x1874669228347776/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828880.116953:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828880.116955:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828880.116957:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828880.116960:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5025 00000100:00080000:0.0:1787828880.116962:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828880.116965:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828880.116968:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828880.116975:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df70c0 x1874669228347904/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828880.117002:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228347648:192.168.203.114@tcp:250:kworker.0 00000800:00000100:3.0:1787828880.117108:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828880.117188:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828880/real 1787828880] req@ffff985607df43c0 x1874669228347648/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828905 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828880.117213:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df43c0 x1874669228347648/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828905 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828880.117234:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828880.117240:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828880.117246:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228347648:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787828880.117256:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228347776:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787828880.117292:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828880.117308:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828880/real 1787828880] req@ffff985607df5680 x1874669228347776/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828905 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828880.117322:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5680 x1874669228347776/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828905 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828880.117330:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828880.117334:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828880.117338:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228347776:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787828880.117392:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228347904:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787828880.117419:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828880.117433:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828880/real 1787828880] req@ffff985607df70c0 x1874669228347904/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828905 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828880.117446:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df70c0 x1874669228347904/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828905 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828880.117455:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828880.117459:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828880.117463:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228347904:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828880.180853:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828881.076740:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828881.204841:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828882.100896:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828882.228773:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828883.124832:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828883.252817:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828884.148785:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828884.276805:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828885.172780:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787828885.236790:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828885.236798:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828885.236801:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828885.236810:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5030 00000100:00080000:2.0:1787828885.236818:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828885.236823:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787828885.236856:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140da1c0 x1874669228348032/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828885.236879:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828885.236892:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828885.236894:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828885.236898:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5030 00000100:00080000:2.0:1787828885.236900:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828885.236903:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787828885.236911:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828885.236922:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140dbc00 x1874669228348160/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828885.236934:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828885.236938:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:3.0:1787828885.236940:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228348032:192.168.203.114@tcp:250:kworker.0 00000100:00080000:2.0:1787828885.236941:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828885.236949:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5030 00000100:00080000:2.0:1787828885.236952:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828885.236954:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787828885.236957:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828885.236966:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db840 x1874669228348288/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828885.237002:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828885.237030:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828885/real 1787828885] req@ffff9856140da1c0 x1874669228348032/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828910 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828885.237044:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140da1c0 x1874669228348032/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828910 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828885.237052:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828885.237054:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828885.237057:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228348032:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787828885.237077:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228348160:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787828885.237091:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828885.237099:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828885/real 1787828885] req@ffff9856140dbc00 x1874669228348160/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828910 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828885.237107:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140dbc00 x1874669228348160/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828910 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828885.237112:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828885.237114:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828885.237116:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228348160:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787828885.237120:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228348288:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787828885.237131:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828885.237137:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828885/real 1787828885] req@ffff9856140db840 x1874669228348288/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828910 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828885.237144:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db840 x1874669228348288/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828910 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828885.237148:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828885.237150:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828885.237152:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228348288:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828885.300788:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828886.196834:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828886.324802:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828887.220902:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828887.348821:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828888.244926:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828888.372931:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828889.268851:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828889.396872:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828890.292805:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787828890.356843:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828890.356863:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828890.356868:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828890.356879:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5035 00000100:00080000:0.0:1787828890.356892:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828890.356900:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787828890.356943:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6580 x1874669228348416/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828890.356982:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828890.356985:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828890.356986:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828890.356989:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5035 00000100:00080000:0.0:1787828890.356992:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828890.356994:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828890.357004:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828890.357015:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5a40 x1874669228348544/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828890.357023:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828890.357026:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828890.357028:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828890.357033:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5035 00000100:00080000:0.0:1787828890.357035:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787828890.357035:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228348416:192.168.203.114@tcp:250:kworker.0 00000100:00080000:0.0:1787828890.357042:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828890.357047:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828890.357058:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df70c0 x1874669228348672/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828890.357114:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828890.357168:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828890/real 1787828890] req@ffff985607df6580 x1874669228348416/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828915 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828890.357186:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6580 x1874669228348416/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828915 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828890.357195:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828890.357199:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828890.357202:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228348416:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787828890.357226:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228348544:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787828890.357247:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828890.357261:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828890/real 1787828890] req@ffff985607df5a40 x1874669228348544/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828915 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828890.357275:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5a40 x1874669228348544/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828915 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828890.357285:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828890.357289:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828890.357293:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228348544:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787828890.357298:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228348672:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787828890.357315:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828890.357323:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828890/real 1787828890] req@ffff985607df70c0 x1874669228348672/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828915 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828890.357331:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df70c0 x1874669228348672/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828915 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828890.357337:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828890.357340:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828890.357342:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228348672:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828890.420933:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828891.316824:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828891.444886:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828892.340778:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828892.468869:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828893.364791:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828893.492842:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828894.388791:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828894.516847:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828895.412823:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787828895.476870:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828895.476881:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828895.476886:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828895.476896:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5041 00000100:00080000:0.0:1787828895.476908:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828895.476915:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787828895.476971:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5680 x1874669228348800/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828895.477000:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828895.477003:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828895.477004:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828895.477007:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5041 00000100:00080000:0.0:1787828895.477010:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828895.477013:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828895.477038:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828895.477049:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df43c0 x1874669228348928/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828895.477057:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828895.477063:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828895.477064:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828895.477067:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5041 00000100:00080000:0.0:1787828895.477070:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828895.477073:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828895.477079:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228348800:192.168.203.114@tcp:250:kworker.0 00000008:00100000:0.0:1787828895.477082:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828895.477104:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df61c0 x1874669228349056/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828895.477181:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828895.477226:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828895/real 1787828895] req@ffff985607df5680 x1874669228348800/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828920 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828895.477249:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5680 x1874669228348800/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828920 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828895.477262:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828895.477265:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828895.477269:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228348800:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787828895.477276:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228348928:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787828895.477308:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828895.477319:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828895/real 1787828895] req@ffff985607df43c0 x1874669228348928/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828920 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828895.477327:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df43c0 x1874669228348928/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828920 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828895.477333:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828895.477336:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828895.477338:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228348928:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787828895.477374:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228349056:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787828895.477386:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828895.477393:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828895/real 1787828895] req@ffff985607df61c0 x1874669228349056/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828920 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828895.477402:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df61c0 x1874669228349056/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828920 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828895.477408:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828895.477410:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828895.477412:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228349056:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828895.540878:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828896.436823:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828896.564896:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828897.460778:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828897.588785:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828898.484870:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828898.612903:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828899.508851:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828899.636908:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828900.532842:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787828900.596759:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828900.596766:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828900.596769:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828900.596788:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5046 00000100:00080000:0.0:1787828900.596796:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828900.596802:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787828900.596833:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5e00 x1874669228349184/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828900.596855:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828900.596858:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828900.596859:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828900.596862:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5046 00000100:00080000:0.0:1787828900.596864:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828900.596867:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828900.596874:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828900.596883:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6d00 x1874669228349312/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828900.596891:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828900.596894:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828900.596897:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1787828900.596899:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228349184:192.168.203.114@tcp:250:kworker.0 00000100:00080000:0.0:1787828900.596902:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5046 00000100:00080000:0.0:1787828900.596909:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828900.596911:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828900.596920:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828900.596928:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4f00 x1874669228349440/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828900.596957:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828900.596992:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828900/real 1787828900] req@ffff985607df5e00 x1874669228349184/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828925 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828900.597013:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5e00 x1874669228349184/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828925 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828900.597022:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828900.597025:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828900.597028:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228349184:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787828900.597047:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228349312:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787828900.597065:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828900.597074:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828900/real 1787828900] req@ffff985607df6d00 x1874669228349312/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828925 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828900.597083:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6d00 x1874669228349312/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828925 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828900.597088:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828900.597090:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828900.597092:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228349312:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787828900.597096:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228349440:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787828900.597106:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828900.597112:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828900/real 1787828900] req@ffff985607df4f00 x1874669228349440/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828925 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828900.597119:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4f00 x1874669228349440/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828925 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828900.597124:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828900.597125:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828900.597127:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228349440:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828900.660864:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828901.556823:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828901.684869:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828902.580894:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828902.708842:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828903.604807:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828903.732917:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828904.628815:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828904.756909:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828905.652854:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787828905.716788:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828905.716799:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828905.716804:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828905.716813:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5051 00000100:00080000:0.0:1787828905.716824:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828905.716851:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787828905.716891:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7840 x1874669228349568/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828905.716946:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828905.716949:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828905.716951:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828905.716955:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5051 00000100:00080000:0.0:1787828905.716959:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828905.716962:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828905.716972:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828905.716986:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df52c0 x1874669228349696/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828905.716998:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828905.717006:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:3.0:1787828905.717006:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228349568:192.168.203.114@tcp:250:kworker.0 00000100:00080000:0.0:1787828905.717010:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828905.717021:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5051 00000100:00080000:0.0:1787828905.717025:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828905.717028:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828905.717032:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828905.717045:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7480 x1874669228349824/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828905.717087:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828905.717124:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828905/real 1787828905] req@ffff985607df7840 x1874669228349568/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828930 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828905.717143:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7840 x1874669228349568/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828930 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828905.717154:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828905.717158:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828905.717162:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228349568:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787828905.717200:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228349696:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787828905.717221:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828905.717233:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828905/real 1787828905] req@ffff985607df52c0 x1874669228349696/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828930 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828905.717245:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df52c0 x1874669228349696/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828930 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828905.717253:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828905.717256:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828905.717259:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228349696:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787828905.717264:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228349824:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787828905.717278:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828905.717287:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828905/real 1787828905] req@ffff985607df7480 x1874669228349824/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828930 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828905.717297:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7480 x1874669228349824/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828930 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828905.717311:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828905.717314:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828905.717316:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228349824:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828905.780934:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828906.676834:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828906.804825:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828907.700796:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828907.828886:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787828908.724801:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828908.852846:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828909.748818:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828909.876864:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828910.772852:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787828910.836870:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828910.836905:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828910.836913:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828910.836933:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5056 00000100:00080000:0.0:1787828910.836971:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828910.836982:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787828910.837025:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4b40 x1874669228349952/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828910.837050:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828910.837053:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828910.837055:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828910.837059:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5056 00000100:00080000:0.0:1787828910.837063:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828910.837067:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828910.837078:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828910.837085:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228349952:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787828910.837094:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6940 x1874669228350080/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828910.837104:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828910.837107:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828910.837108:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828910.837111:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5056 00000100:00080000:0.0:1787828910.837114:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828910.837134:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828910.837142:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828910.837158:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4780 x1874669228350208/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787828910.837239:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828910.837290:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828910/real 1787828910] req@ffff985607df4b40 x1874669228349952/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828935 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828910.837315:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4b40 x1874669228349952/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828935 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828910.837330:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828910.837344:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828910.837350:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228349952:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787828910.837382:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228350080:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828910.837416:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828910.837435:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828910/real 1787828910] req@ffff985607df6940 x1874669228350080/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828935 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828910.837452:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6940 x1874669228350080/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828935 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828910.837464:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828910.837469:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828910.837474:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228350080:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787828910.837482:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228350208:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828910.837514:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828910.837555:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828910/real 1787828910] req@ffff985607df4780 x1874669228350208/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828935 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828910.837583:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4780 x1874669228350208/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828935 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828910.837611:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828910.837617:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828910.837622:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228350208:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828910.900863:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828911.796771:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828911.924837:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828912.820802:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828912.948801:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828913.844766:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828913.972866:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828914.868807:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828914.996823:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828915.892809:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787828915.956853:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828915.956868:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828915.956874:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828915.956899:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5061 00000100:00080000:0.0:1787828915.956915:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828915.956925:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787828915.956979:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7c00 x1874669228350336/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828915.957011:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828915.957015:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828915.957017:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828915.957021:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5061 00000100:00080000:0.0:1787828915.957025:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828915.957029:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828915.957042:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828915.957056:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4000 x1874669228350464/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828915.957068:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828915.957072:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828915.957076:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1787828915.957077:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228350336:192.168.203.114@tcp:250:kworker.0 00000100:00080000:0.0:1787828915.957083:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5061 00000100:00080000:0.0:1787828915.957090:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828915.957095:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828915.957100:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828915.957114:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4780 x1874669228350592/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787828915.957175:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828915.957222:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828915/real 1787828915] req@ffff985607df7c00 x1874669228350336/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828940 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828915.957242:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7c00 x1874669228350336/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828940 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828915.957255:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828915.957260:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828915.957265:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228350336:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787828915.957292:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228350464:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828915.957318:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828915.957332:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828915/real 1787828915] req@ffff985607df4000 x1874669228350464/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828940 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828915.957353:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4000 x1874669228350464/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828940 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828915.957362:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828915.957366:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828915.957369:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228350464:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787828915.957375:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228350592:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828915.957396:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828915.957424:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828915/real 1787828915] req@ffff985607df4780 x1874669228350592/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828940 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828915.957440:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4780 x1874669228350592/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828940 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828915.957448:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828915.957452:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828915.957456:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228350592:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828916.020848:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828916.916817:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828917.044877:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828917.940850:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828918.068857:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828918.964809:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828919.092856:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828919.988869:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828920.116801:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828921.012813:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787828921.076807:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828921.076815:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828921.076818:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828921.076826:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5066 00000100:00080000:2.0:1787828921.076836:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828921.076841:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787828921.076873:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8f00 x1874669228350720/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828921.076898:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828921.076901:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828921.076902:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828921.076905:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5066 00000100:00080000:2.0:1787828921.076907:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828921.076909:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787828921.076918:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828921.076929:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140dad00 x1874669228350848/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828921.076936:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828921.076938:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828921.076940:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828921.076944:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5066 00000100:00080000:2.0:1787828921.076946:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828921.076948:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787828921.076951:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828921.076963:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db0c0 x1874669228350976/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787828921.076969:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228350720:192.168.203.114@tcp:250:kworker.0 00000800:00000100:1.0:1787828921.077052:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828921.077099:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828921/real 1787828921] req@ffff9856140d8f00 x1874669228350720/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828946 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828921.077115:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8f00 x1874669228350720/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828946 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828921.077127:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828921.077131:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828921.077134:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228350720:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787828921.077140:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228350848:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828921.077162:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828921.077177:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828921/real 1787828921] req@ffff9856140dad00 x1874669228350848/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828946 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828921.077193:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140dad00 x1874669228350848/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828946 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828921.077204:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828921.077208:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828921.077212:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228350848:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787828921.077262:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228350976:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828921.077284:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828921.077294:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828921/real 1787828921] req@ffff9856140db0c0 x1874669228350976/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828946 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828921.077304:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db0c0 x1874669228350976/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828946 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828921.077313:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828921.077317:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828921.077321:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228350976:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828921.140940:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828922.036864:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828922.164819:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828923.060817:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828923.188956:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828924.084778:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828924.212806:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828925.108785:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828925.236922:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828926.132788:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787828926.196813:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828926.196824:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828926.196829:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828926.196841:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5071 00000100:00080000:3.0:1787828926.196855:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828926.196863:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828926.196911:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81680 x1874669228351104/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828926.196941:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828926.196945:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828926.196947:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828926.196951:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5071 00000100:00080000:3.0:1787828926.196954:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828926.196957:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828926.196968:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828926.196981:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81a40 x1874669228351232/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787828926.196998:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228351104:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787828926.197001:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828926.197010:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828926.197013:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828926.197019:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5071 00000100:00080000:3.0:1787828926.197023:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828926.197028:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828926.197035:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828926.197055:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83840 x1874669228351360/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787828926.197089:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828926.197132:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828926/real 1787828926] req@ffff985610f81680 x1874669228351104/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828951 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828926.197160:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81680 x1874669228351104/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828951 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828926.197170:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828926.197174:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828926.197178:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228351104:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787828926.197205:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228351232:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828926.197225:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828926.197236:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828926/real 1787828926] req@ffff985610f81a40 x1874669228351232/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828951 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828926.197246:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81a40 x1874669228351232/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828951 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828926.197254:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828926.197257:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828926.197260:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228351232:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787828926.197264:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228351360:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828926.197278:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828926.197286:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828926/real 1787828926] req@ffff985610f83840 x1874669228351360/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828951 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828926.197296:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83840 x1874669228351360/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828951 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828926.197302:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828926.197304:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828926.197307:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228351360:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828926.260914:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828927.156784:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828927.284865:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828928.180819:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828928.308818:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828929.204774:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828929.332870:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828930.228815:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828930.356816:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828931.252791:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787828931.316929:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828931.316944:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828931.316950:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828931.316965:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5076 00000100:00080000:2.0:1787828931.316981:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828931.316993:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787828931.317045:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140da940 x1874669228351488/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828931.317092:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828931.317098:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828931.317100:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828931.317107:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5076 00000100:00080000:2.0:1787828931.317112:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828931.317118:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787828931.317131:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828931.317149:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8b40 x1874669228351616/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787828931.317160:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228351488:192.168.203.114@tcp:250:kworker.0 00000100:00080000:2.0:1787828931.317168:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828931.317174:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828931.317177:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828931.317184:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5076 00000100:00080000:2.0:1787828931.317189:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828931.317195:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787828931.317202:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828931.317221:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d9680 x1874669228351744/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787828931.317251:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828931.317297:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828931/real 1787828931] req@ffff9856140da940 x1874669228351488/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828956 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828931.317314:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140da940 x1874669228351488/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828956 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828931.317323:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828931.317326:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828931.317329:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228351488:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787828931.317351:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228351616:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828931.317371:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828931.317380:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828931/real 1787828931] req@ffff9856140d8b40 x1874669228351616/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828956 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828931.317388:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8b40 x1874669228351616/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828956 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828931.317395:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828931.317397:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828931.317399:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228351616:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787828931.317403:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228351744:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828931.317414:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828931.317420:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828931/real 1787828931] req@ffff9856140d9680 x1874669228351744/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828956 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828931.317428:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d9680 x1874669228351744/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828956 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828931.317433:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828931.317435:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828931.317437:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228351744:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828931.380856:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828932.276775:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828932.404845:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828933.300777:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828933.428872:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828934.324790:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828934.452879:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828935.348796:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828935.476872:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828936.372818:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787828936.436828:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828936.436837:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828936.436842:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828936.436852:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5082 00000100:00080000:2.0:1787828936.436864:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828936.436871:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787828936.436911:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d83c0 x1874669228351872/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828936.436942:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828936.436946:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828936.436948:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828936.436951:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5082 00000100:00080000:2.0:1787828936.436955:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828936.436958:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787828936.436968:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828936.436981:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d9a40 x1874669228352000/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828936.436992:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828936.436995:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828936.436997:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828936.437001:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5082 00000100:00080000:2.0:1787828936.437004:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828936.437008:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787828936.437012:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828936.437021:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db480 x1874669228352128/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787828936.437037:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228351872:192.168.203.114@tcp:250:kworker.0 00000800:00000100:1.0:1787828936.437144:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828936.437193:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828936/real 1787828936] req@ffff9856140d83c0 x1874669228351872/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828961 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828936.437217:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d83c0 x1874669228351872/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828961 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828936.437231:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828936.437236:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828936.437241:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228351872:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787828936.437251:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228352000:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828936.437278:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828936.437291:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828936/real 1787828936] req@ffff9856140d9a40 x1874669228352000/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828961 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828936.437300:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d9a40 x1874669228352000/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828961 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828936.437306:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828936.437309:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828936.437311:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228352000:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787828936.437338:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228352128:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828936.437352:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828936.437359:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828936/real 1787828936] req@ffff9856140db480 x1874669228352128/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828961 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828936.437366:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db480 x1874669228352128/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828961 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828936.437372:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828936.437374:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828936.437377:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228352128:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828936.500969:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828937.396882:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828937.524870:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828938.420828:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828938.548919:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828939.444858:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828939.572858:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828940.468890:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828940.596832:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828941.492850:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787828941.556832:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828941.556842:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828941.556847:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828941.556858:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5087 00000100:00080000:3.0:1787828941.556872:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828941.556880:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828941.556939:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f803c0 x1874669228352256/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828941.556967:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828941.556970:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828941.556971:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828941.556975:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5087 00000100:00080000:3.0:1787828941.556977:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828941.556980:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828941.556991:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828941.557001:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81e00 x1874669228352384/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828941.557008:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828941.557011:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828941.557012:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828941.557014:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5087 00000100:00080000:3.0:1787828941.557016:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828941.557018:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828941.557031:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828941.557037:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80f00 x1874669228352512/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787828941.557113:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228352256:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787828941.557264:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228352384:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787828941.557303:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228352512:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828941.620959:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828942.516810:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828942.644874:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828943.540809:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828943.668836:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828944.564837:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000400:00000100:3.0:1787828944.628985:0:2373:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.114:988 00000400:00000100:3.0:1787828944.629034:0:2373:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.114@tcp at host 192.168.203.114 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:3.0:1787828944.629053:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:3.0:1787828944.629101:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:3.0:1787828944.629116:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000100:00100000:1.0:1787828944.629169:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828941/real 1787828944] req@ffff985610f803c0 x1874669228352256/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828966 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828944.629212:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f803c0 x1874669228352256/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828966 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828944.629227:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828944.629234:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828944.629239:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228352256:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787828944.629249:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828941/real 1787828944] req@ffff985610f81e00 x1874669228352384/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828966 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828944.629265:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81e00 x1874669228352384/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828966 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828944.629276:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828944.629281:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828944.629285:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228352384:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787828944.629293:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828941/real 1787828944] req@ffff985610f80f00 x1874669228352512/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828966 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828944.629309:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80f00 x1874669228352512/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828966 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828944.629329:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828944.629335:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828944.629339:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228352512:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787828944.692905:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828945.588828:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828945.716839:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828946.612819:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787828946.676947:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828946.676958:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828946.676961:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828946.676973:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5092 00000100:00080000:3.0:1787828946.676985:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828946.676993:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828946.677032:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82d00 x1874669228352640/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828946.677073:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828946.677077:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828946.677080:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828946.677093:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5092 00000100:00080000:3.0:1787828946.677101:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828946.677107:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828946.677119:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828946.677138:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82940 x1874669228352768/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787828946.677139:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228352640:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787828946.677158:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828946.677163:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828946.677164:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828946.677179:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5092 00000100:00080000:3.0:1787828946.677184:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828946.677188:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828946.677194:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828946.677207:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82580 x1874669228352896/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787828946.677247:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828946.677295:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828946/real 1787828946] req@ffff985610f82d00 x1874669228352640/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828971 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828946.677318:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82d00 x1874669228352640/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828971 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828946.677332:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828946.677338:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828946.677343:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228352640:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787828946.677398:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228352768:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828946.677428:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828946.677444:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828946/real 1787828946] req@ffff985610f82940 x1874669228352768/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828971 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828946.677477:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82940 x1874669228352768/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828971 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828946.677488:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828946.677492:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828946.677496:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228352768:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787828946.677505:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228352896:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828946.677532:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828946.677545:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828946/real 1787828946] req@ffff985610f82580 x1874669228352896/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828971 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828946.677565:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82580 x1874669228352896/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828971 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828946.677573:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828946.677577:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828946.677580:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228352896:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828946.740950:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828947.636814:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828947.764781:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828948.660771:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828948.788875:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828949.684780:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828949.812854:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828950.708787:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828950.836872:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828951.732847:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787828951.796821:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828951.796832:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828951.796837:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828951.796847:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5097 00000100:00080000:3.0:1787828951.796860:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828951.796867:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828951.796908:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83480 x1874669228353024/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828951.796935:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828951.796938:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828951.796939:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828951.796942:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5097 00000100:00080000:3.0:1787828951.796945:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828951.796947:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828951.796956:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828951.796966:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f821c0 x1874669228353152/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828951.796973:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828951.796976:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828951.796978:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828951.796980:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5097 00000100:00080000:3.0:1787828951.796982:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828951.796985:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828951.796988:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828951.796992:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228353024:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787828951.796999:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82580 x1874669228353280/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787828951.797071:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828951.797115:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828951/real 1787828951] req@ffff985610f83480 x1874669228353024/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828976 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828951.797136:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83480 x1874669228353024/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828976 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828951.797148:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828951.797153:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828951.797158:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228353024:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787828951.797166:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f821c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228353152:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828951.797190:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828951.797201:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828951/real 1787828951] req@ffff985610f821c0 x1874669228353152/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828976 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828951.797210:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f821c0 x1874669228353152/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828976 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828951.797217:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828951.797219:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828951.797222:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f821c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228353152:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787828951.797244:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228353280:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828951.797256:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828951.797263:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828951/real 1787828951] req@ffff985610f82580 x1874669228353280/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828976 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828951.797277:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82580 x1874669228353280/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828976 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828951.797283:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828951.797285:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828951.797287:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228353280:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828951.860844:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828952.756789:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828952.884890:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828953.780789:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828953.908796:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828954.804791:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828954.932869:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828955.828820:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828955.956921:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828956.852827:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787828956.916811:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828956.916824:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828956.916829:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828956.916841:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5102 00000100:00080000:3.0:1787828956.916855:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828956.916874:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828956.916921:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82940 x1874669228353408/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828956.916953:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828956.916957:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828956.916958:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828956.916962:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5102 00000100:00080000:3.0:1787828956.916965:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828956.916968:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828956.916979:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828956.916991:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82d00 x1874669228353536/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828956.917002:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:1.0:1787828956.917009:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228353408:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787828956.917009:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828956.917014:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828956.917023:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5102 00000100:00080000:3.0:1787828956.917027:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828956.917029:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828956.917033:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828956.917043:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80f00 x1874669228353664/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787828956.917114:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828956.917167:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828956/real 1787828956] req@ffff985610f82940 x1874669228353408/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828981 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828956.917186:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82940 x1874669228353408/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828981 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828956.917198:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828956.917202:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828956.917206:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228353408:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787828956.917234:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228353536:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828956.917256:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828956.917267:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828956/real 1787828956] req@ffff985610f82d00 x1874669228353536/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828981 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828956.917278:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82d00 x1874669228353536/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828981 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828956.917286:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828956.917289:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828956.917292:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228353536:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787828956.917297:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228353664:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828956.917310:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828956.917318:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828956/real 1787828956] req@ffff985610f80f00 x1874669228353664/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828981 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828956.917328:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80f00 x1874669228353664/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828981 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828956.917335:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828956.917337:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828956.917339:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228353664:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828956.980888:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828957.876778:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828958.004883:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828958.900791:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828959.028832:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828959.924771:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828960.052862:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828960.948814:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828961.076814:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828961.972761:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787828962.036802:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828962.036829:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828962.036833:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828962.036844:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5107 00000100:00080000:3.0:1787828962.036857:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828962.036864:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828962.036920:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81e00 x1874669228353792/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828962.036946:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828962.036949:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828962.036951:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828962.036954:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5107 00000100:00080000:3.0:1787828962.036956:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828962.036959:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828962.036968:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828962.036978:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f803c0 x1874669228353920/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828962.036985:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828962.036987:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828962.036988:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828962.036991:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5107 00000100:00080000:3.0:1787828962.036993:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828962.036996:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828962.037001:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828962.037009:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83840 x1874669228354048/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787828962.037028:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228353792:192.168.203.114@tcp:250:kworker.0 00000800:00000100:1.0:1787828962.037100:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828962.037134:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828962/real 1787828962] req@ffff985610f81e00 x1874669228353792/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828987 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828962.037148:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81e00 x1874669228353792/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828987 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828962.037158:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828962.037161:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828962.037165:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228353792:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787828962.037169:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228353920:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828962.037183:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828962.037191:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828962/real 1787828962] req@ffff985610f803c0 x1874669228353920/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828987 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828962.037199:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f803c0 x1874669228353920/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828987 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828962.037204:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828962.037206:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828962.037208:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228353920:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787828962.037231:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228354048:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828962.037242:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828962.037249:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828962/real 1787828962] req@ffff985610f83840 x1874669228354048/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828987 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828962.037257:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83840 x1874669228354048/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828987 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828962.037262:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828962.037264:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828962.037265:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228354048:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828962.100890:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828962.996824:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828963.124838:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828964.020782:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828964.148886:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828965.044873:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828965.172874:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828966.068820:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828966.196815:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828967.092798:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787828967.156750:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828967.156758:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828967.156761:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828967.156769:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5112 00000100:00080000:3.0:1787828967.156778:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828967.156783:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828967.156813:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81a40 x1874669228354176/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828967.156835:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828967.156838:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828967.156839:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828967.156842:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5112 00000100:00080000:3.0:1787828967.156844:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828967.156846:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828967.156853:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828967.156860:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81680 x1874669228354304/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828967.156867:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828967.156870:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828967.156871:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828967.156873:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5112 00000100:00080000:3.0:1787828967.156876:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828967.156878:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828967.156881:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828967.156887:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f812c0 x1874669228354432/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787828967.156912:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228354176:192.168.203.114@tcp:250:kworker.0 00000800:00000100:1.0:1787828967.156966:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828967.156992:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828967/real 1787828967] req@ffff985610f81a40 x1874669228354176/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828992 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828967.157006:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81a40 x1874669228354176/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828992 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828967.157014:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828967.157017:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828967.157019:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228354176:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787828967.157024:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228354304:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828967.157038:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828967.157045:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828967/real 1787828967] req@ffff985610f81680 x1874669228354304/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828992 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828967.157053:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81680 x1874669228354304/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828992 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828967.157059:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828967.157061:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828967.157063:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228354304:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787828967.157066:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228354432:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828967.157076:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828967.157081:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828967/real 1787828967] req@ffff985610f812c0 x1874669228354432/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828992 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828967.157088:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f812c0 x1874669228354432/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828992 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828967.157093:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828967.157095:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828967.157097:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228354432:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828967.220879:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828968.116775:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828968.244838:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828969.140798:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828969.268797:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828970.164800:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828970.292907:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828971.188816:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828971.316854:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828972.212826:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787828972.276784:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828972.276795:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828972.276799:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828972.276810:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5117 00000100:00080000:1.0:1787828972.276833:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828972.276840:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787828972.276881:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a03c0 x1874669228354560/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828972.276908:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828972.276912:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828972.276913:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828972.276916:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5117 00000100:00080000:1.0:1787828972.276919:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828972.276922:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787828972.276932:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828972.276945:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3840 x1874669228354688/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828972.276946:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228354560:192.168.203.114@tcp:250:kworker.0 00000100:00080000:1.0:1787828972.276964:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828972.276969:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828972.276971:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828972.276975:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5117 00000100:00080000:1.0:1787828972.276989:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828972.276994:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787828972.276999:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828972.277013:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2580 x1874669228354816/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828972.277039:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828972.277086:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828972/real 1787828972] req@ffff9856085a03c0 x1874669228354560/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828997 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828972.277101:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a03c0 x1874669228354560/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828997 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828972.277110:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828972.277113:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828972.277117:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228354560:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787828972.277140:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228354688:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787828972.277156:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828972.277172:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828972/real 1787828972] req@ffff9856085a3840 x1874669228354688/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828997 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828972.277180:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3840 x1874669228354688/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828997 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828972.277186:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828972.277189:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828972.277192:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228354688:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787828972.277196:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228354816:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787828972.277207:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828972.277213:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828972/real 1787828972] req@ffff9856085a2580 x1874669228354816/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828997 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828972.277220:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2580 x1874669228354816/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828997 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828972.277225:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828972.277227:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828972.277229:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228354816:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828972.340888:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828973.236842:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828973.364854:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828974.260816:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828974.388836:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828975.284807:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828975.412823:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828976.308791:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828976.436819:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828977.332781:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787828977.396809:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828977.396824:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828977.396828:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828977.396838:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5123 00000100:00080000:1.0:1787828977.396849:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828977.396856:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787828977.396913:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a12c0 x1874669228354944/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828977.396958:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828977.396961:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828977.396963:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828977.396966:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5123 00000100:00080000:1.0:1787828977.396968:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828977.396971:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787828977.396980:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828977.396990:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a1a40 x1874669228355072/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828977.397005:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828977.397009:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:3.0:1787828977.397012:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228354944:192.168.203.114@tcp:250:kworker.0 00000100:00080000:1.0:1787828977.397013:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828977.397021:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5123 00000100:00080000:1.0:1787828977.397025:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828977.397027:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787828977.397031:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828977.397038:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2940 x1874669228355200/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828977.397086:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828977.397119:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828977/real 1787828977] req@ffff9856085a12c0 x1874669228354944/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829002 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828977.397152:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a12c0 x1874669228354944/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829002 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828977.397163:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828977.397167:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828977.397171:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228354944:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787828977.397194:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228355072:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787828977.397214:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828977.397226:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828977/real 1787828977] req@ffff9856085a1a40 x1874669228355072/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829002 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828977.397238:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a1a40 x1874669228355072/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829002 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828977.397269:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828977.397273:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828977.397276:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228355072:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787828977.397282:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228355200:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787828977.397302:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828977.397333:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828977/real 1787828977] req@ffff9856085a2940 x1874669228355200/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829002 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828977.397346:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2940 x1874669228355200/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829002 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828977.397355:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828977.397359:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828977.397362:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228355200:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828977.460914:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828978.356791:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828978.484816:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828979.380839:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828979.508883:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828980.404795:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828980.532814:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828981.428767:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828981.556804:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828982.452862:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787828982.516847:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828982.516860:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828982.516865:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828982.516879:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5128 00000100:00080000:0.0:1787828982.516892:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828982.516902:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787828982.516952:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6940 x1874669228355328/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828982.516980:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828982.516983:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828982.516984:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828982.516987:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5128 00000100:00080000:0.0:1787828982.516990:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828982.516993:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828982.517002:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828982.517013:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4b40 x1874669228355456/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828982.517023:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828982.517026:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828982.517028:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828982.517034:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5128 00000100:00100000:3.0:1787828982.517034:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228355328:192.168.203.114@tcp:250:kworker.0 00000100:00080000:0.0:1787828982.517056:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828982.517061:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828982.517064:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828982.517073:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7480 x1874669228355584/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828982.517141:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828982.517184:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828982/real 1787828982] req@ffff985607df6940 x1874669228355328/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829007 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828982.517254:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6940 x1874669228355328/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829007 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828982.517269:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828982.517275:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828982.517280:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228355328:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787828982.517329:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228355456:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787828982.517362:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828982.517383:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828982/real 1787828982] req@ffff985607df4b40 x1874669228355456/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829007 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828982.517394:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4b40 x1874669228355456/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829007 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828982.517402:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828982.517406:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828982.517409:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228355456:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787828982.517418:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228355584:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787828982.517449:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787828982.517462:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828982/real 1787828982] req@ffff985607df7480 x1874669228355584/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829007 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828982.517480:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7480 x1874669228355584/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829007 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828982.517554:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828982.517558:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787828982.517565:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228355584:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828982.580827:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828983.476780:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828983.604877:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828984.500797:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828984.628874:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828985.524774:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828985.652856:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828986.548821:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828986.676867:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828987.572762:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787828987.636862:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828987.636873:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828987.636878:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828987.636888:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5133 00000100:00080000:3.0:1787828987.636900:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828987.636908:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828987.636950:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80b40 x1874669228355712/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828987.636988:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828987.636992:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828987.636995:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828987.636999:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5133 00000100:00080000:3.0:1787828987.637004:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828987.637008:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828987.637021:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828987.637035:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80000 x1874669228355840/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828987.637043:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828987.637046:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828987.637048:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828987.637054:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5133 00000100:00080000:3.0:1787828987.637057:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828987.637060:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828987.637066:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828987.637070:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228355712:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787828987.637077:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f830c0 x1874669228355968/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787828987.637158:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828987.637211:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828987/real 1787828987] req@ffff985610f80b40 x1874669228355712/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829012 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828987.637225:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80b40 x1874669228355712/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829012 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828987.637235:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828987.637238:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828987.637241:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228355712:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787828987.637247:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228355840:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828987.637264:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828987.637273:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828987/real 1787828987] req@ffff985610f80000 x1874669228355840/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829012 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828987.637281:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80000 x1874669228355840/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829012 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828987.637286:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828987.637288:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828987.637291:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228355840:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787828987.637311:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228355968:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828987.637322:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828987.637328:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828987/real 1787828987] req@ffff985610f830c0 x1874669228355968/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829012 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828987.637340:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f830c0 x1874669228355968/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829012 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828987.637345:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828987.637347:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828987.637349:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228355968:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828987.700926:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828988.596819:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828988.724875:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828989.620796:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828989.748837:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828990.644839:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828990.772778:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828991.668799:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828991.796858:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828992.692812:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787828992.756816:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828992.756824:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828992.756827:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828992.756834:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5138 00000100:00080000:0.0:1787828992.756842:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828992.756848:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787828992.756880:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df52c0 x1874669228356096/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828992.756907:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828992.756910:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828992.756912:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828992.756914:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5138 00000100:00080000:0.0:1787828992.756917:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828992.756919:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828992.756927:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828992.756935:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7840 x1874669228356224/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828992.756943:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828992.756946:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828992.756947:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828992.756950:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5138 00000100:00080000:0.0:1787828992.756952:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828992.756954:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787828992.756957:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828992.756963:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4f00 x1874669228356352/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787828992.756969:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228356096:192.168.203.114@tcp:250:kworker.0 00000800:00000100:1.0:1787828992.757034:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828992.757065:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828992/real 1787828992] req@ffff985607df52c0 x1874669228356096/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829017 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828992.757082:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df52c0 x1874669228356096/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829017 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828992.757093:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828992.757098:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828992.757102:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228356096:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787828992.757110:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228356224:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828992.757157:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828992.757172:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828992/real 1787828992] req@ffff985607df7840 x1874669228356224/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829017 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828992.757186:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7840 x1874669228356224/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829017 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828992.757202:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828992.757206:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828992.757210:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228356224:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787828992.757234:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228356352:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828992.757258:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828992.757271:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828992/real 1787828992] req@ffff985607df4f00 x1874669228356352/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829017 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828992.757286:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4f00 x1874669228356352/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829017 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828992.757296:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828992.757299:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828992.757303:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228356352:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828992.820846:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828993.716856:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828993.844813:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828994.740840:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828994.868841:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828995.764815:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828995.892851:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828996.788789:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828996.916879:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828997.812793:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787828997.876794:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828997.876804:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828997.876814:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828997.876828:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5143 00000100:00080000:3.0:1787828997.876841:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828997.876851:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787828997.876897:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83c00 x1874669228356480/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828997.876924:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828997.876927:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828997.876928:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828997.876931:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5143 00000100:00080000:3.0:1787828997.876934:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828997.876937:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787828997.876946:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828997.876956:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80780 x1874669228356608/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828997.876964:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828997.876967:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828997.876969:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828997.876971:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5143 00000100:00080000:3.0:1787828997.876975:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828997.876980:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787828997.876983:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228356480:192.168.203.114@tcp:250:kworker.0 00000008:00100000:3.0:1787828997.876984:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828997.876997:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f830c0 x1874669228356736/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787828997.877069:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828997.877123:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828997/real 1787828997] req@ffff985610f83c00 x1874669228356480/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829022 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828997.877143:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83c00 x1874669228356480/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829022 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828997.877156:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828997.877160:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828997.877166:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228356480:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787828997.877174:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228356608:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828997.877206:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828997.877219:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828997/real 1787828997] req@ffff985610f80780 x1874669228356608/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829022 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828997.877234:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80780 x1874669228356608/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829022 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828997.877243:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828997.877247:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828997.877250:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228356608:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787828997.877278:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228356736:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787828997.877297:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787828997.877307:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828997/real 1787828997] req@ffff985610f830c0 x1874669228356736/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829022 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828997.877319:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f830c0 x1874669228356736/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829022 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828997.877328:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828997.877332:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787828997.877335:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228356736:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787828997.940854:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828998.836828:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828998.964842:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828999.860789:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787828999.988837:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829000.884799:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829001.012868:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829001.908767:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829002.036839:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829002.932835:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787829002.996815:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829002.996826:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829002.996830:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829002.996841:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5148 00000100:00080000:3.0:1787829002.996852:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829002.996859:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787829002.996901:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80000 x1874669228356864/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829002.996928:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829002.996931:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829002.996932:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829002.996935:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5148 00000100:00080000:3.0:1787829002.996938:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829002.996940:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829002.996950:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829002.996960:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80b40 x1874669228356992/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829002.996968:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:1.0:1787829002.996974:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228356864:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787829002.996974:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829002.996977:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829002.996986:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5148 00000100:00080000:3.0:1787829002.996989:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829002.996991:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829002.996994:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829002.997003:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f812c0 x1874669228357120/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787829002.997062:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829002.997111:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829002/real 1787829002] req@ffff985610f80000 x1874669228356864/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829027 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829002.997127:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80000 x1874669228356864/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829027 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829002.997136:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829002.997139:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829002.997143:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228356864:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787829002.997166:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228356992:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787829002.997183:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829002.997192:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829002/real 1787829002] req@ffff985610f80b40 x1874669228356992/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829027 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829002.997200:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80b40 x1874669228356992/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829027 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829002.997207:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829002.997209:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829002.997211:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228356992:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787829002.997215:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228357120:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787829002.997226:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829002.997231:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829002/real 1787829002] req@ffff985610f812c0 x1874669228357120/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829027 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829002.997243:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f812c0 x1874669228357120/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829027 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829002.997249:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829002.997250:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829002.997252:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228357120:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787829003.060902:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829003.956830:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829004.084845:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829004.980826:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829005.108860:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829006.004829:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829006.132872:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829007.028759:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829007.156812:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829008.052815:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787829008.116806:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829008.116816:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829008.116820:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829008.116830:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5153 00000100:00080000:1.0:1787829008.116843:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829008.116850:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787829008.116892:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a30c0 x1874669228357248/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829008.116919:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829008.116923:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829008.116925:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829008.116930:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5153 00000100:00080000:1.0:1787829008.116934:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829008.116940:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787829008.116952:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829008.116970:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228357248:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787829008.116972:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a21c0 x1874669228357376/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829008.116987:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829008.116989:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829008.116991:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829008.116994:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5153 00000100:00080000:1.0:1787829008.116997:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829008.117000:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787829008.117004:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829008.117014:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3480 x1874669228357504/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829008.117113:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228357376:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787829008.117149:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228357504:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787829008.180825:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829009.076771:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829009.204827:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829010.100819:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829010.228785:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829011.124786:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000400:00000100:0.0:1787829011.189045:0:2370:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.114:988 00000400:00000100:0.0:1787829011.189103:0:2370:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.114@tcp at host 192.168.203.114 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:0.0:1787829011.189127:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:0.0:1787829011.189176:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:0.0:1787829011.189189:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000100:00100000:3.0:1787829011.189229:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829008/real 1787829011] req@ffff9856085a21c0 x1874669228357376/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829033 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829011.189262:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a21c0 x1874669228357376/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829033 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829011.189280:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829011.189288:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829011.189295:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228357376:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787829011.189307:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829008/real 1787829011] req@ffff9856085a3480 x1874669228357504/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829033 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829011.189321:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3480 x1874669228357504/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829033 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829011.189331:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829011.189337:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829011.189341:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228357504:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787829011.189350:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829008/real 1787829011] req@ffff9856085a30c0 x1874669228357248/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829033 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829011.189369:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a30c0 x1874669228357248/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829033 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829011.189380:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829011.189385:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829011.189391:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228357248:192.168.203.114@tcp:250:kworker.0 00010000:00010000:1.0:1787829011.252846:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829012.148788:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829012.276784:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829013.172883:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787829013.237006:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829013.237022:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829013.237027:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829013.237039:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5158 00000100:00080000:0.0:1787829013.237064:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829013.237072:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787829013.237115:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6d00 x1874669228357632/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829013.237143:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829013.237146:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829013.237148:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829013.237151:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5158 00000100:00080000:0.0:1787829013.237153:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829013.237158:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787829013.237168:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829013.237178:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5e00 x1874669228357760/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829013.237190:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829013.237194:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829013.237196:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829013.237201:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5158 00000100:00080000:0.0:1787829013.237205:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787829013.237211:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228357632:192.168.203.114@tcp:250:kworker.0 00000100:00080000:0.0:1787829013.237214:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787829013.237237:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829013.237252:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df61c0 x1874669228357888/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829013.237311:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829013.237358:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829013/real 1787829013] req@ffff985607df6d00 x1874669228357632/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829038 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829013.237381:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6d00 x1874669228357632/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829038 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829013.237410:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829013.237415:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829013.237420:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228357632:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787829013.237429:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228357760:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829013.237480:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829013.237495:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829013/real 1787829013] req@ffff985607df5e00 x1874669228357760/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829038 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829013.237507:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5e00 x1874669228357760/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829038 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829013.237515:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829013.237519:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829013.237522:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228357760:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787829013.237557:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228357888:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829013.237580:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829013.237593:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829013/real 1787829013] req@ffff985607df61c0 x1874669228357888/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829038 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829013.237605:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df61c0 x1874669228357888/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829038 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829013.237615:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829013.237618:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829013.237622:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228357888:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787829013.300881:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829014.196823:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829014.324870:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829015.220811:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829015.348905:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829016.244878:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829016.372951:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829017.268803:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829017.396852:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829018.292810:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787829018.356904:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829018.356916:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829018.356921:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829018.356933:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5163 00000100:00080000:0.0:1787829018.356947:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829018.356977:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787829018.357060:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df43c0 x1874669228358016/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829018.357101:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829018.357104:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829018.357106:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829018.357122:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5163 00000100:00080000:0.0:1787829018.357126:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829018.357130:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787829018.357141:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829018.357172:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5680 x1874669228358144/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829018.357181:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829018.357185:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829018.357186:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829018.357189:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5163 00000100:00080000:0.0:1787829018.357192:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829018.357195:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787829018.357201:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829018.357212:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df70c0 x1874669228358272/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829018.357214:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228358016:192.168.203.114@tcp:250:kworker.0 00000800:00000100:3.0:1787829018.357324:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829018.357375:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829018/real 1787829018] req@ffff985607df43c0 x1874669228358016/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829043 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829018.357435:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df43c0 x1874669228358016/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829043 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829018.357446:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829018.357451:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829018.357456:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228358016:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787829018.357466:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228358144:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829018.357493:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829018.357511:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829018/real 1787829018] req@ffff985607df5680 x1874669228358144/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829043 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829018.357529:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5680 x1874669228358144/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829043 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829018.357539:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829018.357546:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829018.357554:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228358144:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787829018.357595:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228358272:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829018.357614:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829018.357623:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829018/real 1787829018] req@ffff985607df70c0 x1874669228358272/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829043 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829018.357634:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df70c0 x1874669228358272/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829043 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829018.357646:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829018.357650:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829018.357652:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228358272:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787829018.421026:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829019.316852:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829019.444909:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829020.340790:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829020.468894:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829021.364873:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829021.492878:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829022.388871:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829022.516947:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829023.412846:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787829023.476850:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829023.476864:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829023.476869:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829023.476882:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5169 00000100:00080000:3.0:1787829023.476897:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829023.476907:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787829023.476971:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81680 x1874669228358400/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829023.477007:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829023.477012:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829023.477014:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829023.477019:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5169 00000100:00080000:3.0:1787829023.477023:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829023.477029:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829023.477041:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829023.477060:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228358400:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787829023.477068:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81a40 x1874669228358528/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829023.477087:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829023.477093:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829023.477095:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829023.477100:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5169 00000100:00080000:3.0:1787829023.477105:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829023.477111:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829023.477117:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000800:00000100:1.0:1787829023.477133:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829023.477135:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83840 x1874669228358656/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787829023.477174:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829023/real 1787829023] req@ffff985610f81680 x1874669228358400/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829048 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829023.477187:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81680 x1874669228358400/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829048 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829023.477196:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829023.477199:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829023.477202:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228358400:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787829023.477225:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228358528:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787829023.477242:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829023.477250:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829023/real 1787829023] req@ffff985610f81a40 x1874669228358528/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829048 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829023.477258:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81a40 x1874669228358528/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829048 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829023.477264:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829023.477266:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829023.477268:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228358528:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787829023.477274:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228358656:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787829023.477292:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829023.477304:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829023/real 1787829023] req@ffff985610f83840 x1874669228358656/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829048 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829023.477318:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83840 x1874669228358656/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829048 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829023.477327:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829023.477329:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829023.477332:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228358656:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787829023.540848:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829024.436828:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829024.564884:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829025.460782:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829025.588847:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829026.484799:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829026.612839:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829027.508842:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829027.636770:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829028.532834:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787829028.596794:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829028.596804:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829028.596809:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829028.596819:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5174 00000100:00080000:3.0:1787829028.596829:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829028.596836:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787829028.596870:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f803c0 x1874669228358784/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829028.596911:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829028.596916:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829028.596918:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829028.596922:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5174 00000100:00080000:3.0:1787829028.596926:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829028.596931:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829028.596941:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829028.596960:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228358784:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787829028.596961:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81e00 x1874669228358912/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829028.596978:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829028.596983:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829028.596985:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829028.596990:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5174 00000100:00080000:3.0:1787829028.596995:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829028.596999:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829028.597004:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829028.597017:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80f00 x1874669228359040/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787829028.597023:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829028.597064:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829028/real 1787829028] req@ffff985610f803c0 x1874669228358784/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829053 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829028.597080:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f803c0 x1874669228358784/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829053 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829028.597091:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829028.597096:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829028.597100:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228358784:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787829028.597124:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228358912:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787829028.597144:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829028.597163:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829028/real 1787829028] req@ffff985610f81e00 x1874669228358912/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829053 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829028.597172:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81e00 x1874669228358912/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829053 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829028.597178:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829028.597180:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829028.597182:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228358912:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787829028.597186:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228359040:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787829028.597198:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829028.597205:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829028/real 1787829028] req@ffff985610f80f00 x1874669228359040/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829053 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829028.597212:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80f00 x1874669228359040/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829053 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829028.597218:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829028.597220:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829028.597221:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228359040:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787829028.660858:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829029.556843:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829029.684792:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829030.580823:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829030.708816:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829031.604844:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829031.732813:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829032.628826:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829032.756831:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829033.652832:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787829033.716810:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829033.716824:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829033.716829:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829033.716842:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5179 00000100:00080000:3.0:1787829033.716869:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829033.716880:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787829033.716943:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82d00 x1874669228359168/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829033.716994:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829033.716999:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829033.717001:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829033.717006:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5179 00000100:00080000:3.0:1787829033.717013:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829033.717019:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829033.717032:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829033.717053:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228359168:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787829033.717073:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82940 x1874669228359296/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829033.717088:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829033.717093:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829033.717095:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829033.717102:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5179 00000100:00080000:3.0:1787829033.717107:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829033.717113:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829033.717120:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000800:00000100:1.0:1787829033.717125:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829033.717152:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82580 x1874669228359424/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787829033.717177:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829033/real 1787829033] req@ffff985610f82d00 x1874669228359168/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829058 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829033.717194:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82d00 x1874669228359168/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829058 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829033.717203:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829033.717207:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829033.717210:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228359168:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787829033.717230:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228359296:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787829033.717258:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829033.717266:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829033/real 1787829033] req@ffff985610f82940 x1874669228359296/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829058 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829033.717273:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82940 x1874669228359296/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829058 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829033.717279:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829033.717282:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829033.717284:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228359296:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787829033.717288:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228359424:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787829033.717298:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829033.717304:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829033/real 1787829033] req@ffff985610f82580 x1874669228359424/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829058 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829033.717321:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82580 x1874669228359424/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829058 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829033.717326:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829033.717328:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829033.717330:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228359424:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787829033.780890:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829034.676823:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829034.804824:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829035.700818:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829035.828820:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829036.724849:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829036.852925:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829037.748845:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829037.876812:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829038.772832:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787829038.836782:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829038.836793:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829038.836797:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829038.836833:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5184 00000100:00080000:1.0:1787829038.836846:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829038.836853:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787829038.836911:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0f00 x1874669228359552/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829038.836943:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829038.836946:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829038.836948:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829038.836951:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5184 00000100:00080000:1.0:1787829038.836953:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829038.836956:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787829038.836966:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829038.836977:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a30c0 x1874669228359680/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829038.836985:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829038.836988:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829038.836992:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1787829038.836993:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228359552:192.168.203.114@tcp:250:kworker.0 00000100:00080000:1.0:1787829038.836996:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5184 00000100:00080000:1.0:1787829038.837002:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829038.837005:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787829038.837008:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829038.837022:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3480 x1874669228359808/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829038.837068:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829038.837101:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829038/real 1787829038] req@ffff9856085a0f00 x1874669228359552/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829063 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829038.837117:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0f00 x1874669228359552/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829063 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829038.837126:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829038.837130:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829038.837134:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228359552:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787829038.837157:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228359680:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829038.837174:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829038.837187:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829038/real 1787829038] req@ffff9856085a30c0 x1874669228359680/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829063 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829038.837201:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a30c0 x1874669228359680/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829063 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829038.837210:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829038.837214:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829038.837217:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228359680:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787829038.837224:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228359808:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829038.837244:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829038.837254:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829038/real 1787829038] req@ffff9856085a3480 x1874669228359808/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829063 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829038.837262:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3480 x1874669228359808/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829063 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829038.837268:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829038.837270:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829038.837272:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228359808:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787829038.900853:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829039.796785:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829039.924973:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829040.820815:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829040.948878:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829041.844779:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829041.972898:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829042.868823:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829042.996884:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829043.892810:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787829043.956819:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829043.956826:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829043.956830:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829043.956837:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5189 00000100:00080000:3.0:1787829043.956847:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829043.956852:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787829043.956882:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f821c0 x1874669228359936/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829043.956904:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829043.956907:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829043.956909:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829043.956911:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5189 00000100:00080000:3.0:1787829043.956913:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829043.956916:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829043.956924:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829043.956934:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83480 x1874669228360064/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787829043.956941:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f821c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228359936:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787829043.956944:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829043.956950:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829043.956951:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829043.956953:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5189 00000100:00080000:3.0:1787829043.956955:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829043.956957:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829043.956961:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829043.956970:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82580 x1874669228360192/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787829043.957002:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829043.957031:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829043/real 1787829043] req@ffff985610f821c0 x1874669228359936/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829068 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829043.957043:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f821c0 x1874669228359936/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829068 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829043.957051:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829043.957054:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829043.957057:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f821c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228359936:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787829043.957075:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228360064:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787829043.957090:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829043.957097:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829043/real 1787829043] req@ffff985610f83480 x1874669228360064/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829068 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829043.957105:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83480 x1874669228360064/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829068 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829043.957110:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829043.957112:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829043.957114:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228360064:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787829043.957118:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228360192:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787829043.957128:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829043.957134:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829043/real 1787829043] req@ffff985610f82580 x1874669228360192/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829068 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829043.957141:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82580 x1874669228360192/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829068 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829043.957151:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829043.957153:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829043.957155:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228360192:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787829044.020918:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829044.916829:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829045.044826:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829045.940801:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829046.068832:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829046.964838:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829047.092874:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829047.988766:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829048.116888:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829049.012838:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787829049.076831:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829049.076841:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829049.076846:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829049.076856:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5194 00000100:00080000:3.0:1787829049.076869:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829049.076877:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787829049.076917:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82940 x1874669228360320/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829049.076945:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829049.076948:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829049.076950:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829049.076953:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5194 00000100:00080000:3.0:1787829049.076955:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829049.076958:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829049.076967:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829049.076976:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82d00 x1874669228360448/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829049.076983:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829049.076986:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829049.076987:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829049.076990:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5194 00000100:00080000:3.0:1787829049.076992:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829049.076996:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829049.077000:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829049.077005:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228360320:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787829049.077010:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80f00 x1874669228360576/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787829049.077092:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829049.077129:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829049/real 1787829049] req@ffff985610f82940 x1874669228360320/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829074 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829049.077144:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82940 x1874669228360320/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829074 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829049.077153:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829049.077157:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829049.077160:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228360320:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787829049.077166:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228360448:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787829049.077183:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829049.077192:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829049/real 1787829049] req@ffff985610f82d00 x1874669228360448/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829074 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829049.077200:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82d00 x1874669228360448/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829074 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829049.077206:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829049.077209:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829049.077211:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228360448:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787829049.077235:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228360576:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787829049.077247:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829049.077254:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829049/real 1787829049] req@ffff985610f80f00 x1874669228360576/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829074 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829049.077262:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80f00 x1874669228360576/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829074 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829049.077270:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829049.077274:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829049.077277:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228360576:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787829049.140877:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829050.036819:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829050.164852:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829051.060825:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829051.188885:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829052.084804:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829052.212888:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829053.108828:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829053.236846:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829054.132779:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787829054.196784:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829054.196791:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829054.196795:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829054.196802:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5199 00000100:00080000:3.0:1787829054.196811:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829054.196822:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787829054.196851:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81e00 x1874669228360704/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829054.196874:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829054.196876:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829054.196878:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829054.196881:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5199 00000100:00080000:3.0:1787829054.196883:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829054.196886:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829054.196894:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829054.196907:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f803c0 x1874669228360832/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787829054.196908:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228360704:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787829054.196922:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829054.196925:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829054.196927:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829054.196930:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5199 00000100:00080000:3.0:1787829054.196933:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829054.196935:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829054.196938:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829054.196948:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83840 x1874669228360960/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787829054.196979:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829054.197015:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829054/real 1787829054] req@ffff985610f81e00 x1874669228360704/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829079 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829054.197031:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81e00 x1874669228360704/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829079 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829054.197040:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829054.197043:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829054.197047:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228360704:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787829054.197068:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228360832:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787829054.197085:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829054.197093:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829054/real 1787829054] req@ffff985610f803c0 x1874669228360832/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829079 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829054.197102:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f803c0 x1874669228360832/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829079 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829054.197108:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829054.197110:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829054.197113:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228360832:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787829054.197117:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228360960:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787829054.197128:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829054.197134:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829054/real 1787829054] req@ffff985610f83840 x1874669228360960/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829079 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829054.197144:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83840 x1874669228360960/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829079 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829054.197153:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829054.197156:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829054.197159:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228360960:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787829054.260841:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829055.156805:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829055.284846:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829056.180850:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829056.308827:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829057.204812:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829057.332909:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829058.228813:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829058.356912:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829059.252774:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787829059.316820:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829059.316834:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829059.316840:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829059.316852:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5204 00000100:00080000:1.0:1787829059.316873:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829059.316883:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787829059.316930:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a21c0 x1874669228361088/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829059.316961:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829059.316965:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829059.316967:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829059.316972:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5204 00000100:00080000:1.0:1787829059.316976:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829059.316981:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787829059.316993:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829059.317008:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2940 x1874669228361216/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829059.317012:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228361088:192.168.203.114@tcp:250:kworker.0 00000100:00080000:1.0:1787829059.317035:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829059.317038:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829059.317040:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829059.317044:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5204 00000100:00080000:1.0:1787829059.317047:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829059.317051:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787829059.317056:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829059.317066:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a1a40 x1874669228361344/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829059.317088:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829059.317132:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829059/real 1787829059] req@ffff9856085a21c0 x1874669228361088/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829084 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829059.317147:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a21c0 x1874669228361088/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829084 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829059.317155:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829059.317159:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829059.317162:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228361088:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787829059.317186:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228361216:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829059.317202:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829059.317211:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829059/real 1787829059] req@ffff9856085a2940 x1874669228361216/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829084 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829059.317218:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2940 x1874669228361216/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829084 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829059.317229:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829059.317232:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829059.317234:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228361216:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787829059.317238:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228361344:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829059.317248:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829059.317254:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829059/real 1787829059] req@ffff9856085a1a40 x1874669228361344/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829084 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829059.317261:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a1a40 x1874669228361344/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829084 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829059.317267:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829059.317269:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829059.317271:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228361344:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787829059.380883:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829060.276753:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829060.404857:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829061.300776:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829061.428813:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829062.324800:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829062.452871:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829063.348786:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829063.476859:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829064.372787:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787829064.436882:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829064.436895:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829064.436900:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829064.436915:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5210 00000100:00080000:0.0:1787829064.436930:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829064.436940:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787829064.436994:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5a40 x1874669228361472/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829064.437033:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829064.437037:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829064.437039:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829064.437043:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5210 00000100:00080000:0.0:1787829064.437048:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829064.437052:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787829064.437064:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829064.437078:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6580 x1874669228361600/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829064.437095:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1787829064.437096:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228361472:192.168.203.114@tcp:250:kworker.0 00000100:00080000:0.0:1787829064.437110:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829064.437114:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829064.437120:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5210 00000100:00080000:0.0:1787829064.437124:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829064.437129:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787829064.437136:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829064.437152:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df70c0 x1874669228361728/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829064.437179:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829064.437217:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829064/real 1787829064] req@ffff985607df5a40 x1874669228361472/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829089 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829064.437232:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5a40 x1874669228361472/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829089 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829064.437241:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829064.437245:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829064.437250:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228361472:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787829064.437278:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228361600:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829064.437304:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829064.437316:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829064/real 1787829064] req@ffff985607df6580 x1874669228361600/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829089 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829064.437325:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6580 x1874669228361600/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829089 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829064.437331:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829064.437334:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829064.437337:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228361600:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787829064.437349:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228361728:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829064.437365:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829064.437372:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829064/real 1787829064] req@ffff985607df70c0 x1874669228361728/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829089 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829064.437380:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df70c0 x1874669228361728/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829089 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829064.437385:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829064.437388:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829064.437390:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228361728:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787829064.500899:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829065.396753:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829065.524976:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829066.420797:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829066.548799:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829067.444777:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829067.572960:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829068.468800:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829068.596811:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829069.492778:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787829069.556808:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829069.556819:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829069.556823:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829069.556832:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5215 00000100:00080000:3.0:1787829069.556842:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829069.556850:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787829069.556887:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81a40 x1874669228361856/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829069.556917:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829069.556921:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829069.556923:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829069.556928:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5215 00000100:00080000:3.0:1787829069.556931:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829069.556936:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829069.556945:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829069.556963:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228361856:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787829069.556964:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81680 x1874669228361984/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829069.556981:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829069.556985:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829069.556987:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829069.556991:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5215 00000100:00080000:3.0:1787829069.556996:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829069.557000:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829069.557006:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829069.557019:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f812c0 x1874669228362112/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787829069.557026:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829069.557068:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829069/real 1787829069] req@ffff985610f81a40 x1874669228361856/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829094 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829069.557081:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81a40 x1874669228361856/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829094 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829069.557089:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829069.557092:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829069.557095:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228361856:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787829069.557113:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228361984:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787829069.557128:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829069.557136:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829069/real 1787829069] req@ffff985610f81680 x1874669228361984/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829094 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829069.557144:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81680 x1874669228361984/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829094 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829069.557150:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829069.557152:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829069.557154:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228361984:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787829069.557158:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228362112:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787829069.557168:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829069.557174:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829069/real 1787829069] req@ffff985610f812c0 x1874669228362112/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829094 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829069.557181:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f812c0 x1874669228362112/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829094 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829069.557186:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829069.557188:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829069.557190:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228362112:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787829069.620856:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829070.516815:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829070.644843:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829071.540815:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829071.668862:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829072.564844:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829072.692893:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829073.588824:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829073.716862:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829074.612770:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787829074.676828:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829074.676836:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829074.676840:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829074.676848:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5220 00000100:00080000:0.0:1787829074.676857:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829074.676864:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787829074.676897:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5680 x1874669228362240/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829074.676919:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829074.676926:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829074.676928:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829074.676932:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5220 00000100:00080000:0.0:1787829074.676935:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829074.676940:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787829074.676950:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829074.676961:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228362240:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829074.676967:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df43c0 x1874669228362368/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829074.676982:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829074.676986:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829074.676987:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829074.676990:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5220 00000100:00080000:0.0:1787829074.676994:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829074.676997:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787829074.677011:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829074.677023:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df61c0 x1874669228362496/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787829074.677054:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228362368:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787829074.677087:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228362496:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787829074.740850:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829075.636789:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829075.764868:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829076.660773:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829076.788849:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829077.684784:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000400:00000100:3.0:1787829077.748919:0:2371:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.114:988 00000400:00000100:3.0:1787829077.748957:0:2371:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.114@tcp at host 192.168.203.114 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:3.0:1787829077.748986:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:3.0:1787829077.749020:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:3.0:1787829077.749029:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000100:00100000:1.0:1787829077.749207:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829074/real 1787829077] req@ffff985607df43c0 x1874669228362368/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829099 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829077.749234:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df43c0 x1874669228362368/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829099 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829077.749248:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829077.749253:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829077.749259:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228362368:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787829077.749266:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829074/real 1787829077] req@ffff985607df61c0 x1874669228362496/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829099 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829077.749278:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df61c0 x1874669228362496/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829099 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829077.749286:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829077.749290:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829077.749294:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228362496:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787829077.749301:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829074/real 1787829077] req@ffff985607df5680 x1874669228362240/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829099 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829077.749314:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5680 x1874669228362240/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829099 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829077.749321:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829077.749325:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829077.749328:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228362240:192.168.203.114@tcp:250:kworker.0 00010000:00010000:1.0:1787829077.812840:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829078.708794:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829078.836825:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829079.732778:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787829079.796782:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829079.796789:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829079.796792:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829079.796801:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5225 00000100:00080000:1.0:1787829079.796809:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829079.796815:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787829079.796846:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a12c0 x1874669228362624/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829079.796870:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829079.796872:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829079.796874:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829079.796877:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5225 00000100:00080000:1.0:1787829079.796879:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829079.796881:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787829079.796889:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829079.796897:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2580 x1874669228362752/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829079.796906:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829079.796909:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829079.796912:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1787829079.796913:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228362624:192.168.203.114@tcp:250:kworker.0 00000100:00080000:1.0:1787829079.796916:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5225 00000100:00080000:1.0:1787829079.796921:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829079.796925:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787829079.796929:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829079.796936:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3840 x1874669228362880/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829079.796983:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829079.797017:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829079/real 1787829079] req@ffff9856085a12c0 x1874669228362624/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829104 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829079.797035:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a12c0 x1874669228362624/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829104 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829079.797046:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829079.797050:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829079.797060:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228362624:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787829079.797087:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228362752:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829079.797111:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829079.797124:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829079/real 1787829079] req@ffff9856085a2580 x1874669228362752/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829104 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829079.797137:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2580 x1874669228362752/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829104 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829079.797152:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829079.797155:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829079.797159:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228362752:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787829079.797165:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228362880:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829079.797185:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829079.797198:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829079/real 1787829079] req@ffff9856085a3840 x1874669228362880/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829104 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829079.797211:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3840 x1874669228362880/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829104 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829079.797220:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829079.797223:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829079.797227:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228362880:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787829079.860813:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829080.756771:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829080.884975:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829081.780796:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829081.908846:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829082.804766:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829082.932913:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829083.828803:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829083.956807:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829084.852773:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787829084.916875:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829084.916885:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829084.916896:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829084.916907:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5230 00000100:00080000:3.0:1787829084.916919:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829084.916927:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787829084.916970:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80b40 x1874669228363008/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829084.916998:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829084.917003:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829084.917004:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829084.917007:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5230 00000100:00080000:3.0:1787829084.917010:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829084.917013:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829084.917023:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829084.917033:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80000 x1874669228363136/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787829084.917039:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228363008:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787829084.917043:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829084.917048:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829084.917050:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829084.917057:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5230 00000100:00080000:3.0:1787829084.917059:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829084.917061:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829084.917064:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829084.917076:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f830c0 x1874669228363264/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787829084.917131:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829084.917167:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829084/real 1787829084] req@ffff985610f80b40 x1874669228363008/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829109 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829084.917182:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80b40 x1874669228363008/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829109 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829084.917191:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829084.917194:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829084.917198:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228363008:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787829084.917222:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228363136:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787829084.917248:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829084.917257:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829084/real 1787829084] req@ffff985610f80000 x1874669228363136/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829109 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829084.917265:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80000 x1874669228363136/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829109 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829084.917271:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829084.917274:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829084.917276:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228363136:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787829084.917279:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228363264:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787829084.917291:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829084.917297:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829084/real 1787829084] req@ffff985610f830c0 x1874669228363264/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829109 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829084.917310:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f830c0 x1874669228363264/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829109 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829084.917315:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829084.917316:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829084.917318:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228363264:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787829084.980893:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829085.876797:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829086.004882:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829086.900741:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829087.028850:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829087.924836:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829088.052826:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829088.948799:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829089.076954:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829089.972831:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787829090.036813:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829090.036823:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829090.036827:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829090.036837:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5235 00000100:00080000:3.0:1787829090.036849:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829090.036856:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787829090.036898:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80780 x1874669228363392/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829090.036924:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829090.036927:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829090.036928:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829090.036931:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5235 00000100:00080000:3.0:1787829090.036934:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829090.036937:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829090.036945:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829090.036954:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83c00 x1874669228363520/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829090.036960:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829090.036963:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829090.036964:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829090.036967:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5235 00000100:00080000:3.0:1787829090.036969:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829090.036971:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829090.036974:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829090.036985:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f830c0 x1874669228363648/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787829090.037020:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228363392:192.168.203.114@tcp:250:kworker.0 00000800:00000100:1.0:1787829090.037091:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829090.037130:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829090/real 1787829090] req@ffff985610f80780 x1874669228363392/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829115 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829090.037145:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80780 x1874669228363392/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829115 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829090.037154:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829090.037158:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829090.037161:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228363392:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787829090.037166:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228363520:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787829090.037185:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829090.037196:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829090/real 1787829090] req@ffff985610f83c00 x1874669228363520/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829115 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829090.037205:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83c00 x1874669228363520/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829115 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829090.037212:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829090.037214:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829090.037218:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228363520:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787829090.037224:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228363648:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787829090.037244:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829090.037262:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829090/real 1787829090] req@ffff985610f830c0 x1874669228363648/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829115 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829090.037278:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f830c0 x1874669228363648/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829115 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829090.037285:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829090.037287:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829090.037289:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228363648:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787829090.100858:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829090.996832:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829091.124927:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829092.020862:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829092.148844:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829093.044855:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829093.172857:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829094.068839:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829094.196851:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829095.092839:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787829095.156809:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829095.156819:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829095.156823:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829095.156833:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5240 00000100:00080000:3.0:1787829095.156845:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829095.156853:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787829095.156895:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80000 x1874669228363776/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829095.156921:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829095.156924:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829095.156927:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829095.156931:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5240 00000100:00080000:3.0:1787829095.156935:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829095.156939:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829095.156948:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829095.156962:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80b40 x1874669228363904/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787829095.156969:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228363776:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787829095.156972:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829095.156975:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829095.156977:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829095.156980:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5240 00000100:00080000:3.0:1787829095.156984:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829095.156987:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829095.156991:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829095.157000:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f812c0 x1874669228364032/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787829095.157028:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829095.157059:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829095/real 1787829095] req@ffff985610f80000 x1874669228363776/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829120 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829095.157076:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80000 x1874669228363776/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829120 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829095.157083:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829095.157087:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829095.157089:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228363776:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787829095.157108:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228363904:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787829095.157126:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829095.157135:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829095/real 1787829095] req@ffff985610f80b40 x1874669228363904/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829120 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829095.157143:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80b40 x1874669228363904/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829120 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829095.157148:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829095.157150:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829095.157153:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228363904:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787829095.157156:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228364032:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787829095.157166:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829095.157172:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829095/real 1787829095] req@ffff985610f812c0 x1874669228364032/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829120 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829095.157179:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f812c0 x1874669228364032/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829120 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829095.157184:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829095.157186:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829095.157188:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228364032:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787829095.220877:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829096.116802:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829096.244845:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829097.140816:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829097.268799:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829098.164774:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829098.292824:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829099.188804:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829099.316819:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829100.212763:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787829100.276733:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829100.276741:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829100.276744:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829100.276752:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5245 00000100:00080000:3.0:1787829100.276761:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829100.276767:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787829100.276794:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81680 x1874669228364160/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829100.276814:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829100.276818:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829100.276820:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829100.276825:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5245 00000100:00080000:3.0:1787829100.276828:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829100.276833:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829100.276843:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829100.276854:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228364160:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787829100.276855:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81a40 x1874669228364288/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829100.276865:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829100.276868:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829100.276870:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829100.276873:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5245 00000100:00080000:3.0:1787829100.276876:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829100.276880:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829100.276884:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829100.276892:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83840 x1874669228364416/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787829100.276912:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829100.276941:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829100/real 1787829100] req@ffff985610f81680 x1874669228364160/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829125 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829100.276954:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81680 x1874669228364160/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829125 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829100.276962:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829100.276965:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829100.276968:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228364160:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787829100.276985:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228364288:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787829100.277005:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829100.277017:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829100/real 1787829100] req@ffff985610f81a40 x1874669228364288/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829125 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829100.277031:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81a40 x1874669228364288/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829125 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829100.277040:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829100.277044:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829100.277048:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228364288:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787829100.277052:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228364416:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787829100.277068:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829100.277075:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829100/real 1787829100] req@ffff985610f83840 x1874669228364416/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829125 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829100.277083:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83840 x1874669228364416/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829125 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829100.277089:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829100.277091:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829100.277093:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228364416:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787829100.340907:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829101.236806:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829101.364858:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829102.260765:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829102.388814:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829103.284832:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829103.412830:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829104.308767:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829104.436841:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829105.332780:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787829105.396758:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829105.396769:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829105.396773:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829105.396783:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5251 00000100:00080000:3.0:1787829105.396796:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829105.396804:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787829105.396844:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f803c0 x1874669228364544/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829105.396869:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829105.396876:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829105.396878:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829105.396881:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5251 00000100:00080000:3.0:1787829105.396883:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829105.396887:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829105.396896:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829105.396907:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81e00 x1874669228364672/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829105.396916:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829105.396921:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:1.0:1787829105.396922:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228364544:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787829105.396923:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829105.396931:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5251 00000100:00080000:3.0:1787829105.396935:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829105.396937:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829105.396940:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829105.396949:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80f00 x1874669228364800/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787829105.397008:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829105.397048:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829105/real 1787829105] req@ffff985610f803c0 x1874669228364544/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829130 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829105.397065:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f803c0 x1874669228364544/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829130 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829105.397074:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829105.397077:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829105.397080:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228364544:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787829105.397110:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228364672:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787829105.397137:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829105.397151:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829105/real 1787829105] req@ffff985610f81e00 x1874669228364672/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829130 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829105.397160:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81e00 x1874669228364672/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829130 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829105.397167:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829105.397169:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829105.397172:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228364672:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787829105.397176:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228364800:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787829105.397192:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829105.397199:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829105/real 1787829105] req@ffff985610f80f00 x1874669228364800/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829130 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829105.397207:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80f00 x1874669228364800/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829130 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829105.397212:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829105.397214:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829105.397217:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228364800:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787829105.460832:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829106.356777:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829106.484859:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829107.380822:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829107.508792:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829108.404740:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829108.532906:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829109.428815:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829109.556853:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829110.452848:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787829110.516801:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829110.516812:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829110.516817:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829110.516827:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5256 00000100:00080000:0.0:1787829110.516839:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829110.516847:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787829110.516889:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5e00 x1874669228364928/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829110.516915:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829110.516918:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829110.516920:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829110.516922:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5256 00000100:00080000:0.0:1787829110.516925:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829110.516927:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787829110.516937:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829110.516947:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6d00 x1874669228365056/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829110.516954:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829110.516957:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829110.516960:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829110.516964:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5256 00000100:00080000:0.0:1787829110.516966:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829110.516969:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787829110.516972:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829110.516980:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228364928:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829110.516981:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4f00 x1874669228365184/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787829110.517062:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829110.517101:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829110/real 1787829110] req@ffff985607df5e00 x1874669228364928/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829135 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829110.517116:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5e00 x1874669228364928/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829135 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829110.517125:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829110.517129:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829110.517132:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228364928:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787829110.517138:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228365056:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787829110.517154:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829110.517163:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829110/real 1787829110] req@ffff985607df6d00 x1874669228365056/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829135 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829110.517171:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6d00 x1874669228365056/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829135 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829110.517177:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829110.517178:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829110.517181:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228365056:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787829110.517203:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228365184:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787829110.517215:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829110.517222:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829110/real 1787829110] req@ffff985607df4f00 x1874669228365184/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829135 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829110.517229:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4f00 x1874669228365184/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829135 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829110.517234:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829110.517236:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829110.517238:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228365184:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787829110.580875:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829111.476799:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829111.604877:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829112.500796:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829112.628798:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829113.524786:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829113.652878:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829114.548790:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829114.676842:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829115.572819:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787829115.636842:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829115.636853:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829115.636858:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829115.636869:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5261 00000100:00080000:3.0:1787829115.636880:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829115.636888:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787829115.636941:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82d00 x1874669228365312/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829115.636968:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829115.636971:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829115.636973:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829115.636976:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5261 00000100:00080000:3.0:1787829115.636978:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829115.636981:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829115.636990:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829115.637000:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82940 x1874669228365440/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829115.637009:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829115.637012:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829115.637014:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829115.637016:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5261 00000100:00080000:3.0:1787829115.637018:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829115.637021:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829115.637024:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829115.637032:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82580 x1874669228365568/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787829115.637034:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228365312:192.168.203.114@tcp:250:kworker.0 00000800:00000100:1.0:1787829115.637125:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829115.637180:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829115/real 1787829115] req@ffff985610f82d00 x1874669228365312/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829140 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829115.637198:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82d00 x1874669228365312/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829140 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829115.637210:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829115.637214:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829115.637218:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228365312:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787829115.637225:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228365440:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787829115.637246:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829115.637258:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829115/real 1787829115] req@ffff985610f82940 x1874669228365440/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829140 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829115.637269:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82940 x1874669228365440/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829140 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829115.637277:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829115.637293:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829115.637297:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228365440:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787829115.637336:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228365568:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787829115.637355:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829115.637365:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829115/real 1787829115] req@ffff985610f82580 x1874669228365568/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829140 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829115.637378:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82580 x1874669228365568/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829140 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829115.637386:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829115.637390:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829115.637393:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228365568:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787829115.700910:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829116.596803:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829116.724824:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829117.620774:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829117.748944:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829118.644835:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829118.772869:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829119.668785:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829119.796869:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829120.692813:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787829120.756834:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829120.756844:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829120.756849:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829120.756860:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5266 00000100:00080000:0.0:1787829120.756873:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829120.756882:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787829120.756933:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7840 x1874669228365696/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829120.756967:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829120.756971:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829120.756973:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829120.756976:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5266 00000100:00080000:0.0:1787829120.756980:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829120.756983:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787829120.756994:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829120.757006:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df52c0 x1874669228365824/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787829120.757015:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228365696:192.168.203.114@tcp:250:kworker.0 00000100:00080000:0.0:1787829120.757018:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829120.757023:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829120.757025:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829120.757029:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5266 00000100:00080000:0.0:1787829120.757031:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829120.757033:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787829120.757036:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829120.757045:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7480 x1874669228365952/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787829120.757094:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829120.757130:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829120/real 1787829120] req@ffff985607df7840 x1874669228365696/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829145 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829120.757144:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7840 x1874669228365696/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829145 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829120.757154:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829120.757158:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829120.757161:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228365696:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787829120.757183:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228365824:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787829120.757199:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829120.757208:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829120/real 1787829120] req@ffff985607df52c0 x1874669228365824/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829145 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829120.757216:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df52c0 x1874669228365824/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829145 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829120.757223:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829120.757225:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829120.757227:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228365824:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787829120.757231:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228365952:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787829120.757242:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829120.757263:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829120/real 1787829120] req@ffff985607df7480 x1874669228365952/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829145 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829120.757271:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7480 x1874669228365952/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829145 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829120.757277:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829120.757279:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829120.757281:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228365952:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787829120.820877:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829121.716763:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829121.844898:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829122.740835:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829122.868781:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829123.764769:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829123.892882:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829124.788815:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829124.916818:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829125.812747:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787829125.876852:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829125.876863:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829125.876867:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829125.876878:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5271 00000100:00080000:0.0:1787829125.876891:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829125.876898:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787829125.876954:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4b40 x1874669228366080/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829125.876979:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829125.876982:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829125.876985:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829125.876988:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5271 00000100:00080000:0.0:1787829125.876990:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829125.876997:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787829125.877008:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829125.877018:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6940 x1874669228366208/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829125.877026:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829125.877029:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829125.877032:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1787829125.877035:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228366080:192.168.203.114@tcp:250:kworker.0 00000100:00080000:0.0:1787829125.877038:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5271 00000100:00080000:0.0:1787829125.877044:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829125.877047:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787829125.877050:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829125.877058:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4780 x1874669228366336/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787829125.877112:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829125.877150:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829125/real 1787829125] req@ffff985607df4b40 x1874669228366080/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829150 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829125.877165:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4b40 x1874669228366080/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829150 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829125.877174:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829125.877179:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829125.877182:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228366080:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787829125.877206:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228366208:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787829125.877244:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829125.877255:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829125/real 1787829125] req@ffff985607df6940 x1874669228366208/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829150 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829125.877264:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6940 x1874669228366208/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829150 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829125.877272:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829125.877276:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829125.877279:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228366208:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787829125.877285:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228366336:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787829125.877300:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829125.877311:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829125/real 1787829125] req@ffff985607df4780 x1874669228366336/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829150 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829125.877319:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4780 x1874669228366336/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829150 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829125.877325:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829125.877328:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829125.877330:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228366336:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787829125.940870:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829126.836798:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829126.964857:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829127.860775:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829127.988875:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829128.884801:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829129.012865:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829129.908824:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829130.036877:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829130.932791:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787829130.996843:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829130.996858:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829130.996864:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829130.996878:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5276 00000100:00080000:1.0:1787829130.996893:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829130.996908:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787829130.996962:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a03c0 x1874669228366464/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829130.997000:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829130.997005:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829130.997007:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829130.997011:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5276 00000100:00080000:1.0:1787829130.997015:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829130.997020:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787829130.997032:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829130.997047:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0780 x1874669228366592/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829130.997058:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829130.997062:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829130.997065:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829130.997069:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5276 00000100:00080000:1.0:1787829130.997074:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829130.997081:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787829130.997081:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228366464:192.168.203.114@tcp:250:kworker.0 00000008:00100000:1.0:1787829130.997091:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829130.997105:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a1680 x1874669228366720/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829130.997192:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829130.997259:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829130/real 1787829130] req@ffff9856085a03c0 x1874669228366464/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829155 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829130.997280:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a03c0 x1874669228366464/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829155 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829130.997292:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829130.997297:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829130.997302:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228366464:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787829130.997314:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228366592:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829130.997340:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829130.997354:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829130/real 1787829130] req@ffff9856085a0780 x1874669228366592/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829155 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829130.997368:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0780 x1874669228366592/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829155 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829130.997377:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829130.997381:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829130.997385:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228366592:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787829130.997427:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228366720:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829130.997448:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829130.997458:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829130/real 1787829130] req@ffff9856085a1680 x1874669228366720/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829155 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829130.997470:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a1680 x1874669228366720/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829155 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829130.997479:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829130.997483:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829130.997486:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228366720:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787829131.060903:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829131.956786:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829132.084875:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829132.980859:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829133.108870:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829134.004823:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829134.132938:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829135.028817:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829135.156826:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829136.052865:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787829136.116845:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829136.116855:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829136.116861:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829136.116872:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5281 00000100:00080000:3.0:1787829136.116884:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829136.116891:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787829136.116932:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83480 x1874669228366848/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829136.116960:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829136.116963:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829136.116965:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829136.116967:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5281 00000100:00080000:3.0:1787829136.116970:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829136.116972:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829136.116981:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829136.116991:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f821c0 x1874669228366976/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829136.116999:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829136.117003:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829136.117005:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1787829136.117007:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228366848:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787829136.117010:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5281 00000100:00080000:3.0:1787829136.117017:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829136.117020:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829136.117023:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829136.117033:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82580 x1874669228367104/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787829136.117086:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829136.117123:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829136/real 1787829136] req@ffff985610f83480 x1874669228366848/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829161 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829136.117150:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83480 x1874669228366848/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829161 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829136.117164:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829136.117170:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829136.117175:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228366848:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787829136.117203:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f821c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228366976:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787829136.117234:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829136.117246:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829136/real 1787829136] req@ffff985610f821c0 x1874669228366976/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829161 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829136.117256:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f821c0 x1874669228366976/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829161 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829136.117263:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829136.117265:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829136.117268:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f821c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228366976:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787829136.117272:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228367104:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787829136.117287:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829136.117294:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829136/real 1787829136] req@ffff985610f82580 x1874669228367104/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829161 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829136.117302:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82580 x1874669228367104/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829161 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829136.117307:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829136.117309:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829136.117311:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228367104:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787829136.180870:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829137.076803:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829137.204880:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829138.100813:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829138.228793:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829139.124787:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829139.252884:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829140.148842:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829140.276933:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829141.172767:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787829141.236831:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829141.236850:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829141.236855:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829141.236869:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5286 00000100:00080000:3.0:1787829141.236882:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829141.236899:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787829141.236951:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82940 x1874669228367232/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829141.236984:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829141.236989:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829141.236991:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829141.236995:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5286 00000100:00080000:3.0:1787829141.237000:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829141.237005:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829141.237017:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829141.237033:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228367232:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787829141.237034:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82d00 x1874669228367360/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829141.237057:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829141.237061:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829141.237063:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829141.237068:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5286 00000100:00080000:3.0:1787829141.237072:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829141.237078:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829141.237087:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829141.237099:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80f00 x1874669228367488/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787829141.237169:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228367360:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787829141.237196:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228367488:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787829141.300845:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829142.196829:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829142.324881:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829143.220775:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829143.348831:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829144.244855:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000400:00000100:1.0:1787829144.309077:0:2372:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.114:988 00000400:00000100:1.0:1787829144.309153:0:2372:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.114@tcp at host 192.168.203.114 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1787829144.309173:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:1.0:1787829144.309225:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:1.0:1787829144.309236:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000100:00100000:3.0:1787829144.309324:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829141/real 1787829144] req@ffff985610f82d00 x1874669228367360/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829166 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829144.309362:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82d00 x1874669228367360/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829166 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829144.309375:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829144.309380:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829144.309385:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228367360:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787829144.309395:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829141/real 1787829144] req@ffff985610f80f00 x1874669228367488/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829166 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829144.309409:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80f00 x1874669228367488/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829166 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829144.309418:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829144.309422:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829144.309426:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228367488:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787829144.309434:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829141/real 1787829144] req@ffff985610f82940 x1874669228367232/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829166 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829144.309447:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82940 x1874669228367232/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829166 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829144.309456:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829144.309460:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829144.309469:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228367232:192.168.203.114@tcp:250:kworker.0 00010000:00010000:1.0:1787829144.372859:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829145.268762:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829145.396826:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829146.292818:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787829146.356885:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829146.356901:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829146.356906:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829146.356919:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5291 00000100:00080000:1.0:1787829146.356932:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829146.356942:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787829146.356990:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3c00 x1874669228367616/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829146.357027:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829146.357032:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829146.357050:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829146.357055:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5291 00000100:00080000:1.0:1787829146.357059:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829146.357065:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787829146.357092:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829146.357107:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0b40 x1874669228367744/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829146.357121:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1787829146.357121:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228367616:192.168.203.114@tcp:250:kworker.0 00000100:00080000:1.0:1787829146.357132:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829146.357137:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829146.357142:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5291 00000100:00080000:1.0:1787829146.357147:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829146.357152:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787829146.357158:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829146.357176:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0000 x1874669228367872/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829146.357233:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829146.357279:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829146/real 1787829146] req@ffff9856085a3c00 x1874669228367616/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829171 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829146.357304:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3c00 x1874669228367616/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829171 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829146.357317:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829146.357322:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829146.357327:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228367616:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787829146.357358:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228367744:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829146.357387:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829146.357404:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829146/real 1787829146] req@ffff9856085a0b40 x1874669228367744/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829171 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829146.357427:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0b40 x1874669228367744/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829171 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829146.357437:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829146.357440:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829146.357444:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228367744:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787829146.357452:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228367872:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829146.357474:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829146.357486:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829146/real 1787829146] req@ffff9856085a0000 x1874669228367872/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829171 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829146.357499:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0000 x1874669228367872/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829171 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829146.357508:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829146.357511:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829146.357515:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228367872:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787829146.420834:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829147.316774:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829147.444872:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829148.340779:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829148.468817:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829149.364849:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829149.492961:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829150.388823:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829150.516872:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829151.412881:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787829151.476939:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829151.476952:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829151.476957:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829151.476973:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5297 00000100:00080000:0.0:1787829151.476989:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829151.477000:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787829151.477099:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4000 x1874669228368000/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829151.477133:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829151.477137:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829151.477138:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829151.477142:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5297 00000100:00080000:0.0:1787829151.477146:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829151.477150:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787829151.477162:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829151.477194:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7c00 x1874669228368128/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829151.477206:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829151.477211:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829151.477213:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829151.477217:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5297 00000100:00080000:0.0:1787829151.477224:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787829151.477227:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228368000:192.168.203.114@tcp:250:kworker.0 00000100:00080000:0.0:1787829151.477232:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787829151.477243:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829151.477261:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4780 x1874669228368256/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829151.477323:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829151.477366:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829151/real 1787829151] req@ffff985607df4000 x1874669228368000/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829176 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829151.477386:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4000 x1874669228368000/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829176 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829151.477395:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829151.477398:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829151.477401:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228368000:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787829151.477408:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228368128:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829151.477435:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829151.477444:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829151/real 1787829151] req@ffff985607df7c00 x1874669228368128/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829176 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829151.477452:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7c00 x1874669228368128/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829176 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829151.477457:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829151.477460:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829151.477462:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228368128:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787829151.477484:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228368256:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829151.477496:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829151.477502:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829151/real 1787829151] req@ffff985607df4780 x1874669228368256/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829176 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829151.477509:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4780 x1874669228368256/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829176 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829151.477514:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829151.477516:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829151.477518:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228368256:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787829151.540954:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829152.436854:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829152.564967:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829153.460819:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829153.588862:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829154.484807:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829154.612945:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829155.508851:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829155.636875:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829156.532810:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787829156.596818:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829156.596830:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829156.596834:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829156.596844:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5302 00000100:00080000:3.0:1787829156.596857:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829156.596864:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787829156.596908:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82940 x1874669228368384/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829156.596938:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829156.596941:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829156.596946:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829156.596949:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5302 00000100:00080000:3.0:1787829156.596952:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829156.596955:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829156.596965:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829156.596975:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80f00 x1874669228368512/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829156.596983:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829156.596986:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829156.596987:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829156.596990:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5302 00000100:00080000:3.0:1787829156.596992:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00100000:1.0:1787829156.597010:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228368384:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787829156.597010:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829156.597020:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829156.597032:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82d00 x1874669228368640/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787829156.597089:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829156.597126:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829156/real 1787829156] req@ffff985610f82940 x1874669228368384/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829181 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829156.597141:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82940 x1874669228368384/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829181 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829156.597149:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829156.597153:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829156.597156:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228368384:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787829156.597161:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228368512:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787829156.597177:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829156.597189:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829156/real 1787829156] req@ffff985610f80f00 x1874669228368512/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829181 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829156.597201:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80f00 x1874669228368512/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829181 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829156.597211:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829156.597215:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829156.597219:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228368512:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787829156.597249:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228368640:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787829156.597266:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829156.597274:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829156/real 1787829156] req@ffff985610f82d00 x1874669228368640/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829181 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829156.597282:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82d00 x1874669228368640/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829181 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829156.597288:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829156.597290:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829156.597292:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228368640:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787829156.660896:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829157.556823:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829157.684924:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829158.580763:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829158.708851:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829159.604823:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829159.732915:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829160.628786:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829160.756910:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829161.652805:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787829161.716910:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829161.716924:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829161.716929:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829161.716943:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5307 00000100:00080000:3.0:1787829161.716956:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829161.716986:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787829161.717036:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81e00 x1874669228368768/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829161.717069:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829161.717073:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829161.717075:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829161.717078:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5307 00000100:00080000:3.0:1787829161.717082:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829161.717086:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829161.717098:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829161.717110:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f803c0 x1874669228368896/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829161.717122:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829161.717130:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:1.0:1787829161.717131:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228368768:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787829161.717137:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829161.717152:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5307 00000100:00080000:3.0:1787829161.717156:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829161.717159:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829161.717162:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829161.717173:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83840 x1874669228369024/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787829161.717221:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829161.717257:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829161/real 1787829161] req@ffff985610f81e00 x1874669228368768/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829186 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829161.717272:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81e00 x1874669228368768/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829186 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829161.717285:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829161.717293:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829161.717297:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228368768:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787829161.717328:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228368896:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787829161.717367:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829161.717383:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829161/real 1787829161] req@ffff985610f803c0 x1874669228368896/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829186 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829161.717394:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f803c0 x1874669228368896/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829186 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829161.717401:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829161.717415:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829161.717419:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228368896:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787829161.717425:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228369024:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787829161.717442:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829161.717452:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829161/real 1787829161] req@ffff985610f83840 x1874669228369024/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829186 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829161.717471:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83840 x1874669228369024/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829186 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829161.717495:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829161.717500:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829161.717506:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228369024:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787829161.780852:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829162.676746:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829162.804862:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829163.700812:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829163.828795:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829164.724765:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829164.852921:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829165.748794:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829165.876780:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829166.772745:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787829166.836812:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829166.836822:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829166.836825:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829166.836833:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5312 00000100:00080000:0.0:1787829166.836842:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829166.836848:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787829166.836878:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6940 x1874669228369152/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829166.836902:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829166.836905:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829166.836906:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829166.836909:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5312 00000100:00080000:0.0:1787829166.836911:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829166.836914:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787829166.836933:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829166.836948:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4b40 x1874669228369280/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787829166.836961:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228369152:192.168.203.114@tcp:250:kworker.0 00000100:00080000:0.0:1787829166.836971:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829166.836974:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829166.836976:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829166.836980:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5312 00000100:00080000:0.0:1787829166.836982:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829166.836987:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787829166.836993:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829166.837010:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7480 x1874669228369408/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787829166.837048:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829166.837085:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829166/real 1787829166] req@ffff985607df6940 x1874669228369152/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829191 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829166.837100:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6940 x1874669228369152/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829191 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829166.837109:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829166.837114:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829166.837117:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228369152:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787829166.837142:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228369280:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787829166.837158:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829166.837167:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829166/real 1787829166] req@ffff985607df4b40 x1874669228369280/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829191 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829166.837175:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4b40 x1874669228369280/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829191 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829166.837181:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829166.837184:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829166.837186:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228369280:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787829166.837190:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228369408:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787829166.837201:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829166.837207:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829166/real 1787829166] req@ffff985607df7480 x1874669228369408/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829191 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829166.837214:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7480 x1874669228369408/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829191 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829166.837225:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829166.837227:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829166.837229:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228369408:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787829166.900899:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829167.796791:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829167.924912:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829168.820874:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829168.948872:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829169.844806:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829169.972914:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829170.868842:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829170.996843:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829171.892847:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787829171.956825:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829171.956836:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829171.956841:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829171.956852:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5317 00000100:00080000:3.0:1787829171.956864:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829171.956871:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787829171.956931:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81a40 x1874669228369536/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829171.956975:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829171.956980:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829171.956987:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829171.956991:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5317 00000100:00080000:3.0:1787829171.956995:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829171.956999:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829171.957010:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829171.957024:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81680 x1874669228369664/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829171.957039:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:1.0:1787829171.957041:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228369536:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787829171.957046:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829171.957052:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829171.957058:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5317 00000100:00080000:3.0:1787829171.957063:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829171.957066:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829171.957072:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829171.957087:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f812c0 x1874669228369792/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787829171.957124:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829171.957162:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829171/real 1787829171] req@ffff985610f81a40 x1874669228369536/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829196 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829171.957200:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81a40 x1874669228369536/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829196 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829171.957215:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829171.957220:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829171.957226:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228369536:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787829171.957258:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228369664:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787829171.957304:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829171.957320:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829171/real 1787829171] req@ffff985610f81680 x1874669228369664/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829196 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829171.957333:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81680 x1874669228369664/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829196 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829171.957363:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829171.957369:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829171.957375:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228369664:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787829171.957385:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228369792:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787829171.957417:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829171.957437:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829171/real 1787829171] req@ffff985610f812c0 x1874669228369792/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829196 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829171.957458:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f812c0 x1874669228369792/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829196 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829171.957467:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829171.957470:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829171.957474:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228369792:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787829172.020966:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829172.916832:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829173.044946:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829173.940802:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829174.068932:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829174.964867:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829175.092847:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829175.988785:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829176.116895:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829177.012884:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787829177.076870:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829177.076880:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829177.076885:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829177.076896:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5322 00000100:00080000:1.0:1787829177.076909:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829177.076916:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787829177.076959:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2d00 x1874669228369920/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829177.076985:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829177.076988:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829177.076990:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829177.076993:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5322 00000100:00080000:1.0:1787829177.076995:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829177.076998:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787829177.077007:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829177.077019:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0000 x1874669228370048/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829177.077027:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829177.077031:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:3.0:1787829177.077032:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228369920:192.168.203.114@tcp:250:kworker.0 00000100:00080000:1.0:1787829177.077033:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829177.077040:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5322 00000100:00080000:1.0:1787829177.077044:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829177.077046:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787829177.077056:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829177.077068:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0b40 x1874669228370176/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829177.077121:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829177.077160:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829177/real 1787829177] req@ffff9856085a2d00 x1874669228369920/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829202 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829177.077179:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2d00 x1874669228369920/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829202 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829177.077188:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829177.077192:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829177.077195:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228369920:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787829177.077218:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228370048:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829177.077235:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829177.077243:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829177/real 1787829177] req@ffff9856085a0000 x1874669228370048/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829202 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829177.077252:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0000 x1874669228370048/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829202 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829177.077257:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829177.077260:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829177.077262:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228370048:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787829177.077266:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228370176:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829177.077277:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829177.077282:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829177/real 1787829177] req@ffff9856085a0b40 x1874669228370176/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829202 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829177.077289:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0b40 x1874669228370176/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829202 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829177.077295:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829177.077297:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829177.077299:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228370176:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787829177.140923:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829178.036793:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829178.164894:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829179.060785:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829179.188832:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829180.084781:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829180.212892:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829181.108811:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829181.236803:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829182.132798:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787829182.196831:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829182.196845:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829182.196851:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829182.196864:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5327 00000100:00080000:3.0:1787829182.196876:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829182.196884:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787829182.196926:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80b40 x1874669228370304/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829182.196953:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829182.196956:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829182.196958:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829182.196962:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5327 00000100:00080000:3.0:1787829182.196965:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829182.196969:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829182.196979:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829182.196989:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228370304:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787829182.196991:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80000 x1874669228370432/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829182.197004:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829182.197007:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829182.197008:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829182.197011:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5327 00000100:00080000:3.0:1787829182.197020:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829182.197023:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829182.197026:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829182.197040:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f830c0 x1874669228370560/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787829182.197072:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829182.197109:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829182/real 1787829182] req@ffff985610f80b40 x1874669228370304/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829207 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829182.197125:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80b40 x1874669228370304/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829207 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829182.197134:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829182.197138:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829182.197141:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228370304:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787829182.197164:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228370432:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787829182.197180:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829182.197189:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829182/real 1787829182] req@ffff985610f80000 x1874669228370432/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829207 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829182.197197:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80000 x1874669228370432/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829207 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829182.197203:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829182.197206:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829182.197208:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228370432:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787829182.197211:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228370560:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787829182.197221:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829182.197227:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829182/real 1787829182] req@ffff985610f830c0 x1874669228370560/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829207 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829182.197235:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f830c0 x1874669228370560/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829207 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829182.197240:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829182.197242:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829182.197244:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228370560:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787829182.260842:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829183.156913:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829183.284835:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829184.180776:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829184.308802:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829185.204777:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829185.332842:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829186.228932:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829186.356873:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829187.252866:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787829187.316830:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829187.316841:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829187.316846:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829187.316857:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5332 00000100:00080000:3.0:1787829187.316869:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829187.316876:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787829187.316918:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83c00 x1874669228370688/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829187.316953:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829187.316956:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829187.316957:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829187.316960:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5332 00000100:00080000:3.0:1787829187.316963:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829187.316965:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829187.316974:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829187.316984:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80780 x1874669228370816/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829187.317002:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829187.317006:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829187.317007:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829187.317010:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5332 00000100:00080000:3.0:1787829187.317012:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829187.317014:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829187.317017:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829187.317029:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f830c0 x1874669228370944/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787829187.317030:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228370688:192.168.203.114@tcp:250:kworker.0 00000800:00000100:1.0:1787829187.317144:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829187.317180:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829187/real 1787829187] req@ffff985610f83c00 x1874669228370688/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829212 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829187.317207:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83c00 x1874669228370688/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829212 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829187.317216:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829187.317219:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829187.317222:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228370688:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787829187.317228:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228370816:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787829187.317243:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829187.317251:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829187/real 1787829187] req@ffff985610f80780 x1874669228370816/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829212 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829187.317259:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80780 x1874669228370816/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829212 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829187.317265:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829187.317267:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829187.317269:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228370816:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787829187.317291:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228370944:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787829187.317305:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829187.317311:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829187/real 1787829187] req@ffff985610f830c0 x1874669228370944/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829212 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829187.317319:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f830c0 x1874669228370944/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829212 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829187.317324:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829187.317326:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829187.317329:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228370944:192.168.203.114@tcp:38:kworker.0 00010000:00010000:1.0:1787829187.380882:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829188.276808:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829188.404929:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829189.300799:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829189.428801:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829190.324799:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829190.452865:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829191.348803:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829191.476836:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829192.372821:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787829192.436883:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829192.436895:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829192.436899:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829192.436911:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5338 00000100:00080000:1.0:1787829192.436924:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829192.436931:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787829192.436983:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3c00 x1874669228371072/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829192.437030:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829192.437034:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829192.437037:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829192.437042:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5338 00000100:00080000:1.0:1787829192.437047:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829192.437053:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787829192.437060:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228371072:192.168.203.114@tcp:250:kworker.0 00000008:00100000:1.0:1787829192.437066:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829192.437085:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a1680 x1874669228371200/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829192.437107:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829192.437112:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829192.437114:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829192.437118:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5338 00000100:00080000:1.0:1787829192.437123:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829192.437127:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787829192.437132:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829192.437144:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0780 x1874669228371328/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829192.437150:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829192.437191:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829192/real 1787829192] req@ffff9856085a3c00 x1874669228371072/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829217 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829192.437209:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3c00 x1874669228371072/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829217 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829192.437218:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829192.437221:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829192.437225:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228371072:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787829192.437258:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228371200:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829192.437282:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829192.437291:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829192/real 1787829192] req@ffff9856085a1680 x1874669228371200/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829217 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829192.437308:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a1680 x1874669228371200/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829217 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829192.437314:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829192.437317:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829192.437319:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228371200:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787829192.437322:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228371328:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829192.437333:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829192.437339:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829192/real 1787829192] req@ffff9856085a0780 x1874669228371328/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829217 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829192.437347:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0780 x1874669228371328/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829217 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829192.437357:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829192.437358:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829192.437360:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228371328:192.168.203.114@tcp:38:kworker.0 00010000:00010000:3.0:1787829192.500887:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829193.396801:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829193.524952:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829194.420780:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829194.548852:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829195.444820:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829195.572921:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829196.468871:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829196.596876:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829197.492803:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787829197.556853:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829197.556862:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829197.556867:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829197.556879:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5343 00000100:00080000:3.0:1787829197.556890:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829197.556907:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787829197.556945:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80000 x1874669228371456/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829197.556977:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829197.556982:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829197.556985:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829197.556990:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5343 00000100:00080000:3.0:1787829197.556996:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829197.557001:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829197.557023:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829197.557041:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80b40 x1874669228371584/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787829197.557042:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228371456:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787829197.557063:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829197.557069:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829197.557071:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829197.557102:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5343 00000100:00080000:3.0:1787829197.557107:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829197.557114:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829197.557121:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829197.557138:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f812c0 x1874669228371712/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787829197.557140:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829197.557171:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829197/real 1787829197] req@ffff985610f80000 x1874669228371456/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829222 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829197.557186:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80000 x1874669228371456/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829222 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829197.557196:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829197.557199:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829197.557212:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228371456:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829197.557253:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228371584:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829197.557291:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829197.557305:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829197/real 1787829197] req@ffff985610f80b40 x1874669228371584/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829222 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829197.557317:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80b40 x1874669228371584/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829222 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829197.557325:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829197.557328:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829197.557331:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228371584:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829197.557572:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228371712:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829197.557604:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829197.557618:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829197/real 1787829197] req@ffff985610f812c0 x1874669228371712/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829222 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829197.557628:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f812c0 x1874669228371712/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829222 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829197.557634:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829197.557637:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829197.557640:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228371712:192.168.203.114@tcp:38:kworker.0 00010000:00010000:3.0:1787829197.620879:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829198.516813:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829198.644847:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829199.540849:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829199.668864:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829200.564836:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829200.692909:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829201.588871:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829201.716876:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829202.612812:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787829202.676848:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829202.676859:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829202.676864:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829202.676874:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5348 00000100:00080000:1.0:1787829202.676886:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829202.676894:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787829202.676937:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a03c0 x1874669228371840/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829202.676967:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829202.676972:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829202.676975:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829202.676979:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5348 00000100:00080000:1.0:1787829202.676984:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829202.676990:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787829202.677002:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829202.677016:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3840 x1874669228371968/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787829202.677030:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228371840:192.168.203.114@tcp:250:kworker.0 00000100:00080000:1.0:1787829202.677031:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829202.677037:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829202.677043:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829202.677050:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5348 00000100:00080000:1.0:1787829202.677058:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829202.677063:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787829202.677066:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829202.677076:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2580 x1874669228372096/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787829202.677142:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829202.677181:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829202/real 1787829202] req@ffff9856085a03c0 x1874669228371840/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829227 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829202.677201:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a03c0 x1874669228371840/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829227 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829202.677214:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829202.677219:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829202.677224:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228371840:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829202.677249:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228371968:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829202.677274:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829202.677288:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829202/real 1787829202] req@ffff9856085a3840 x1874669228371968/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829227 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829202.677302:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3840 x1874669228371968/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829227 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829202.677311:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829202.677315:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829202.677319:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228371968:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829202.677326:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228372096:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829202.677348:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829202.677362:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829202/real 1787829202] req@ffff9856085a2580 x1874669228372096/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829227 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829202.677382:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2580 x1874669228372096/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829227 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829202.677391:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829202.677395:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829202.677398:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228372096:192.168.203.114@tcp:38:kworker.0 00010000:00010000:3.0:1787829202.740875:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829203.636814:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829203.764934:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829204.660889:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829204.788874:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829205.684810:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829205.812879:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829206.708793:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829206.836834:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829207.732831:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787829207.796791:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829207.796814:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829207.796819:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829207.796828:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5353 00000100:00080000:1.0:1787829207.796840:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829207.796847:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787829207.796883:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a12c0 x1874669228372224/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829207.796912:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829207.796916:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829207.796926:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829207.796930:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5353 00000100:00080000:1.0:1787829207.796939:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829207.796946:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787829207.796973:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829207.796987:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a1a40 x1874669228372352/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787829207.796997:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228372224:192.168.203.114@tcp:250:kworker.0 00000100:00080000:1.0:1787829207.797002:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829207.797008:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829207.797010:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829207.797014:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5353 00000100:00080000:1.0:1787829207.797017:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829207.797020:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787829207.797025:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829207.797036:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2940 x1874669228372480/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787829207.797092:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228372352:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829207.797125:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228372480:192.168.203.114@tcp:38:kworker.0 00010000:00010000:3.0:1787829207.860838:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829208.756754:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829208.884859:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829209.780792:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829209.908834:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829210.804791:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000400:00000100:3.0:1787829210.868995:0:2373:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.114:988 00000400:00000100:3.0:1787829210.869041:0:2373:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.114@tcp at host 192.168.203.114 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:3.0:1787829210.869058:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:3.0:1787829210.869096:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:3.0:1787829210.869106:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000100:00100000:0.0:1787829210.869156:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829207/real 1787829210] req@ffff9856085a1a40 x1874669228372352/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829232 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829210.869182:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a1a40 x1874669228372352/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829232 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829210.869202:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829210.869208:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829210.869212:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228372352:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829210.869221:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829207/real 1787829210] req@ffff9856085a2940 x1874669228372480/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829232 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829210.869234:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2940 x1874669228372480/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829232 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829210.869243:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829210.869246:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829210.869257:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228372480:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829210.869264:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829207/real 1787829210] req@ffff9856085a12c0 x1874669228372224/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829232 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829210.869278:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a12c0 x1874669228372224/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829232 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829210.869286:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829210.869289:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829210.869293:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228372224:192.168.203.114@tcp:250:kworker.0 00010000:00010000:1.0:1787829210.932852:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829211.828791:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829211.956883:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829212.852772:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787829212.916793:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829212.916800:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829212.916804:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829212.916813:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5358 00000100:00080000:1.0:1787829212.916822:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829212.916829:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787829212.916859:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a21c0 x1874669228372608/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829212.916878:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829212.916880:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829212.916882:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829212.916884:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5358 00000100:00080000:1.0:1787829212.916887:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829212.916890:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787829212.916897:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829212.916907:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228372608:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787829212.916909:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3480 x1874669228372736/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829212.916928:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829212.916930:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829212.916931:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829212.916933:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5358 00000100:00080000:1.0:1787829212.916936:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829212.916938:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787829212.916941:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829212.916951:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a30c0 x1874669228372864/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787829212.916975:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829212.917020:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829212/real 1787829212] req@ffff9856085a21c0 x1874669228372608/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829237 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829212.917039:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a21c0 x1874669228372608/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829237 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829212.917049:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829212.917053:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829212.917057:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228372608:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829212.917080:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228372736:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829212.917106:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829212.917117:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829212/real 1787829212] req@ffff9856085a3480 x1874669228372736/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829237 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829212.917128:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3480 x1874669228372736/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829237 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829212.917137:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829212.917140:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829212.917143:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228372736:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829212.917150:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228372864:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829212.917170:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829212.917182:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829212/real 1787829212] req@ffff9856085a30c0 x1874669228372864/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829237 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829212.917199:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a30c0 x1874669228372864/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829237 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829212.917207:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829212.917210:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829212.917214:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228372864:192.168.203.114@tcp:38:kworker.0 00010000:00010000:3.0:1787829212.980888:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829213.876786:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829214.004939:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829214.900776:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829215.028884:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829215.924789:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829216.052828:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829216.948766:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829217.076800:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829217.972807:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787829218.036863:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829218.036871:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829218.036874:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829218.036882:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5363 00000100:00080000:1.0:1787829218.036890:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829218.036896:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787829218.036929:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0f00 x1874669228372992/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829218.036949:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829218.036952:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829218.036954:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829218.036957:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5363 00000100:00080000:1.0:1787829218.036960:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829218.036963:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787829218.036971:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829218.036979:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228372992:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787829218.036984:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a30c0 x1874669228373120/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829218.036993:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829218.036995:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829218.036996:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829218.036998:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5363 00000100:00080000:1.0:1787829218.037000:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829218.037004:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787829218.037007:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829218.037015:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3480 x1874669228373248/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787829218.037038:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829218.037073:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829218/real 1787829218] req@ffff9856085a0f00 x1874669228372992/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829243 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829218.037104:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0f00 x1874669228372992/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829243 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829218.037115:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829218.037119:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829218.037124:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228372992:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829218.037148:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228373120:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829218.037174:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829218.037184:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829218/real 1787829218] req@ffff9856085a30c0 x1874669228373120/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829243 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829218.037193:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a30c0 x1874669228373120/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829243 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829218.037199:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829218.037201:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829218.037204:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228373120:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829218.037207:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228373248:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829218.037219:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829218.037225:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829218/real 1787829218] req@ffff9856085a3480 x1874669228373248/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829243 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829218.037232:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3480 x1874669228373248/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829243 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829218.037237:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829218.037239:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829218.037240:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228373248:192.168.203.114@tcp:38:kworker.0 00010000:00010000:3.0:1787829218.100867:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829218.996773:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829219.124897:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829220.020834:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829220.148858:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829221.044807:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829221.172889:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829222.068839:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829222.196839:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829223.092782:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787829223.156853:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829223.156863:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829223.156867:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829223.156878:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5368 00000100:00080000:3.0:1787829223.156892:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829223.156902:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787829223.156952:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81680 x1874669228373376/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829223.156985:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829223.156990:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829223.156992:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829223.156997:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5368 00000100:00080000:3.0:1787829223.157001:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829223.157005:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829223.157017:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829223.157033:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81a40 x1874669228373504/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829223.157044:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829223.157049:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829223.157051:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829223.157056:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5368 00000100:00080000:3.0:1787829223.157062:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829223.157073:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787829223.157075:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228373376:192.168.203.114@tcp:250:kworker.0 00000008:00100000:3.0:1787829223.157080:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829223.157099:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83840 x1874669228373632/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787829223.157161:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829223.157203:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829223/real 1787829223] req@ffff985610f81680 x1874669228373376/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829248 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829223.157223:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81680 x1874669228373376/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829248 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829223.157232:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829223.157235:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829223.157239:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228373376:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829223.157245:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228373504:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829223.157270:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829223.157284:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829223/real 1787829223] req@ffff985610f81a40 x1874669228373504/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829248 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829223.157297:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81a40 x1874669228373504/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829248 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829223.157306:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829223.157310:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829223.157314:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228373504:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829223.157339:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228373632:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829223.157360:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829223.157368:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829223/real 1787829223] req@ffff985610f83840 x1874669228373632/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829248 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829223.157377:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83840 x1874669228373632/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829248 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829223.157382:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829223.157384:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829223.157386:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228373632:192.168.203.114@tcp:38:kworker.0 00010000:00010000:3.0:1787829223.220862:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829224.116826:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829224.244887:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829225.140799:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829225.268848:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829226.164987:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829226.292838:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829227.188780:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829227.316854:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829228.212799:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787829228.276831:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829228.276842:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829228.276846:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829228.276856:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5373 00000100:00080000:1.0:1787829228.276868:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829228.276875:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787829228.276919:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a21c0 x1874669228373760/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829228.276944:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829228.276947:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829228.276948:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829228.276951:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5373 00000100:00080000:1.0:1787829228.276953:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829228.276956:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787829228.276965:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829228.276974:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a12c0 x1874669228373888/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829228.276981:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829228.276985:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829228.276986:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829228.276989:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5373 00000100:00080000:1.0:1787829228.276991:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829228.276993:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787829228.276996:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829228.277003:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2940 x1874669228374016/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787829228.277032:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228373760:192.168.203.114@tcp:250:kworker.0 00000800:00000100:0.0:1787829228.277107:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829228.277144:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829228/real 1787829228] req@ffff9856085a21c0 x1874669228373760/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829253 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829228.277159:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a21c0 x1874669228373760/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829253 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829228.277168:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829228.277172:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829228.277175:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228373760:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829228.277180:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228373888:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829228.277198:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829228.277207:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829228/real 1787829228] req@ffff9856085a12c0 x1874669228373888/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829253 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829228.277225:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a12c0 x1874669228373888/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829253 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829228.277231:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829228.277233:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829228.277235:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228373888:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829228.277239:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228374016:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829228.277251:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829228.277258:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829228/real 1787829228] req@ffff9856085a2940 x1874669228374016/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829253 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829228.277265:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2940 x1874669228374016/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829253 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829228.277270:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829228.277272:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829228.277274:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228374016:192.168.203.114@tcp:38:kworker.0 00010000:00010000:3.0:1787829228.340849:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829229.236817:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829229.364852:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829230.260808:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829230.388879:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829231.284831:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829231.412940:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829232.308803:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829232.436863:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829233.332780:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787829233.396799:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829233.396809:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829233.396813:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829233.396825:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5379 00000100:00080000:3.0:1787829233.396837:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829233.396846:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787829233.396895:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f803c0 x1874669228374144/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829233.396929:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829233.396933:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829233.396935:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829233.396940:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5379 00000100:00080000:3.0:1787829233.396945:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829233.396951:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829233.396960:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829233.396980:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228374144:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787829233.396980:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81e00 x1874669228374272/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829233.396999:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829233.397002:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829233.397005:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829233.397010:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5379 00000100:00080000:3.0:1787829233.397015:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829233.397021:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829233.397028:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829233.397042:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82d00 x1874669228374400/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787829233.397057:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829233.397099:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829233/real 1787829233] req@ffff985610f803c0 x1874669228374144/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829258 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829233.397118:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f803c0 x1874669228374144/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829258 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829233.397130:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829233.397135:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829233.397139:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228374144:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787829233.397164:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228374272:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787829233.397190:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829233.397204:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829233/real 1787829233] req@ffff985610f81e00 x1874669228374272/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829258 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829233.397225:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81e00 x1874669228374272/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829258 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829233.397234:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829233.397238:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829233.397242:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228374272:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787829233.397248:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228374400:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787829233.397270:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829233.397282:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829233/real 1787829233] req@ffff985610f82d00 x1874669228374400/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829258 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829233.397295:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82d00 x1874669228374400/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829258 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829233.397304:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829233.397307:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829233.397311:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228374400:192.168.203.114@tcp:38:kworker.0 00010000:00010000:3.0:1787829233.460924:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829234.356808:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829234.484908:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829235.380837:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829235.508814:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829236.404790:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829236.532901:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829237.428820:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829237.556838:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829238.452802:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787829238.516744:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829238.516752:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829238.516755:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829238.516764:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5384 00000100:00080000:3.0:1787829238.516774:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829238.516786:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787829238.516815:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80f00 x1874669228374528/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829238.516834:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829238.516837:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829238.516838:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829238.516841:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5384 00000100:00080000:3.0:1787829238.516843:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829238.516846:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829238.516853:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829238.516861:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82940 x1874669228374656/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829238.516870:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829238.516874:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829238.516877:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829238.516881:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5384 00000100:00080000:3.0:1787829238.516885:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829238.516889:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829238.516894:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829238.516904:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228374528:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787829238.516909:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82580 x1874669228374784/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787829238.516962:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829238.516988:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829238/real 1787829238] req@ffff985610f80f00 x1874669228374528/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829263 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829238.517001:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80f00 x1874669228374528/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829263 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829238.517009:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829238.517012:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829238.517015:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228374528:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787829238.517020:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228374656:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787829238.517033:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829238.517041:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829238/real 1787829238] req@ffff985610f82940 x1874669228374656/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829263 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829238.517049:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82940 x1874669228374656/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829263 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829238.517054:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829238.517056:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829238.517058:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228374656:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787829238.517077:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228374784:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787829238.517087:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829238.517093:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829238/real 1787829238] req@ffff985610f82580 x1874669228374784/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829263 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829238.517100:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82580 x1874669228374784/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829263 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829238.517105:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829238.517107:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829238.517109:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228374784:192.168.203.114@tcp:38:kworker.0 00010000:00010000:3.0:1787829238.580911:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829239.476837:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829239.604863:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829240.500800:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829240.628871:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829241.524804:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829241.652877:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829242.548768:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829242.676886:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829243.572816:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787829243.636856:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829243.636867:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829243.636872:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829243.636882:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5389 00000100:00080000:0.0:1787829243.636894:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829243.636901:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787829243.636973:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df52c0 x1874669228374912/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829243.637003:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829243.637006:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829243.637008:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829243.637011:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5389 00000100:00080000:0.0:1787829243.637014:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829243.637018:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787829243.637029:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829243.637058:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7840 x1874669228375040/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829243.637069:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829243.637074:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829243.637076:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1787829243.637087:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228374912:192.168.203.114@tcp:250:kworker.0 00000100:00080000:0.0:1787829243.637089:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5389 00000100:00080000:0.0:1787829243.637098:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829243.637104:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787829243.637110:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829243.637123:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4f00 x1874669228375168/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787829243.637200:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829243.637259:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829243/real 1787829243] req@ffff985607df52c0 x1874669228374912/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829268 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829243.637275:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df52c0 x1874669228374912/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829268 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829243.637285:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829243.637289:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829243.637292:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228374912:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787829243.637298:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228375040:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787829243.637316:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829243.637325:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829243/real 1787829243] req@ffff985607df7840 x1874669228375040/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829268 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829243.637333:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7840 x1874669228375040/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829268 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829243.637340:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829243.637341:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829243.637344:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228375040:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787829243.637366:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228375168:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787829243.637378:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829243.637386:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829243/real 1787829243] req@ffff985607df4f00 x1874669228375168/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829268 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829243.637416:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4f00 x1874669228375168/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829268 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829243.637423:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829243.637425:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829243.637428:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228375168:192.168.203.114@tcp:38:kworker.0 00010000:00010000:3.0:1787829243.700970:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829244.596830:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829244.724897:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829245.620793:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829245.748878:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829246.644882:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829246.773012:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829247.668824:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829247.796838:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829248.692844:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787829248.756833:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829248.756844:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829248.756849:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829248.756859:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5394 00000100:00080000:3.0:1787829248.756884:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829248.756891:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787829248.756935:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f821c0 x1874669228375296/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829248.756960:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829248.756964:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829248.756965:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829248.756968:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5394 00000100:00080000:3.0:1787829248.756971:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829248.756975:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829248.756985:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829248.756995:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83480 x1874669228375424/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829248.757003:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829248.757007:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829248.757008:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829248.757011:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5394 00000100:00080000:3.0:1787829248.757023:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00100000:1.0:1787829248.757028:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f821c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228375296:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787829248.757029:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829248.757044:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829248.757057:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82580 x1874669228375552/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787829248.757113:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829248.757171:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829248/real 1787829248] req@ffff985610f821c0 x1874669228375296/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829273 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829248.757189:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f821c0 x1874669228375296/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829273 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829248.757199:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829248.757205:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829248.757210:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f821c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228375296:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787829248.757242:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228375424:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787829248.757268:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829248.757280:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829248/real 1787829248] req@ffff985610f83480 x1874669228375424/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829273 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829248.757290:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83480 x1874669228375424/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829273 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829248.757298:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829248.757301:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829248.757303:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228375424:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787829248.757308:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228375552:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787829248.757321:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829248.757328:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829248/real 1787829248] req@ffff985610f82580 x1874669228375552/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829273 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829248.757337:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82580 x1874669228375552/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829273 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829248.757343:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829248.757345:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829248.757347:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228375552:192.168.203.114@tcp:38:kworker.0 00010000:00010000:3.0:1787829248.820972:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829249.716822:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829249.844960:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829250.740840:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829250.868863:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829251.764785:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829251.892936:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829252.788862:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829252.916841:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829253.812800:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787829253.876819:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829253.876833:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829253.876839:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829253.876852:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5399 00000100:00080000:3.0:1787829253.876866:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829253.876875:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787829253.876931:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82940 x1874669228375680/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829253.876966:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829253.876972:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829253.876974:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829253.876979:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5399 00000100:00080000:3.0:1787829253.876983:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829253.876987:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829253.876998:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829253.877010:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80f00 x1874669228375808/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829253.877020:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829253.877023:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829253.877026:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1787829253.877027:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228375680:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787829253.877032:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5399 00000100:00080000:3.0:1787829253.877036:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829253.877039:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829253.877042:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829253.877052:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82d00 x1874669228375936/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787829253.877105:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829253.877141:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829253/real 1787829253] req@ffff985610f82940 x1874669228375680/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829278 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829253.877156:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82940 x1874669228375680/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829278 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829253.877165:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829253.877168:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829253.877172:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228375680:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787829253.877195:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228375808:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787829253.877211:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829253.877220:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829253/real 1787829253] req@ffff985610f80f00 x1874669228375808/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829278 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829253.877228:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80f00 x1874669228375808/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829278 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829253.877234:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829253.877237:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829253.877239:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228375808:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787829253.877242:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228375936:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787829253.877253:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829253.877258:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829253/real 1787829253] req@ffff985610f82d00 x1874669228375936/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829278 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829253.877266:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82d00 x1874669228375936/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829278 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829253.877271:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829253.877273:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829253.877274:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228375936:192.168.203.114@tcp:38:kworker.0 00010000:00010000:3.0:1787829253.940957:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829254.836803:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829254.964867:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829255.860773:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829255.988961:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829256.884779:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829257.012898:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829257.908761:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829258.036845:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829258.932780:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787829258.996795:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829258.996801:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829258.996805:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829258.996812:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5404 00000100:00080000:3.0:1787829258.996821:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829258.996828:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787829258.996868:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81e00 x1874669228376064/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829258.996890:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829258.996892:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829258.996894:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829258.996896:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5404 00000100:00080000:3.0:1787829258.996899:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829258.996901:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829258.996908:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829258.996920:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f803c0 x1874669228376192/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829258.996932:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829258.996936:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829258.996938:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829258.996944:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5404 00000100:00100000:1.0:1787829258.996946:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228376064:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787829258.996949:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829258.996957:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829258.996963:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829258.996977:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83840 x1874669228376320/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787829258.997013:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829258.997042:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829258/real 1787829258] req@ffff985610f81e00 x1874669228376064/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829283 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829258.997074:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81e00 x1874669228376064/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829283 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829258.997083:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829258.997086:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829258.997089:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228376064:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787829258.997095:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228376192:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787829258.997113:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829258.997127:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829258/real 1787829258] req@ffff985610f803c0 x1874669228376192/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829283 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829258.997140:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f803c0 x1874669228376192/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829283 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829258.997149:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829258.997153:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829258.997156:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228376192:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787829258.997180:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228376320:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787829258.997196:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787829258.997204:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829258/real 1787829258] req@ffff985610f83840 x1874669228376320/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829283 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829258.997217:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83840 x1874669228376320/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829283 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829258.997222:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829258.997224:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787829258.997226:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228376320:192.168.203.114@tcp:38:kworker.0 00010000:00010000:3.0:1787829259.060949:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829259.956854:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829260.084991:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829260.980927:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829261.108821:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829262.004853:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829262.132912:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829263.028810:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829263.156804:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829264.052803:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787829264.116865:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829264.116877:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829264.116881:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829264.116892:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5409 00000100:00080000:1.0:1787829264.116905:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829264.116912:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787829264.116980:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a1a40 x1874669228376448/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829264.117007:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829264.117010:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829264.117011:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829264.117014:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5409 00000100:00080000:1.0:1787829264.117017:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829264.117039:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787829264.117052:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829264.117068:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2580 x1874669228376576/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787829264.117080:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228376448:192.168.203.114@tcp:250:kworker.0 00000100:00080000:1.0:1787829264.117088:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829264.117095:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829264.117098:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829264.117104:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5409 00000100:00080000:1.0:1787829264.117109:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829264.117114:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787829264.117120:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829264.117152:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3840 x1874669228376704/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787829264.117174:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829264.117211:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829264/real 1787829264] req@ffff9856085a1a40 x1874669228376448/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829289 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829264.117224:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a1a40 x1874669228376448/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829289 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829264.117233:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829264.117237:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829264.117240:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228376448:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829264.117276:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228376576:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829264.117303:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829264.117312:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829264/real 1787829264] req@ffff9856085a2580 x1874669228376576/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829289 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829264.117326:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2580 x1874669228376576/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829289 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829264.117331:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829264.117334:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829264.117336:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228376576:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829264.117340:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228376704:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829264.117352:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829264.117367:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829264/real 1787829264] req@ffff9856085a3840 x1874669228376704/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829289 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829264.117375:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3840 x1874669228376704/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829289 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829264.117379:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829264.117381:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829264.117383:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228376704:192.168.203.114@tcp:38:kworker.0 00010000:00010000:3.0:1787829264.180858:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829265.076791:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829265.204818:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829266.100831:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829266.228830:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829267.124783:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829267.252875:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829268.148893:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829268.276796:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829269.172772:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787829269.236789:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829269.236798:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829269.236801:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829269.236812:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5414 00000100:00080000:1.0:1787829269.236821:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829269.236829:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787829269.236866:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a03c0 x1874669228376832/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829269.236896:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829269.236900:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829269.236902:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829269.236908:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5414 00000100:00080000:1.0:1787829269.236913:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829269.236918:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787829269.236927:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829269.236931:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228376832:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787829269.236943:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0780 x1874669228376960/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829269.236954:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829269.236956:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829269.236958:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829269.236964:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5414 00000100:00080000:1.0:1787829269.236966:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829269.236969:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787829269.236972:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829269.236980:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a1680 x1874669228377088/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829269.236991:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829269.237026:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829269/real 1787829269] req@ffff9856085a03c0 x1874669228376832/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829294 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829269.237041:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a03c0 x1874669228376832/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829294 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829269.237049:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829269.237051:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829269.237054:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228376832:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787829269.237162:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228376960:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829269.237338:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829269.237355:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829269/real 1787829269] req@ffff9856085a0780 x1874669228376960/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829294 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829269.237421:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0780 x1874669228376960/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829294 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829269.237431:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829269.237435:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829269.237439:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228376960:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787829269.237451:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228377088:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829269.237491:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829269.237521:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829269/real 1787829269] req@ffff9856085a1680 x1874669228377088/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829294 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829269.237535:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a1680 x1874669228377088/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829294 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829269.237542:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829269.237547:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829269.237551:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228377088:192.168.203.114@tcp:38:kworker.0 00010000:00010000:3.0:1787829269.300838:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829270.196777:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829270.324849:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829271.220783:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829271.348783:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829272.244789:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829272.372830:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829273.268775:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829273.396817:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829274.292801:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787829274.356804:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829274.356812:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829274.356815:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829274.356823:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5419 00000100:00080000:3.0:1787829274.356832:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829274.356838:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787829274.356867:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81a40 x1874669228377216/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829274.356890:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829274.356892:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829274.356894:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829274.356896:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5419 00000100:00080000:3.0:1787829274.356899:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829274.356902:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829274.356909:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829274.356918:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81680 x1874669228377344/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787829274.356924:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228377216:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787829274.356930:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829274.356933:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829274.356934:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829274.356937:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5419 00000100:00080000:3.0:1787829274.356939:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829274.356941:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829274.356945:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829274.356953:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f812c0 x1874669228377472/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787829274.357011:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228377344:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829274.357036:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228377472:192.168.203.114@tcp:38:kworker.0 00010000:00010000:3.0:1787829274.420846:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829275.316778:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829275.444869:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829276.340803:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829276.468839:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829277.364756:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000400:00000100:1.0:1787829277.428917:0:2370:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.114:988 00000400:00000100:1.0:1787829277.428965:0:2370:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.114@tcp at host 192.168.203.114 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1787829277.428981:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:1.0:1787829277.429019:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:1.0:1787829277.429028:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000100:00100000:0.0:1787829277.429075:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829274/real 1787829277] req@ffff985610f81680 x1874669228377344/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829299 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829277.429113:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81680 x1874669228377344/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829299 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829277.429126:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829277.429132:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829277.429137:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228377344:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829277.429144:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829274/real 1787829277] req@ffff985610f812c0 x1874669228377472/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829299 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829277.429173:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f812c0 x1874669228377472/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829299 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829277.429182:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829277.429186:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829277.429190:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228377472:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829277.429197:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829274/real 1787829277] req@ffff985610f81a40 x1874669228377216/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829299 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829277.429209:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81a40 x1874669228377216/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829299 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829277.429217:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829277.429220:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829277.429224:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228377216:192.168.203.114@tcp:250:kworker.0 00010000:00010000:1.0:1787829277.492956:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829278.388769:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829278.516779:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829279.412766:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787829279.476808:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829279.476819:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829279.476823:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829279.476834:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5425 00000100:00080000:0.0:1787829279.476845:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829279.476852:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787829279.476890:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6d00 x1874669228377600/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829279.476921:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829279.476926:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829279.476929:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829279.476933:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5425 00000100:00080000:0.0:1787829279.476939:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829279.476944:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787829279.476955:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829279.476973:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228377600:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829279.476981:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5e00 x1874669228377728/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829279.476997:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829279.477001:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829279.477003:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829279.477009:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5425 00000100:00080000:0.0:1787829279.477014:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829279.477019:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787829279.477025:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829279.477041:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5680 x1874669228377856/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829279.477050:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829279.477095:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829279/real 1787829279] req@ffff985607df6d00 x1874669228377600/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829304 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829279.477117:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6d00 x1874669228377600/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829304 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829279.477130:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829279.477134:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829279.477139:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228377600:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787829279.477188:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228377728:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829279.477219:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829279.477234:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829279/real 1787829279] req@ffff985607df5e00 x1874669228377728/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829304 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829279.477248:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5e00 x1874669228377728/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829304 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829279.477257:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829279.477261:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829279.477265:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228377728:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787829279.477271:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228377856:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829279.477293:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829279.477305:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829279/real 1787829279] req@ffff985607df5680 x1874669228377856/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829304 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829279.477317:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5680 x1874669228377856/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829304 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829279.477325:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829279.477329:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829279.477332:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228377856:192.168.203.114@tcp:38:kworker.0 00010000:00010000:3.0:1787829279.540849:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829280.436811:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829280.564839:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829281.460755:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829281.588813:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829282.484790:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829282.612831:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829283.508763:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829283.636844:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829284.532801:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787829284.596805:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829284.596813:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829284.596816:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829284.596834:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5430 00000100:00080000:2.0:1787829284.596845:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829284.596853:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787829284.596891:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8780 x1874669228377984/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829284.596917:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829284.596921:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829284.596923:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829284.596927:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5430 00000100:00080000:2.0:1787829284.596932:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829284.596936:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787829284.596947:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829284.596963:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228377984:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787829284.596964:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db480 x1874669228378112/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829284.596978:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829284.596981:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829284.596982:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829284.596986:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5430 00000100:00080000:2.0:1787829284.596989:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829284.596992:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787829284.596997:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829284.597005:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d9a40 x1874669228378240/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829284.597021:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829284.597055:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829284/real 1787829284] req@ffff9856140d8780 x1874669228377984/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829309 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829284.597069:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8780 x1874669228377984/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829309 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829284.597077:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829284.597080:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829284.597083:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228377984:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787829284.597101:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228378112:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829284.597132:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829284.597143:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829284/real 1787829284] req@ffff9856140db480 x1874669228378112/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829309 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829284.597154:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db480 x1874669228378112/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829309 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829284.597163:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829284.597166:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829284.597170:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228378112:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787829284.597176:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228378240:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829284.597209:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829284.597223:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829284/real 1787829284] req@ffff9856140d9a40 x1874669228378240/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829309 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829284.597237:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d9a40 x1874669228378240/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829309 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829284.597253:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829284.597256:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829284.597260:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228378240:192.168.203.114@tcp:38:kworker.0 00010000:00010000:3.0:1787829284.660850:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829285.556834:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829285.684859:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829286.580805:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829286.708772:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829287.604807:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829287.732863:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829288.628776:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829288.756787:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829289.652824:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787829289.716826:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829289.716837:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829289.716842:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829289.716853:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5435 00000100:00080000:0.0:1787829289.716866:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829289.716876:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787829289.716927:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df61c0 x1874669228378368/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829289.716956:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829289.716960:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829289.716961:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829289.716964:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5435 00000100:00080000:0.0:1787829289.716968:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829289.716971:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787829289.716981:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829289.716991:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df43c0 x1874669228378496/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829289.717000:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829289.717004:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:3.0:1787829289.717004:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228378368:192.168.203.114@tcp:250:kworker.0 00000100:00080000:0.0:1787829289.717006:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829289.717014:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5435 00000100:00080000:0.0:1787829289.717017:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829289.717019:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787829289.717022:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829289.717031:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df70c0 x1874669228378624/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829289.717094:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829289.717134:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829289/real 1787829289] req@ffff985607df61c0 x1874669228378368/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829314 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829289.717160:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df61c0 x1874669228378368/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829314 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829289.717170:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829289.717174:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829289.717177:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228378368:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787829289.717199:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228378496:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829289.717218:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829289.717227:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829289/real 1787829289] req@ffff985607df43c0 x1874669228378496/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829314 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829289.717236:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df43c0 x1874669228378496/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829314 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829289.717241:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829289.717244:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829289.717247:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228378496:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787829289.717251:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228378624:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829289.717262:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829289.717271:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829289/real 1787829289] req@ffff985607df70c0 x1874669228378624/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829314 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829289.717283:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df70c0 x1874669228378624/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829314 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829289.717291:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829289.717295:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829289.717299:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228378624:192.168.203.114@tcp:38:kworker.0 00010000:00010000:3.0:1787829289.780885:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829290.676772:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829290.804914:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829291.700809:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829291.828867:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829292.724752:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829292.852877:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829293.748803:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829293.876797:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829294.772756:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787829294.836882:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829294.836892:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829294.836897:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829294.836907:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5440 00000100:00080000:2.0:1787829294.836930:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829294.836938:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787829294.836978:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d83c0 x1874669228378752/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829294.837017:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829294.837021:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829294.837024:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829294.837028:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5440 00000100:00080000:2.0:1787829294.837032:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829294.837056:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787829294.837086:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829294.837106:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d9680 x1874669228378880/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829294.837122:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829294.837130:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:3.0:1787829294.837131:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228378752:192.168.203.114@tcp:250:kworker.0 00000100:00080000:2.0:1787829294.837162:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829294.837172:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5440 00000100:00080000:2.0:1787829294.837184:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829294.837191:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787829294.837200:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829294.837218:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8b40 x1874669228379008/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829294.837240:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829294.837311:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829294/real 1787829294] req@ffff9856140d83c0 x1874669228378752/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829319 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829294.837331:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d83c0 x1874669228378752/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829319 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829294.837343:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829294.837347:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829294.837352:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228378752:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787829294.837377:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228378880:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829294.837399:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829294.837412:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829294/real 1787829294] req@ffff9856140d9680 x1874669228378880/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829319 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829294.837424:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d9680 x1874669228378880/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829319 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829294.837432:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829294.837435:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829294.837438:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228378880:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787829294.837444:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228379008:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829294.837459:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829294.837468:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829294/real 1787829294] req@ffff9856140d8b40 x1874669228379008/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829319 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829294.837479:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8b40 x1874669228379008/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829319 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829294.837487:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829294.837490:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829294.837493:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228379008:192.168.203.114@tcp:38:kworker.0 00010000:00010000:3.0:1787829294.900986:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829295.796837:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829295.924846:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829296.820748:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829296.948851:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829297.844846:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829297.972826:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829298.868784:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829298.996825:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829299.892825:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787829299.956816:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829299.956827:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829299.956831:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829299.956841:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5445 00000100:00080000:0.0:1787829299.956850:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829299.956856:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787829299.956885:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6580 x1874669228379136/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829299.956906:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829299.956909:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829299.956910:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829299.956913:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5445 00000100:00080000:0.0:1787829299.956915:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829299.956917:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787829299.956924:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829299.956932:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5a40 x1874669228379264/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829299.956939:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829299.956941:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829299.956942:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829299.956945:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5445 00000100:00080000:0.0:1787829299.956946:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829299.956948:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787829299.956952:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829299.956963:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df70c0 x1874669228379392/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829299.956995:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228379136:192.168.203.114@tcp:250:kworker.0 00000800:00000100:3.0:1787829299.957108:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829299.957147:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829299/real 1787829299] req@ffff985607df6580 x1874669228379136/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829324 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829299.957163:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6580 x1874669228379136/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829324 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829299.957173:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829299.957177:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829299.957181:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228379136:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787829299.957187:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228379264:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829299.957214:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829299.957223:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829299/real 1787829299] req@ffff985607df5a40 x1874669228379264/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829324 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829299.957232:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5a40 x1874669228379264/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829324 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829299.957253:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829299.957256:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829299.957259:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228379264:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787829299.957263:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228379392:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829299.957277:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829299.957284:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829299/real 1787829299] req@ffff985607df70c0 x1874669228379392/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829324 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829299.957298:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df70c0 x1874669228379392/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829324 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829299.957304:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829299.957306:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829299.957314:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228379392:192.168.203.114@tcp:38:kworker.0 00010000:00010000:3.0:1787829300.020924:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829300.916774:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829301.044917:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829301.940853:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829302.068780:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829302.964799:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829303.092865:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829303.988794:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829304.116860:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829305.012829:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787829305.076776:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829305.076785:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829305.076788:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829305.076797:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5450 00000100:00080000:2.0:1787829305.076807:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829305.076812:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787829305.076850:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140da940 x1874669228379520/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829305.076881:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829305.076884:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829305.076887:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829305.076892:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5450 00000100:00080000:2.0:1787829305.076897:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829305.076901:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787829305.076911:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829305.076918:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228379520:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787829305.076926:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db0c0 x1874669228379648/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829305.076940:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829305.076942:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829305.076945:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829305.076953:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5450 00000100:00080000:2.0:1787829305.076957:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829305.076961:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787829305.076979:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829305.076998:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140dad00 x1874669228379776/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787829305.077029:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829305.077080:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829305/real 1787829305] req@ffff9856140da940 x1874669228379520/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829330 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829305.077102:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140da940 x1874669228379520/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829330 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829305.077111:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829305.077114:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829305.077118:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228379520:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829305.077146:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228379648:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829305.077169:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829305.077184:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829305/real 1787829305] req@ffff9856140db0c0 x1874669228379648/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829330 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829305.077198:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db0c0 x1874669228379648/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829330 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829305.077207:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829305.077211:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829305.077214:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228379648:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829305.077219:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228379776:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829305.077237:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829305.077245:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829305/real 1787829305] req@ffff9856140dad00 x1874669228379776/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829330 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829305.077254:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140dad00 x1874669228379776/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829330 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829305.077259:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829305.077262:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829305.077264:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228379776:192.168.203.114@tcp:38:kworker.0 00010000:00010000:3.0:1787829305.140928:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829306.036763:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829306.164923:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829307.060830:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829307.188898:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829308.084763:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829308.212881:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829309.108804:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829309.236843:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829310.132826:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787829310.196848:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829310.196857:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829310.196862:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829310.196870:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5455 00000100:00080000:3.0:1787829310.196880:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829310.196887:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787829310.196918:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80b40 x1874669228379904/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829310.196952:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829310.196954:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829310.196956:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829310.196959:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5455 00000100:00080000:3.0:1787829310.196962:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829310.196966:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829310.196974:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829310.196985:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80000 x1874669228380032/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787829310.196998:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228379904:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787829310.197001:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829310.197016:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829310.197018:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829310.197022:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5455 00000100:00080000:3.0:1787829310.197025:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829310.197027:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829310.197031:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829310.197042:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f830c0 x1874669228380160/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787829310.197098:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829310.197139:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829310/real 1787829310] req@ffff985610f80b40 x1874669228379904/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829335 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829310.197155:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80b40 x1874669228379904/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829335 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829310.197164:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829310.197168:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829310.197171:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228379904:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829310.197195:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228380032:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829310.197212:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829310.197220:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829310/real 1787829310] req@ffff985610f80000 x1874669228380032/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829335 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829310.197228:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80000 x1874669228380032/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829335 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829310.197235:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829310.197237:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829310.197239:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228380032:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829310.197243:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228380160:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829310.197254:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829310.197260:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829310/real 1787829310] req@ffff985610f830c0 x1874669228380160/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829335 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829310.197276:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f830c0 x1874669228380160/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829335 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829310.197281:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829310.197284:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829310.197286:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228380160:192.168.203.114@tcp:38:kworker.0 00010000:00010000:3.0:1787829310.260883:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829311.156782:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829311.284821:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829312.180781:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829312.308815:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829313.204825:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829313.332917:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829314.228756:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829314.356824:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829315.252777:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787829315.316836:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829315.316846:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829315.316851:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829315.316863:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5460 00000100:00080000:1.0:1787829315.316874:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829315.316881:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787829315.316947:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3c00 x1874669228380288/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829315.316979:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829315.316984:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829315.316986:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829315.316990:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5460 00000100:00080000:1.0:1787829315.316995:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829315.317000:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787829315.317010:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829315.317022:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0b40 x1874669228380416/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829315.317030:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829315.317033:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829315.317035:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829315.317038:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5460 00000100:00080000:1.0:1787829315.317041:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829315.317044:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787829315.317048:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829315.317055:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0000 x1874669228380544/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787829315.317064:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228380288:192.168.203.114@tcp:250:kworker.0 00000800:00000100:0.0:1787829315.317139:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829315.317174:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829315/real 1787829315] req@ffff9856085a3c00 x1874669228380288/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829340 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829315.317190:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3c00 x1874669228380288/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829340 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829315.317199:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829315.317202:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829315.317206:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228380288:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829315.317210:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228380416:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829315.317225:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829315.317233:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829315/real 1787829315] req@ffff9856085a0b40 x1874669228380416/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829340 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829315.317241:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0b40 x1874669228380416/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829340 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829315.317246:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829315.317249:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829315.317251:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228380416:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829315.317271:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228380544:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829315.317282:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829315.317288:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829315/real 1787829315] req@ffff9856085a0000 x1874669228380544/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829340 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829315.317295:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0000 x1874669228380544/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829340 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829315.317300:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829315.317302:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829315.317304:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228380544:192.168.203.114@tcp:38:kworker.0 00010000:00010000:3.0:1787829315.380906:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829316.276789:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829316.404950:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829317.300809:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829317.428803:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829318.324761:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829318.452840:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829319.348812:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829319.476803:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829320.372779:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787829320.436779:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829320.436787:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829320.436791:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829320.436801:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5466 00000100:00080000:3.0:1787829320.436812:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829320.436820:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787829320.436876:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80780 x1874669228380672/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829320.436906:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829320.436909:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829320.436911:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829320.436914:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5466 00000100:00080000:3.0:1787829320.436917:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829320.436920:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829320.436929:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829320.436940:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83c00 x1874669228380800/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829320.436948:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829320.436951:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829320.436952:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829320.436955:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5466 00000100:00080000:3.0:1787829320.436957:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829320.436960:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787829320.436966:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228380672:192.168.203.114@tcp:250:kworker.0 00000008:00100000:3.0:1787829320.436966:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829320.436980:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f830c0 x1874669228380928/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787829320.437042:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829320.437076:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829320/real 1787829320] req@ffff985610f80780 x1874669228380672/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829345 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829320.437090:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80780 x1874669228380672/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829345 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829320.437099:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829320.437102:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829320.437105:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228380672:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829320.437110:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228380800:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829320.437125:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829320.437133:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829320/real 1787829320] req@ffff985610f83c00 x1874669228380800/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829345 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829320.437141:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83c00 x1874669228380800/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829345 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829320.437146:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829320.437148:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829320.437150:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228380800:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829320.437173:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228380928:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829320.437187:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829320.437193:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829320/real 1787829320] req@ffff985610f830c0 x1874669228380928/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829345 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829320.437200:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f830c0 x1874669228380928/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829345 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829320.437206:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829320.437207:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829320.437209:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228380928:192.168.203.114@tcp:38:kworker.0 00010000:00010000:3.0:1787829320.500839:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829321.396821:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829321.524918:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829322.420835:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829322.548902:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829323.444829:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829323.572868:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829324.468812:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829324.596824:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829325.492772:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787829325.556778:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829325.556785:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829325.556789:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829325.556796:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5471 00000100:00080000:3.0:1787829325.556805:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829325.556811:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787829325.556855:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80000 x1874669228381056/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829325.556876:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829325.556879:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829325.556880:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829325.556882:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5471 00000100:00080000:3.0:1787829325.556884:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829325.556887:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829325.556895:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829325.556903:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80b40 x1874669228381184/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829325.556912:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829325.556915:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829325.556916:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829325.556919:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5471 00000100:00080000:3.0:1787829325.556920:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829325.556923:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829325.556927:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829325.556937:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81a40 x1874669228381312/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787829325.556938:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228381056:192.168.203.114@tcp:250:kworker.0 00000800:00000100:0.0:1787829325.557011:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829325.557044:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829325/real 1787829325] req@ffff985610f80000 x1874669228381056/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829350 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829325.557063:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80000 x1874669228381056/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829350 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829325.557074:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829325.557078:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829325.557082:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228381056:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829325.557090:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228381184:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829325.557111:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829325.557122:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829325/real 1787829325] req@ffff985610f80b40 x1874669228381184/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829350 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829325.557144:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80b40 x1874669228381184/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829350 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829325.557154:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829325.557157:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829325.557161:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228381184:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829325.557185:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228381312:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829325.557204:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829325.557221:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829325/real 1787829325] req@ffff985610f81a40 x1874669228381312/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829350 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829325.557233:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81a40 x1874669228381312/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829350 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829325.557242:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829325.557245:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829325.557249:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228381312:192.168.203.114@tcp:38:kworker.0 00010000:00010000:3.0:1787829325.620860:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829326.516829:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829326.644892:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829327.540803:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829327.668882:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829328.564787:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829328.692836:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829329.588752:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829329.716821:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829330.612780:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787829330.676883:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829330.676891:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829330.676894:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829330.676901:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5476 00000100:00080000:3.0:1787829330.676922:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829330.676929:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787829330.676972:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f812c0 x1874669228381440/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829330.676993:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829330.676996:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829330.676997:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829330.677000:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5476 00000100:00080000:3.0:1787829330.677002:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829330.677005:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829330.677011:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829330.677021:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81680 x1874669228381568/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829330.677028:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829330.677031:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829330.677032:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829330.677035:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5476 00000100:00100000:0.0:1787829330.677042:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228381440:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787829330.677043:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829330.677059:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829330.677083:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829330.677094:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83840 x1874669228381696/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787829330.677100:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829330.677133:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829330/real 1787829330] req@ffff985610f812c0 x1874669228381440/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829355 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829330.677146:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f812c0 x1874669228381440/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829355 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829330.677157:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829330.677161:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829330.677166:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228381440:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829330.677174:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228381568:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829330.677199:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829330.677211:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829330/real 1787829330] req@ffff985610f81680 x1874669228381568/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829355 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829330.677222:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81680 x1874669228381568/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829355 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829330.677228:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829330.677230:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829330.677232:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228381568:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829330.677250:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228381696:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829330.677264:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829330.677271:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829330/real 1787829330] req@ffff985610f83840 x1874669228381696/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829355 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829330.677278:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83840 x1874669228381696/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829355 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829330.677283:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829330.677285:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829330.677287:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228381696:192.168.203.114@tcp:38:kworker.0 00010000:00010000:3.0:1787829330.740862:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829331.636794:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829331.764865:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829332.660804:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829332.788860:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829333.684788:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829333.812894:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829334.708847:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829334.836822:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829335.732787:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787829335.796811:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829335.796821:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829335.796825:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829335.796836:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5481 00000100:00080000:1.0:1787829335.796848:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829335.796858:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787829335.796907:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2d00 x1874669228381824/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829335.796934:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829335.796937:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829335.796939:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829335.796942:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5481 00000100:00080000:1.0:1787829335.796945:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829335.796948:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787829335.796958:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829335.796971:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0000 x1874669228381952/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787829335.796980:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228381824:192.168.203.114@tcp:250:kworker.0 00000100:00080000:1.0:1787829335.796982:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829335.796991:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829335.796992:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829335.796996:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5481 00000100:00080000:1.0:1787829335.796998:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829335.797000:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787829335.797003:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829335.797013:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0b40 x1874669228382080/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787829335.797064:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829335.797105:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829335/real 1787829335] req@ffff9856085a2d00 x1874669228381824/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829360 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829335.797120:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2d00 x1874669228381824/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829360 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829335.797129:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829335.797132:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829335.797136:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228381824:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829335.797160:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228381952:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829335.797177:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829335.797186:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829335/real 1787829335] req@ffff9856085a0000 x1874669228381952/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829360 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829335.797193:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0000 x1874669228381952/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829360 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829335.797199:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829335.797202:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829335.797204:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228381952:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829335.797214:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228382080:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829335.797228:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829335.797235:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829335/real 1787829335] req@ffff9856085a0b40 x1874669228382080/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829360 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829335.797242:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0b40 x1874669228382080/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829360 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829335.797248:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829335.797250:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829335.797252:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228382080:192.168.203.114@tcp:38:kworker.0 00010000:00010000:3.0:1787829335.860904:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829336.756815:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829336.884868:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829337.780808:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829337.908900:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829338.804847:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829338.932885:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829339.828783:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829339.956864:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829340.852845:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787829340.916820:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829340.916834:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829340.916840:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829340.916853:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5486 00000100:00080000:1.0:1787829340.916867:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829340.916879:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787829340.916932:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3c00 x1874669228382208/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829340.916966:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829340.916971:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829340.916974:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829340.916979:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5486 00000100:00080000:1.0:1787829340.916985:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829340.916991:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787829340.917000:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228382208:192.168.203.114@tcp:250:kworker.0 00000008:00100000:1.0:1787829340.917005:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829340.917026:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a1680 x1874669228382336/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829340.917040:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829340.917044:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829340.917047:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829340.917052:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5486 00000100:00080000:1.0:1787829340.917056:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829340.917062:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787829340.917068:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829340.917085:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0780 x1874669228382464/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829340.917187:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228382336:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787829340.917237:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228382464:192.168.203.114@tcp:38:kworker.0 00010000:00010000:3.0:1787829340.980848:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829341.876855:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829342.004948:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829342.900811:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829343.028825:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829343.924833:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000400:00000100:3.0:1787829343.989058:0:2371:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.114:988 00000400:00000100:3.0:1787829343.989121:0:2371:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.114@tcp at host 192.168.203.114 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:3.0:1787829343.989145:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:3.0:1787829343.989190:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:3.0:1787829343.989202:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000100:00100000:0.0:1787829343.989231:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829340/real 1787829343] req@ffff9856085a1680 x1874669228382336/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829365 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829343.989263:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a1680 x1874669228382336/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829365 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829343.989277:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829343.989284:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829343.989289:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228382336:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829343.989298:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829340/real 1787829343] req@ffff9856085a0780 x1874669228382464/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829365 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829343.989312:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0780 x1874669228382464/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829365 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829343.989330:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829343.989335:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829343.989340:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228382464:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829343.989348:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829340/real 1787829343] req@ffff9856085a3c00 x1874669228382208/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829365 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829343.989362:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3c00 x1874669228382208/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829365 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829343.989370:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829343.989374:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829343.989377:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228382208:192.168.203.114@tcp:250:kworker.0 00010000:00010000:1.0:1787829344.052863:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829344.948830:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829345.076883:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829345.972850:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787829346.036820:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829346.036828:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829346.036832:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829346.036840:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5491 00000100:00080000:0.0:1787829346.036848:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829346.036854:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787829346.036881:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df43c0 x1874669228382592/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829346.036903:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829346.036905:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829346.036907:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829346.036910:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5491 00000100:00080000:0.0:1787829346.036912:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829346.036915:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787829346.036922:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829346.036931:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df61c0 x1874669228382720/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829346.036938:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829346.036940:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829346.036942:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829346.036945:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5491 00000100:00080000:0.0:1787829346.036948:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829346.036952:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787829346.036957:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829346.036967:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5680 x1874669228382848/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829346.036995:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228382592:192.168.203.114@tcp:250:kworker.0 00000800:00000100:3.0:1787829346.037090:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829346.037139:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829346/real 1787829346] req@ffff985607df43c0 x1874669228382592/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829371 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829346.037164:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df43c0 x1874669228382592/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829371 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829346.037177:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829346.037182:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829346.037187:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228382592:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787829346.037196:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228382720:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829346.037223:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829346.037236:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829346/real 1787829346] req@ffff985607df61c0 x1874669228382720/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829371 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829346.037250:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df61c0 x1874669228382720/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829371 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829346.037260:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829346.037263:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829346.037267:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228382720:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787829346.037302:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228382848:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829346.037325:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829346.037337:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829346/real 1787829346] req@ffff985607df5680 x1874669228382848/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829371 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829346.037350:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5680 x1874669228382848/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829371 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829346.037359:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829346.037363:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829346.037367:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228382848:192.168.203.114@tcp:38:kworker.0 00010000:00010000:3.0:1787829346.100862:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829346.996765:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829347.124900:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829348.020846:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829348.148839:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829349.044810:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829349.172875:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829350.068809:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829350.196800:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829351.092779:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787829351.156789:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829351.156797:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829351.156800:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829351.156809:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5496 00000100:00080000:0.0:1787829351.156817:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829351.156822:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787829351.156851:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5e00 x1874669228382976/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829351.156875:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829351.156877:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829351.156879:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829351.156881:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5496 00000100:00080000:0.0:1787829351.156883:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829351.156886:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787829351.156893:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829351.156902:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6d00 x1874669228383104/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829351.156909:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829351.156911:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829351.156913:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829351.156916:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5496 00000100:00080000:0.0:1787829351.156918:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829351.156920:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787829351.156924:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829351.156930:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228382976:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829351.156932:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4f00 x1874669228383232/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829351.156990:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829351.157019:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829351/real 1787829351] req@ffff985607df5e00 x1874669228382976/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829376 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829351.157033:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5e00 x1874669228382976/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829376 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829351.157041:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829351.157044:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829351.157047:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228382976:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787829351.157052:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228383104:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829351.157066:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829351.157077:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829351/real 1787829351] req@ffff985607df6d00 x1874669228383104/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829376 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829351.157090:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6d00 x1874669228383104/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829376 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829351.157098:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829351.157102:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829351.157105:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228383104:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787829351.157129:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228383232:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829351.157146:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829351.157162:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829351/real 1787829351] req@ffff985607df4f00 x1874669228383232/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829376 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829351.157170:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4f00 x1874669228383232/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829376 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829351.157176:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829351.157178:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829351.157180:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228383232:192.168.203.114@tcp:38:kworker.0 00010000:00010000:3.0:1787829351.220879:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829352.116829:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829352.244896:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829353.140751:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829353.268865:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829354.164789:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829354.292807:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829355.188745:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829355.316852:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829356.212853:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787829356.276835:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829356.276846:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829356.276850:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829356.276858:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5501 00000100:00080000:1.0:1787829356.276867:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829356.276873:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787829356.276911:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a03c0 x1874669228383360/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829356.276938:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829356.276941:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829356.276943:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829356.276946:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5501 00000100:00080000:1.0:1787829356.276951:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829356.276955:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787829356.276965:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829356.276982:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3840 x1874669228383488/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829356.276996:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829356.276999:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829356.277001:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829356.277008:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5501 00000100:00100000:3.0:1787829356.277011:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228383360:192.168.203.114@tcp:250:kworker.0 00000100:00080000:1.0:1787829356.277012:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829356.277020:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787829356.277032:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829356.277041:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2580 x1874669228383616/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829356.277114:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829356.277174:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829356/real 1787829356] req@ffff9856085a03c0 x1874669228383360/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829381 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829356.277193:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a03c0 x1874669228383360/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829381 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829356.277203:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829356.277207:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829356.277210:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228383360:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787829356.277234:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228383488:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829356.277255:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829356.277264:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829356/real 1787829356] req@ffff9856085a3840 x1874669228383488/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829381 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829356.277273:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3840 x1874669228383488/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829381 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829356.277279:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829356.277281:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829356.277283:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228383488:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787829356.277287:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228383616:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829356.277310:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829356.277317:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829356/real 1787829356] req@ffff9856085a2580 x1874669228383616/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829381 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829356.277328:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2580 x1874669228383616/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829381 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829356.277336:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829356.277340:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829356.277343:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228383616:192.168.203.114@tcp:38:kworker.0 00010000:00010000:3.0:1787829356.340885:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829357.236771:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829357.364930:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829358.260799:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829358.388804:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829359.284790:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829359.412893:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829360.308865:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829360.436844:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829361.332820:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787829361.396867:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829361.396878:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829361.396883:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829361.396895:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5507 00000100:00080000:3.0:1787829361.396907:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829361.396920:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787829361.396965:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f803c0 x1874669228383744/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829361.396993:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829361.396995:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829361.396997:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829361.397000:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5507 00000100:00080000:3.0:1787829361.397003:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829361.397006:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829361.397016:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829361.397026:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81e00 x1874669228383872/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829361.397034:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829361.397037:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829361.397039:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829361.397043:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5507 00000100:00100000:0.0:1787829361.397047:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228383744:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787829361.397047:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829361.397053:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829361.397058:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829361.397065:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82d00 x1874669228384000/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787829361.397125:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829361.397160:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829361/real 1787829361] req@ffff985610f803c0 x1874669228383744/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829386 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829361.397176:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f803c0 x1874669228383744/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829386 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829361.397185:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829361.397188:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829361.397192:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228383744:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829361.397213:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228383872:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829361.397230:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829361.397239:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829361/real 1787829361] req@ffff985610f81e00 x1874669228383872/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829386 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829361.397247:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81e00 x1874669228383872/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829386 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829361.397253:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829361.397256:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829361.397258:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228383872:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829361.397262:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228384000:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829361.397272:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829361.397278:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829361/real 1787829361] req@ffff985610f82d00 x1874669228384000/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829386 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829361.397290:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82d00 x1874669228384000/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829386 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829361.397296:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829361.397297:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829361.397299:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228384000:192.168.203.114@tcp:38:kworker.0 00010000:00010000:3.0:1787829361.460940:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829362.356800:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829362.484865:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829363.380800:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829363.508874:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829364.404782:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787829364.532893:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829365.428798:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829365.556813:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829366.452789:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787829366.516744:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829366.516752:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829366.516755:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829366.516763:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5512 00000100:00080000:3.0:1787829366.516771:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829366.516776:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787829366.516812:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80f00 x1874669228384128/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829366.516832:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829366.516835:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829366.516836:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829366.516839:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5512 00000100:00080000:3.0:1787829366.516841:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829366.516844:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829366.516851:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829366.516859:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82940 x1874669228384256/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829366.516866:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829366.516869:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829366.516870:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829366.516873:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5512 00000100:00080000:3.0:1787829366.516875:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829366.516877:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829366.516880:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829366.516884:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228384128:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787829366.516893:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82580 x1874669228384384/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787829366.516959:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829366.516992:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829366/real 1787829366] req@ffff985610f80f00 x1874669228384128/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829391 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829366.517010:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80f00 x1874669228384128/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829391 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829366.517022:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829366.517027:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829366.517031:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228384128:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829366.517039:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228384256:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829366.517064:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829366.517077:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829366/real 1787829366] req@ffff985610f82940 x1874669228384256/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829391 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829366.517090:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82940 x1874669228384256/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829391 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829366.517100:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829366.517103:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829366.517106:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228384256:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829366.517131:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228384384:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829366.517154:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829366.517166:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829366/real 1787829366] req@ffff985610f82580 x1874669228384384/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829391 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829366.517179:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82580 x1874669228384384/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829391 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829366.517188:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829366.517191:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829366.517195:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228384384:192.168.203.114@tcp:38:kworker.0 00010000:00010000:3.0:1787829366.580900:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829367.476759:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829367.604814:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829368.500768:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829368.628816:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829369.524799:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787829369.652845:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829370.548767:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829370.676824:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829371.572780:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787829371.636789:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829371.636796:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829371.636799:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829371.636807:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5517 00000100:00080000:2.0:1787829371.636816:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829371.636823:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787829371.636851:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8f00 x1874669228384512/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829371.636872:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829371.636875:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829371.636876:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829371.636879:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5517 00000100:00080000:2.0:1787829371.636881:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829371.636883:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787829371.636890:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829371.636899:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db840 x1874669228384640/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829371.636906:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829371.636908:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829371.636910:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829371.636912:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5517 00000100:00080000:2.0:1787829371.636915:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00100000:0.0:1787829371.636919:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228384512:192.168.203.114@tcp:250:kworker.0 00000100:00080000:2.0:1787829371.636920:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787829371.636927:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829371.636937:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140dbc00 x1874669228384768/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787829371.636976:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829371.637001:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829371/real 1787829371] req@ffff9856140d8f00 x1874669228384512/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829396 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829371.637014:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8f00 x1874669228384512/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829396 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829371.637032:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829371.637036:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829371.637039:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228384512:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829371.637045:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228384640:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829371.637060:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829371.637069:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829371/real 1787829371] req@ffff9856140db840 x1874669228384640/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829396 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829371.637077:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db840 x1874669228384640/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829396 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829371.637082:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829371.637084:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829371.637087:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228384640:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829371.637106:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228384768:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829371.637117:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829371.637123:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829371/real 1787829371] req@ffff9856140dbc00 x1874669228384768/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829396 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829371.637131:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140dbc00 x1874669228384768/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829396 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829371.637139:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829371.637142:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829371.637146:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228384768:192.168.203.114@tcp:38:kworker.0 00010000:00010000:3.0:1787829371.700837:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829372.596752:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829372.724875:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829373.620798:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829373.748797:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829374.644764:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787829374.772869:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829375.668814:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829375.796810:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829376.692754:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787829376.756783:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829376.756792:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829376.756796:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829376.756804:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5522 00000100:00080000:0.0:1787829376.756813:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829376.756819:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787829376.756856:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7840 x1874669228384896/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829376.756887:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829376.756891:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829376.756893:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829376.756899:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5522 00000100:00080000:0.0:1787829376.756904:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829376.756909:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787829376.756919:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829376.756936:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df52c0 x1874669228385024/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829376.756948:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829376.756952:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829376.756955:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829376.756960:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5522 00000100:00080000:0.0:1787829376.756965:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787829376.756971:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228384896:192.168.203.114@tcp:250:kworker.0 00000100:00080000:0.0:1787829376.756975:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787829376.756989:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829376.757010:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7480 x1874669228385152/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829376.757058:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829376.757181:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829376/real 1787829376] req@ffff985607df7840 x1874669228384896/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829401 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829376.757353:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7840 x1874669228384896/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829401 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829376.757365:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829376.757369:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829376.757372:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228384896:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787829376.757381:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228385024:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829376.757407:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829376.757419:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829376/real 1787829376] req@ffff985607df52c0 x1874669228385024/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829401 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829376.757427:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df52c0 x1874669228385024/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829401 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829376.757433:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829376.757436:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829376.757438:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228385024:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787829376.757460:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228385152:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829376.757493:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829376.757502:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829376/real 1787829376] req@ffff985607df7480 x1874669228385152/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829401 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829376.757510:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7480 x1874669228385152/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829401 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829376.757520:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829376.757523:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829376.757525:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228385152:192.168.203.114@tcp:38:kworker.0 00010000:00010000:3.0:1787829376.820890:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829377.716782:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829377.844857:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829378.740764:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829378.868874:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829379.764833:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787829379.892862:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829380.788761:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829380.916838:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829381.812796:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787829381.876810:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829381.876819:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829381.876824:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829381.876834:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5527 00000100:00080000:0.0:1787829381.876844:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829381.876851:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787829381.876889:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4b40 x1874669228385280/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829381.876923:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829381.876927:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829381.876929:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829381.876934:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5527 00000100:00080000:0.0:1787829381.876938:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829381.876943:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787829381.876953:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829381.876969:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6940 x1874669228385408/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829381.876981:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228385280:192.168.203.114@tcp:250:kworker.0 00000100:00080000:0.0:1787829381.876985:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829381.876990:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829381.876993:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829381.876998:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5527 00000100:00080000:0.0:1787829381.877003:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829381.877007:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787829381.877012:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829381.877025:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4780 x1874669228385536/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829381.877049:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829381.877091:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829381/real 1787829381] req@ffff985607df4b40 x1874669228385280/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829406 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829381.877107:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4b40 x1874669228385280/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829406 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829381.877119:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829381.877124:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829381.877128:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228385280:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787829381.877165:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228385408:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829381.877192:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829381.877205:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829381/real 1787829381] req@ffff985607df6940 x1874669228385408/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829406 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829381.877215:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6940 x1874669228385408/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829406 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829381.877221:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829381.877224:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829381.877226:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228385408:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787829381.877231:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228385536:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829381.877245:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829381.877252:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829381/real 1787829381] req@ffff985607df4780 x1874669228385536/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829406 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829381.877260:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4780 x1874669228385536/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829406 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829381.877265:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829381.877267:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829381.877269:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228385536:192.168.203.114@tcp:38:kworker.0 00010000:00010000:3.0:1787829381.940807:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829382.836855:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829382.964903:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829383.860784:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829383.988905:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829384.884798:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787829385.012913:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829385.908805:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829386.036896:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829386.932804:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787829386.996841:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829386.996861:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829386.996866:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829386.996877:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5532 00000100:00080000:2.0:1787829386.996889:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829386.996897:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787829386.996943:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140da1c0 x1874669228385664/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829386.996968:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829386.996971:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829386.996973:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829386.996976:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5532 00000100:00080000:2.0:1787829386.996979:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829386.996982:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787829386.996992:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829386.997010:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140dbc00 x1874669228385792/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829386.997020:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829386.997025:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829386.997030:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1787829386.997033:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228385664:192.168.203.114@tcp:250:kworker.0 00000100:00080000:2.0:1787829386.997035:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5532 00000100:00080000:2.0:1787829386.997039:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829386.997048:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787829386.997057:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829386.997065:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db840 x1874669228385920/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829386.997121:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829386.997176:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829386/real 1787829386] req@ffff9856140da1c0 x1874669228385664/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829411 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829386.997192:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140da1c0 x1874669228385664/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829411 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829386.997202:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829386.997205:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829386.997208:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228385664:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787829386.997231:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228385792:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829386.997250:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829386.997259:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829386/real 1787829386] req@ffff9856140dbc00 x1874669228385792/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829411 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829386.997268:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140dbc00 x1874669228385792/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829411 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829386.997274:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829386.997276:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829386.997279:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228385792:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787829386.997285:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228385920:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829386.997305:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829386.997316:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829386/real 1787829386] req@ffff9856140db840 x1874669228385920/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829411 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829386.997325:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db840 x1874669228385920/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829411 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829386.997330:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829386.997333:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829386.997335:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228385920:192.168.203.114@tcp:38:kworker.0 00010000:00010000:3.0:1787829387.060902:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829387.956784:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829388.084927:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829388.980809:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829389.108793:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829390.004782:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787829390.132866:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829391.028814:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829391.156817:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829392.052771:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787829392.116815:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829392.116824:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829392.116829:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829392.116839:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5537 00000100:00080000:0.0:1787829392.116850:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829392.116858:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787829392.116910:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7c00 x1874669228386048/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829392.116939:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829392.116943:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829392.116945:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829392.116949:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5537 00000100:00080000:0.0:1787829392.116953:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829392.116957:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787829392.116968:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829392.116985:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228386048:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829392.116986:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4000 x1874669228386176/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829392.117007:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829392.117010:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829392.117012:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829392.117016:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5537 00000100:00080000:0.0:1787829392.117021:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829392.117025:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787829392.117031:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829392.117044:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4780 x1874669228386304/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829392.117050:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829392.117085:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829392/real 1787829392] req@ffff985607df7c00 x1874669228386048/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829417 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829392.117105:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7c00 x1874669228386048/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829417 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829392.117116:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829392.117120:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829392.117125:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228386048:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787829392.117156:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228386176:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829392.117175:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829392.117185:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829392/real 1787829392] req@ffff985607df4000 x1874669228386176/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829417 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829392.117193:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4000 x1874669228386176/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829417 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829392.117200:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829392.117202:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829392.117204:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228386176:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787829392.117208:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228386304:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829392.117219:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829392.117225:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829392/real 1787829392] req@ffff985607df4780 x1874669228386304/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829417 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829392.117237:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4780 x1874669228386304/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829417 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829392.117242:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829392.117244:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829392.117246:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228386304:192.168.203.114@tcp:38:kworker.0 00010000:00010000:3.0:1787829392.180876:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829393.076817:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829393.204921:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829394.100785:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829394.228826:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829395.124797:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787829395.252822:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829396.148805:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829396.276838:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829397.172830:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787829397.236807:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829397.236817:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829397.236822:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829397.236832:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5542 00000100:00080000:2.0:1787829397.236851:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829397.236858:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787829397.236905:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8f00 x1874669228386432/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829397.236938:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829397.236943:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829397.236945:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829397.236950:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5542 00000100:00080000:2.0:1787829397.236954:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829397.236959:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787829397.236971:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829397.236990:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140dad00 x1874669228386560/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829397.237000:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829397.237007:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:3.0:1787829397.237008:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228386432:192.168.203.114@tcp:250:kworker.0 00000100:00080000:2.0:1787829397.237010:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829397.237020:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5542 00000100:00080000:2.0:1787829397.237026:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829397.237030:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787829397.237036:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829397.237078:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db0c0 x1874669228386688/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829397.237092:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829397.237138:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829397/real 1787829397] req@ffff9856140d8f00 x1874669228386432/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829422 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829397.237160:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8f00 x1874669228386432/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829422 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829397.237171:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829397.237176:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829397.237181:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228386432:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787829397.237212:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228386560:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829397.237237:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829397.237250:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829397/real 1787829397] req@ffff9856140dad00 x1874669228386560/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829422 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829397.237262:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140dad00 x1874669228386560/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829422 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829397.237272:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829397.237275:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829397.237279:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228386560:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787829397.237285:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228386688:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829397.237303:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829397.237312:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829397/real 1787829397] req@ffff9856140db0c0 x1874669228386688/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829422 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829397.237340:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db0c0 x1874669228386688/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829422 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829397.237351:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829397.237355:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829397.237358:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228386688:192.168.203.114@tcp:38:kworker.0 00010000:00010000:3.0:1787829397.300896:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829398.196773:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829398.324886:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829399.220837:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829399.348851:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829400.244757:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829400.372951:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829401.268820:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829401.396842:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829402.292834:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787829402.356792:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829402.356805:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829402.356811:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829402.356823:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5547 00000100:00080000:2.0:1787829402.356838:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829402.356847:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787829402.356894:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140da940 x1874669228386816/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829402.356928:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829402.356932:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829402.356934:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829402.356938:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5547 00000100:00080000:2.0:1787829402.356942:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829402.356946:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787829402.356968:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829402.356982:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8b40 x1874669228386944/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829402.356993:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829402.356998:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829402.357000:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829402.357004:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5547 00000100:00080000:2.0:1787829402.357009:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829402.357016:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787829402.357017:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228386816:192.168.203.114@tcp:250:kworker.0 00000008:00100000:2.0:1787829402.357021:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829402.357033:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d9680 x1874669228387072/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829402.357094:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829402.357131:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829402/real 1787829402] req@ffff9856140da940 x1874669228386816/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829427 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829402.357148:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140da940 x1874669228386816/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829427 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829402.357156:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829402.357160:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829402.357163:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228386816:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787829402.357169:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228386944:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829402.357184:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829402.357192:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829402/real 1787829402] req@ffff9856140d8b40 x1874669228386944/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829427 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829402.357200:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8b40 x1874669228386944/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829427 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829402.357206:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829402.357207:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829402.357210:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228386944:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787829402.357235:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228387072:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829402.357246:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829402.357253:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829402/real 1787829402] req@ffff9856140d9680 x1874669228387072/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829427 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829402.357260:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d9680 x1874669228387072/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829427 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829402.357265:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829402.357267:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829402.357269:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228387072:192.168.203.114@tcp:38:kworker.0 00010000:00010000:3.0:1787829402.420913:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829403.316816:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829403.444851:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829404.340821:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829404.468865:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829405.364803:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829405.492902:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829406.388778:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829406.516887:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829407.412799:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787829407.476851:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829407.476865:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829407.476870:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829407.476884:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5553 00000100:00080000:2.0:1787829407.476899:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829407.476909:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787829407.476959:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d83c0 x1874669228387200/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829407.476993:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829407.476998:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829407.477000:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829407.477005:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5553 00000100:00080000:2.0:1787829407.477009:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829407.477014:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787829407.477027:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829407.477042:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d9a40 x1874669228387328/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829407.477054:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1787829407.477055:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228387200:192.168.203.114@tcp:250:kworker.0 00000100:00080000:2.0:1787829407.477063:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829407.477067:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829407.477073:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5553 00000100:00080000:2.0:1787829407.477077:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829407.477082:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787829407.477088:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829407.477102:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db480 x1874669228387456/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829407.477166:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228387328:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787829407.477190:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228387456:192.168.203.114@tcp:38:kworker.0 00010000:00010000:3.0:1787829407.540902:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829408.436813:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829408.564863:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829409.460781:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829409.588858:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829410.484837:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000400:00000100:1.0:1787829410.548973:0:2372:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.114:988 00000400:00000100:1.0:1787829410.549029:0:2372:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.114@tcp at host 192.168.203.114 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1787829410.549048:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:1.0:1787829410.549096:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:1.0:1787829410.549108:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000100:00100000:3.0:1787829410.549155:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829407/real 1787829410] req@ffff9856140d9a40 x1874669228387328/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829432 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829410.549188:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d9a40 x1874669228387328/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829432 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829410.549202:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829410.549209:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829410.549215:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228387328:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787829410.549224:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829407/real 1787829410] req@ffff9856140db480 x1874669228387456/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829432 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829410.549250:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db480 x1874669228387456/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829432 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829410.549259:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829410.549263:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829410.549266:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228387456:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787829410.549274:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829407/real 1787829410] req@ffff9856140d83c0 x1874669228387200/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829432 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829410.549296:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d83c0 x1874669228387200/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829432 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829410.549303:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829410.549307:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829410.549311:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228387200:192.168.203.114@tcp:250:kworker.0 00010000:00010000:1.0:1787829410.612891:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829411.508786:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829411.636876:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829412.532803:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787829412.596898:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829412.596906:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829412.596910:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829412.596921:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5558 00000100:00080000:2.0:1787829412.596933:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829412.596941:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787829412.596978:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8780 x1874669228387584/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829412.597012:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829412.597017:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829412.597020:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829412.597026:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5558 00000100:00080000:2.0:1787829412.597031:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829412.597051:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787829412.597065:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829412.597084:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d83c0 x1874669228387712/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787829412.597089:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228387584:192.168.203.114@tcp:250:kworker.0 00000100:00080000:2.0:1787829412.597105:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829412.597110:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829412.597112:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829412.597117:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5558 00000100:00080000:2.0:1787829412.597122:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829412.597126:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787829412.597132:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829412.597148:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db480 x1874669228387840/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787829412.597192:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829412.597251:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829412/real 1787829412] req@ffff9856140d8780 x1874669228387584/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829437 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829412.597277:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8780 x1874669228387584/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829437 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829412.597293:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829412.597297:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829412.597302:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228387584:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829412.597345:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228387712:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829412.597377:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829412.597393:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829412/real 1787829412] req@ffff9856140d83c0 x1874669228387712/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829437 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829412.597406:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d83c0 x1874669228387712/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829437 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829412.597415:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829412.597419:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829412.597422:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228387712:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829412.597429:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228387840:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829412.597448:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829412.597460:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829412/real 1787829412] req@ffff9856140db480 x1874669228387840/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829437 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829412.597471:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db480 x1874669228387840/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829437 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829412.597480:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829412.597484:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829412.597487:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228387840:192.168.203.114@tcp:38:kworker.0 00010000:00010000:3.0:1787829412.660888:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829413.556777:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829413.684965:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829414.580839:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829414.708798:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829415.604825:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829415.732919:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829416.628812:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829416.756810:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829417.652799:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787829417.716804:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829417.716811:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829417.716815:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829417.716824:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5563 00000100:00080000:2.0:1787829417.716833:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829417.716839:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787829417.716873:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d9a40 x1874669228387968/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829417.716896:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829417.716899:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829417.716900:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829417.716903:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5563 00000100:00080000:2.0:1787829417.716905:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829417.716907:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787829417.716915:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829417.716924:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d9680 x1874669228388096/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829417.716931:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829417.716934:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829417.716935:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829417.716938:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5563 00000100:00080000:2.0:1787829417.716939:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829417.716942:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787829417.716946:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829417.716954:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8b40 x1874669228388224/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787829417.716963:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228387968:192.168.203.114@tcp:250:kworker.0 00000800:00000100:0.0:1787829417.717053:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829417.717091:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829417/real 1787829417] req@ffff9856140d9a40 x1874669228387968/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829442 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829417.717106:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d9a40 x1874669228387968/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829442 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829417.717116:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829417.717120:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829417.717123:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228387968:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829417.717129:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228388096:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829417.717145:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829417.717152:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829417/real 1787829417] req@ffff9856140d9680 x1874669228388096/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829442 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829417.717160:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d9680 x1874669228388096/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829442 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829417.717166:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829417.717168:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829417.717170:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228388096:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829417.717211:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228388224:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829417.717230:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829417.717238:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829417/real 1787829417] req@ffff9856140d8b40 x1874669228388224/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829442 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829417.717246:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8b40 x1874669228388224/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829442 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829417.717251:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829417.717254:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829417.717256:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228388224:192.168.203.114@tcp:38:kworker.0 00010000:00010000:3.0:1787829417.780950:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829418.676792:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829418.804877:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829419.700797:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829419.828824:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829420.724834:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829420.852908:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829421.748851:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829421.876797:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829422.772839:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787829422.836885:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829422.836900:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829422.836905:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829422.836919:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5568 00000100:00080000:3.0:1787829422.836934:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829422.836945:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787829422.836997:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83480 x1874669228388352/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829422.837023:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829422.837026:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829422.837027:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829422.837030:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5568 00000100:00080000:3.0:1787829422.837034:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829422.837036:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829422.837048:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829422.837058:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f821c0 x1874669228388480/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829422.837066:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829422.837069:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829422.837071:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829422.837074:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5568 00000100:00080000:3.0:1787829422.837076:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829422.837079:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829422.837082:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829422.837093:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82580 x1874669228388608/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787829422.837108:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228388352:192.168.203.114@tcp:250:kworker.0 00000800:00000100:0.0:1787829422.837197:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829422.837238:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829422/real 1787829422] req@ffff985610f83480 x1874669228388352/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829447 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829422.837257:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83480 x1874669228388352/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829447 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829422.837269:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829422.837273:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829422.837278:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228388352:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829422.837287:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f821c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228388480:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829422.837312:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829422.837325:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829422/real 1787829422] req@ffff985610f821c0 x1874669228388480/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829447 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829422.837338:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f821c0 x1874669228388480/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829447 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829422.837347:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829422.837351:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829422.837355:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f821c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228388480:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829422.837383:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228388608:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829422.837408:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829422.837421:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829422/real 1787829422] req@ffff985610f82580 x1874669228388608/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829447 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829422.837434:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82580 x1874669228388608/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829447 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829422.837443:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829422.837447:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829422.837450:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228388608:192.168.203.114@tcp:38:kworker.0 00010000:00010000:3.0:1787829422.900912:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829423.796801:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829423.924855:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829424.820806:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829424.948831:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829425.844837:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829425.972937:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829426.868793:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829426.996921:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829427.892834:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787829427.956780:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829427.956792:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829427.956796:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829427.956807:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5573 00000100:00080000:1.0:1787829427.956820:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829427.956827:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787829427.956870:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a1a40 x1874669228388736/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829427.956893:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829427.956897:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829427.956898:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829427.956901:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5573 00000100:00080000:1.0:1787829427.956903:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829427.956906:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787829427.956916:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829427.956925:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2940 x1874669228388864/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829427.956933:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829427.956936:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829427.956937:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829427.956940:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5573 00000100:00080000:1.0:1787829427.956943:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829427.956946:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787829427.956950:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829427.956956:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228388736:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787829427.956960:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a12c0 x1874669228388992/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787829427.957049:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829427.957093:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829427/real 1787829427] req@ffff9856085a1a40 x1874669228388736/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829452 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829427.957131:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a1a40 x1874669228388736/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829452 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829427.957145:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829427.957151:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829427.957156:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228388736:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829427.957164:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228388864:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829427.957195:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829427.957210:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829427/real 1787829427] req@ffff9856085a2940 x1874669228388864/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829452 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829427.957224:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2940 x1874669228388864/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829452 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829427.957233:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829427.957237:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829427.957241:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228388864:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829427.957272:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228388992:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829427.957294:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829427.957306:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829427/real 1787829427] req@ffff9856085a12c0 x1874669228388992/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829452 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829427.957318:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a12c0 x1874669228388992/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829452 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829427.957327:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829427.957331:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829427.957335:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228388992:192.168.203.114@tcp:38:kworker.0 00010000:00010000:3.0:1787829428.020942:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829428.916780:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829429.044943:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829429.940825:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829430.068858:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829430.964778:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829431.092924:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829431.988805:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829432.116820:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829433.012842:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787829433.076751:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829433.076760:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829433.076764:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829433.076771:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5578 00000100:00080000:1.0:1787829433.076780:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829433.076786:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787829433.076814:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a21c0 x1874669228389120/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829433.076834:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829433.076837:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829433.076838:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829433.076841:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5578 00000100:00080000:1.0:1787829433.076843:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829433.076845:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787829433.076852:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829433.076860:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3480 x1874669228389248/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829433.076867:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829433.076870:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829433.076872:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829433.076875:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5578 00000100:00080000:1.0:1787829433.076877:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829433.076879:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787829433.076882:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829433.076890:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a30c0 x1874669228389376/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787829433.076895:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228389120:192.168.203.114@tcp:250:kworker.0 00000800:00000100:0.0:1787829433.076962:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829433.076995:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829433/real 1787829433] req@ffff9856085a21c0 x1874669228389120/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829458 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829433.077013:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a21c0 x1874669228389120/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829458 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829433.077025:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829433.077029:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829433.077033:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228389120:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829433.077041:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228389248:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829433.077063:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829433.077086:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829433/real 1787829433] req@ffff9856085a3480 x1874669228389248/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829458 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829433.077100:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3480 x1874669228389248/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829458 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829433.077109:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829433.077112:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829433.077116:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228389248:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829433.077137:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228389376:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829433.077162:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829433.077175:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829433/real 1787829433] req@ffff9856085a30c0 x1874669228389376/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829458 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829433.077188:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a30c0 x1874669228389376/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829458 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829433.077196:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829433.077200:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829433.077203:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228389376:192.168.203.114@tcp:38:kworker.0 00010000:00010000:3.0:1787829433.140873:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829434.036827:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829434.164879:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829435.060800:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829435.188889:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829436.084812:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829436.212828:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829437.108807:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829437.236895:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829438.132833:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787829438.196813:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829438.196824:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829438.196827:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829438.196838:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5583 00000100:00080000:3.0:1787829438.196850:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829438.196857:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787829438.196897:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82940 x1874669228389504/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829438.196924:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829438.196927:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829438.196929:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829438.196932:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5583 00000100:00080000:3.0:1787829438.196934:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829438.196937:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829438.196947:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829438.196959:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80f00 x1874669228389632/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787829438.196969:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228389504:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787829438.196973:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829438.196979:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829438.196981:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829438.196987:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5583 00000100:00080000:3.0:1787829438.196992:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829438.196997:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829438.197003:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829438.197018:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82d00 x1874669228389760/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787829438.197056:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829438.197091:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829438/real 1787829438] req@ffff985610f82940 x1874669228389504/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829463 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829438.197106:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82940 x1874669228389504/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829463 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829438.197116:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829438.197125:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829438.197129:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228389504:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829438.197152:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228389632:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829438.197170:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829438.197179:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829438/real 1787829438] req@ffff985610f80f00 x1874669228389632/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829463 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829438.197188:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80f00 x1874669228389632/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829463 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829438.197194:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829438.197196:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829438.197199:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228389632:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829438.197202:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228389760:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829438.197213:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829438.197218:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829438/real 1787829438] req@ffff985610f82d00 x1874669228389760/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829463 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829438.197226:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82d00 x1874669228389760/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829463 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829438.197231:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829438.197233:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829438.197235:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228389760:192.168.203.114@tcp:38:kworker.0 00010000:00010000:3.0:1787829438.260876:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829439.156831:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829439.284859:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829440.180814:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829440.308859:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829441.204848:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829441.332898:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829442.228798:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829442.356868:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829443.252775:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787829443.316791:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829443.316804:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829443.316809:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829443.316823:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5588 00000100:00080000:3.0:1787829443.316838:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829443.316848:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787829443.316895:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81e00 x1874669228389888/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829443.316932:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829443.316936:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829443.316939:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829443.316943:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5588 00000100:00080000:3.0:1787829443.316947:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829443.316952:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829443.316964:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829443.316979:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f803c0 x1874669228390016/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829443.316990:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829443.316994:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829443.316997:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829443.317019:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5588 00000100:00080000:3.0:1787829443.317024:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829443.317028:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829443.317035:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829443.317039:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228389888:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787829443.317051:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83840 x1874669228390144/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787829443.317115:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829443.317152:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829443/real 1787829443] req@ffff985610f81e00 x1874669228389888/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829468 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829443.317167:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81e00 x1874669228389888/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829468 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829443.317177:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829443.317180:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829443.317183:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228389888:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829443.317188:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228390016:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829443.317203:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829443.317211:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829443/real 1787829443] req@ffff985610f803c0 x1874669228390016/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829468 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829443.317218:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f803c0 x1874669228390016/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829468 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829443.317224:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829443.317226:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829443.317228:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228390016:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829443.317253:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228390144:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829443.317264:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829443.317270:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829443/real 1787829443] req@ffff985610f83840 x1874669228390144/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829468 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829443.317278:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83840 x1874669228390144/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829468 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829443.317283:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829443.317285:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829443.317287:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228390144:192.168.203.114@tcp:38:kworker.0 00010000:00010000:3.0:1787829443.380876:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829444.276798:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829444.404837:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829445.300798:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829445.428815:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829446.324795:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829446.452866:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829447.348803:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829447.476852:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829448.372790:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787829448.436819:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829448.436828:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829448.436832:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829448.436840:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5594 00000100:00080000:1.0:1787829448.436849:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829448.436855:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787829448.436885:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0f00 x1874669228390272/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829448.436908:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829448.436912:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829448.436913:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829448.436917:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5594 00000100:00080000:1.0:1787829448.436920:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829448.436924:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787829448.436932:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829448.436944:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228390272:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787829448.436946:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a30c0 x1874669228390400/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829448.436963:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829448.436977:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829448.436979:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829448.436985:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5594 00000100:00080000:1.0:1787829448.436990:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829448.436995:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787829448.437002:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829448.437018:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3480 x1874669228390528/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829448.437019:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829448.437056:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829448/real 1787829448] req@ffff9856085a0f00 x1874669228390272/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829473 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829448.437072:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0f00 x1874669228390272/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829473 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829448.437080:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829448.437083:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829448.437086:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228390272:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787829448.437106:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228390400:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829448.437212:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829448.437330:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829448/real 1787829448] req@ffff9856085a30c0 x1874669228390400/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829473 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829448.437344:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a30c0 x1874669228390400/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829473 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829448.437351:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829448.437354:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829448.437357:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228390400:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787829448.437363:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228390528:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829448.437438:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829448.437454:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829448/real 1787829448] req@ffff9856085a3480 x1874669228390528/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829473 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829448.437468:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3480 x1874669228390528/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829473 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829448.437474:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829448.437478:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829448.437481:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228390528:192.168.203.114@tcp:38:kworker.0 00010000:00010000:3.0:1787829448.500885:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829449.396822:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829449.524926:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829450.420771:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829450.548901:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829451.444796:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829451.572895:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829452.468770:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829452.596848:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829453.492841:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787829453.556847:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829453.556859:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829453.556863:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829453.556873:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5599 00000100:00080000:3.0:1787829453.556884:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829453.556891:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787829453.556947:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81680 x1874669228390656/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829453.556976:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829453.556978:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829453.556980:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829453.556983:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5599 00000100:00080000:3.0:1787829453.556988:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829453.556994:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829453.557007:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829453.557013:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228390656:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787829453.557025:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f812c0 x1874669228390784/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829453.557039:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829453.557043:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829453.557045:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829453.557049:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5599 00000100:00080000:3.0:1787829453.557053:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829453.557056:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829453.557061:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829453.557071:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81a40 x1874669228390912/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787829453.557100:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829453.557137:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829453/real 1787829453] req@ffff985610f81680 x1874669228390656/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829478 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829453.557151:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81680 x1874669228390656/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829478 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829453.557160:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829453.557163:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829453.557167:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228390656:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829453.557190:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228390784:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829453.557216:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829453.557225:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829453/real 1787829453] req@ffff985610f812c0 x1874669228390784/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829478 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829453.557234:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f812c0 x1874669228390784/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829478 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829453.557240:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829453.557242:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829453.557245:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228390784:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829453.557251:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228390912:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829453.557262:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829453.557268:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829453/real 1787829453] req@ffff985610f81a40 x1874669228390912/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829478 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829453.557276:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81a40 x1874669228390912/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829478 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829453.557281:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829453.557283:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829453.557285:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228390912:192.168.203.114@tcp:38:kworker.0 00010000:00010000:3.0:1787829453.620842:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829454.516818:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829454.644842:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829455.540811:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829455.668854:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829456.564765:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829456.692833:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829457.588818:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829457.716859:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829458.612784:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787829458.676827:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829458.676839:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829458.676844:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829458.676857:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5604 00000100:00080000:2.0:1787829458.676871:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829458.676880:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787829458.676938:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140da940 x1874669228391040/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829458.676971:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829458.676975:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829458.676977:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829458.676981:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5604 00000100:00080000:2.0:1787829458.676985:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829458.676989:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787829458.677000:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829458.677015:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db0c0 x1874669228391168/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829458.677027:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829458.677030:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829458.677032:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829458.677038:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5604 00000100:00080000:2.0:1787829458.677042:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00100000:0.0:1787829458.677043:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228391040:192.168.203.114@tcp:250:kworker.0 00000100:00080000:2.0:1787829458.677049:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787829458.677057:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829458.677068:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140dad00 x1874669228391296/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787829458.677133:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829458.677170:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829458/real 1787829458] req@ffff9856140da940 x1874669228391040/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829483 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829458.677185:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140da940 x1874669228391040/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829483 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829458.677194:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829458.677198:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829458.677201:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228391040:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829458.677227:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228391168:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829458.677247:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829458.677255:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829458/real 1787829458] req@ffff9856140db0c0 x1874669228391168/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829483 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829458.677264:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db0c0 x1874669228391168/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829483 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829458.677270:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829458.677272:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829458.677275:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228391168:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829458.677278:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228391296:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829458.677290:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829458.677296:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829458/real 1787829458] req@ffff9856140dad00 x1874669228391296/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829483 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829458.677303:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140dad00 x1874669228391296/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829483 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829458.677309:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829458.677311:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829458.677313:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228391296:192.168.203.114@tcp:38:kworker.0 00010000:00010000:3.0:1787829458.740934:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829459.636803:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829459.764838:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829460.660826:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829460.788841:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829461.684800:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829461.812888:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829462.708834:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829462.836844:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829463.732785:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787829463.796835:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829463.796848:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829463.796853:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829463.796881:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5609 00000100:00080000:2.0:1787829463.796895:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829463.796905:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787829463.796957:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8f00 x1874669228391424/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829463.796992:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829463.796996:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829463.796998:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829463.797001:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5609 00000100:00080000:2.0:1787829463.797004:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829463.797008:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787829463.797019:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829463.797035:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db840 x1874669228391552/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829463.797049:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829463.797054:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829463.797056:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1787829463.797061:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228391424:192.168.203.114@tcp:250:kworker.0 00000100:00080000:2.0:1787829463.797063:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5609 00000100:00080000:2.0:1787829463.797071:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829463.797078:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787829463.797082:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829463.797094:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140dbc00 x1874669228391680/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787829463.797158:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829463.797201:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829463/real 1787829463] req@ffff9856140d8f00 x1874669228391424/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829488 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829463.797218:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8f00 x1874669228391424/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829488 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829463.797227:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829463.797230:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829463.797233:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228391424:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829463.797264:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228391552:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829463.797291:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829463.797305:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829463/real 1787829463] req@ffff9856140db840 x1874669228391552/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829488 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829463.797317:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db840 x1874669228391552/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829488 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829463.797326:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829463.797330:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829463.797333:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228391552:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829463.797340:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228391680:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829463.797361:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829463.797372:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829463/real 1787829463] req@ffff9856140dbc00 x1874669228391680/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829488 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829463.797385:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140dbc00 x1874669228391680/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829488 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829463.797397:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829463.797399:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829463.797401:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228391680:192.168.203.114@tcp:38:kworker.0 00010000:00010000:3.0:1787829463.860934:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829464.756779:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829464.884842:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829465.780811:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829465.908913:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829466.804851:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829466.932865:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829467.828798:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829467.956884:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829468.852811:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787829468.916837:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829468.916850:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829468.916855:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829468.916867:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5614 00000100:00080000:0.0:1787829468.916881:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829468.916891:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787829468.916937:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6940 x1874669228391808/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829468.916969:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829468.916974:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829468.916976:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829468.916980:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5614 00000100:00080000:0.0:1787829468.916985:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829468.916991:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787829468.917003:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829468.917015:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228391808:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829468.917030:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4b40 x1874669228391936/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829468.917040:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829468.917044:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829468.917046:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829468.917049:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5614 00000100:00080000:0.0:1787829468.917053:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829468.917055:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787829468.917059:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829468.917067:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7480 x1874669228392064/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829468.917099:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829468.917136:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829468/real 1787829468] req@ffff985607df6940 x1874669228391808/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829493 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829468.917152:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6940 x1874669228391808/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829493 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829468.917162:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829468.917165:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829468.917168:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228391808:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787829468.917193:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228391936:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829468.917209:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829468.917228:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829468/real 1787829468] req@ffff985607df4b40 x1874669228391936/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829493 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829468.917243:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4b40 x1874669228391936/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829493 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829468.917252:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829468.917256:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829468.917264:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228391936:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787829468.917272:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228392064:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829468.917293:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829468.917302:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829468/real 1787829468] req@ffff985607df7480 x1874669228392064/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829493 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829468.917311:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7480 x1874669228392064/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829493 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829468.917316:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829468.917319:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829468.917321:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228392064:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829468.980862:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829469.876769:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829470.004918:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829470.900843:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829471.028858:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829471.924843:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829472.052898:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829472.948849:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829473.076850:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829473.972825:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787829474.036809:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829474.036820:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829474.036825:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829474.036835:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5619 00000100:00080000:1.0:1787829474.036847:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829474.036854:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787829474.036898:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a21c0 x1874669228392192/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829474.036925:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829474.036929:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829474.036931:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829474.036935:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5619 00000100:00080000:1.0:1787829474.036939:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829474.036950:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787829474.036960:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829474.036972:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a12c0 x1874669228392320/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829474.036973:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228392192:192.168.203.114@tcp:250:kworker.0 00000100:00080000:1.0:1787829474.036983:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829474.036986:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829474.036994:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829474.036998:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5619 00000100:00080000:1.0:1787829474.037002:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829474.037005:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787829474.037009:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829474.037019:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2940 x1874669228392448/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829474.037068:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228392320:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787829474.037094:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228392448:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829474.100816:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829474.996839:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829475.124855:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829476.020791:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829476.148851:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829477.044833:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000400:00000100:2.0:1787829477.109060:0:2373:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.114:988 00000400:00000100:2.0:1787829477.109121:0:2373:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.114@tcp at host 192.168.203.114 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:2.0:1787829477.109145:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:2.0:1787829477.109189:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:2.0:1787829477.109204:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000100:00100000:3.0:1787829477.109248:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829474/real 1787829477] req@ffff9856085a12c0 x1874669228392320/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829499 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829477.109282:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a12c0 x1874669228392320/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829499 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829477.109298:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829477.109304:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829477.109309:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228392320:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787829477.109319:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829474/real 1787829477] req@ffff9856085a2940 x1874669228392448/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829499 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829477.109332:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2940 x1874669228392448/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829499 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829477.109340:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829477.109344:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829477.109348:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228392448:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787829477.109356:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829474/real 1787829477] req@ffff9856085a21c0 x1874669228392192/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829499 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829477.109376:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a21c0 x1874669228392192/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829499 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829477.109384:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829477.109389:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829477.109393:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228392192:192.168.203.114@tcp:250:kworker.0 00010000:00010000:1.0:1787829477.173056:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829478.068835:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829478.196919:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829479.092819:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787829479.156849:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829479.156863:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829479.156868:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829479.156881:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5624 00000100:00080000:2.0:1787829479.156896:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829479.156905:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787829479.156950:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140da1c0 x1874669228392576/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829479.156981:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829479.156985:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829479.156987:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829479.156991:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5624 00000100:00080000:2.0:1787829479.156995:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829479.156999:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787829479.157012:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829479.157028:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140dbc00 x1874669228392704/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829479.157041:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228392576:192.168.203.114@tcp:250:kworker.0 00000100:00080000:2.0:1787829479.157043:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829479.157053:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829479.157055:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829479.157061:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5624 00000100:00080000:2.0:1787829479.157064:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829479.157069:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787829479.157074:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829479.157086:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db840 x1874669228392832/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829479.157142:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829479.157184:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829479/real 1787829479] req@ffff9856140da1c0 x1874669228392576/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829504 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829479.157206:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140da1c0 x1874669228392576/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829504 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829479.157219:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829479.157223:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829479.157227:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228392576:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787829479.157260:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228392704:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829479.157286:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829479.157300:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829479/real 1787829479] req@ffff9856140dbc00 x1874669228392704/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829504 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829479.157314:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140dbc00 x1874669228392704/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829504 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829479.157323:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829479.157327:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829479.157330:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228392704:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787829479.157336:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228392832:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829479.157356:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829479.157367:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829479/real 1787829479] req@ffff9856140db840 x1874669228392832/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829504 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829479.157380:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db840 x1874669228392832/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829504 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829479.157394:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829479.157397:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829479.157400:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228392832:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829479.220883:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829480.116821:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829480.244954:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829481.140818:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829481.268815:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829482.164820:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829482.292937:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829483.188807:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829483.316897:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829484.212816:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787829484.276807:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829484.276814:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829484.276817:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829484.276825:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5629 00000100:00080000:1.0:1787829484.276833:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829484.276840:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787829484.276866:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a1a40 x1874669228392960/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829484.276892:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829484.276894:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829484.276896:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829484.276899:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5629 00000100:00080000:1.0:1787829484.276901:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829484.276904:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787829484.276911:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829484.276926:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228392960:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787829484.276929:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2580 x1874669228393088/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829484.276944:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829484.276947:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829484.276948:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829484.276953:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5629 00000100:00080000:1.0:1787829484.276957:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829484.276960:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787829484.276963:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829484.276973:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3840 x1874669228393216/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829484.276992:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787829484.277034:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829484/real 1787829484] req@ffff9856085a1a40 x1874669228392960/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829509 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829484.277050:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a1a40 x1874669228392960/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829509 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829484.277059:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829484.277062:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787829484.277065:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228392960:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787829484.277084:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228393088:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787829484.277098:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787829484.277105:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829484/real 1787829484] req@ffff9856085a2580 x1874669228393088/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829509 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829484.277112:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2580 x1874669228393088/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829509 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829484.277116:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829484.277118:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787829484.277120:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228393088:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787829484.277123:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228393216:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787829484.277132:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787829484.277136:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829484/real 1787829484] req@ffff9856085a3840 x1874669228393216/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829509 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829484.277142:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3840 x1874669228393216/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829509 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829484.277147:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829484.277149:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787829484.277151:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228393216:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829484.340856:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829485.236810:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829485.364946:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829486.260837:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829486.388796:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829487.284822:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829487.413025:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829488.308775:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829488.436806:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829489.332767:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787829489.396796:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829489.396805:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829489.396808:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829489.396814:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5635 00000100:00080000:2.0:1787829489.396822:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829489.396826:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787829489.396855:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8f00 x1874669228393344/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829489.396883:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829489.396887:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829489.396889:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829489.396893:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5635 00000100:00080000:2.0:1787829489.396897:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829489.396901:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787829489.396910:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829489.396923:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140dad00 x1874669228393472/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829489.396933:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228393344:192.168.203.114@tcp:250:kworker.0 00000100:00080000:2.0:1787829489.396936:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829489.396941:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829489.396943:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829489.396947:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5635 00000100:00080000:2.0:1787829489.396951:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829489.396954:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787829489.396959:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829489.396968:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db0c0 x1874669228393600/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829489.396984:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829489.397009:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829489/real 1787829489] req@ffff9856140d8f00 x1874669228393344/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829514 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829489.397021:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8f00 x1874669228393344/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829514 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829489.397028:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829489.397030:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829489.397033:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228393344:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787829489.397051:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228393472:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829489.397064:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829489.397070:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829489/real 1787829489] req@ffff9856140dad00 x1874669228393472/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829514 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829489.397077:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140dad00 x1874669228393472/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829514 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829489.397082:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829489.397084:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829489.397086:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228393472:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787829489.397089:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228393600:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829489.397097:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829489.397102:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829489/real 1787829489] req@ffff9856140db0c0 x1874669228393600/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829514 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829489.397108:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db0c0 x1874669228393600/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829514 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829489.397113:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829489.397114:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829489.397116:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228393600:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829489.460820:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829490.356823:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829490.484888:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829491.380764:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829491.508859:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829492.404833:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829492.532871:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829493.428759:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829493.556864:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829494.452764:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787829494.516793:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829494.516802:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829494.516806:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829494.516815:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5640 00000100:00080000:2.0:1787829494.516825:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829494.516832:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787829494.516865:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140da940 x1874669228393728/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829494.516894:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829494.516898:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829494.516899:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829494.516903:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5640 00000100:00080000:2.0:1787829494.516907:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829494.516911:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787829494.516920:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829494.516934:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228393728:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787829494.516935:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8b40 x1874669228393856/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829494.516949:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829494.516952:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829494.516954:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829494.516959:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5640 00000100:00080000:2.0:1787829494.516963:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829494.516968:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787829494.516973:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829494.516983:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d9680 x1874669228393984/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829494.516995:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829494.517023:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829494/real 1787829494] req@ffff9856140da940 x1874669228393728/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829519 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829494.517035:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140da940 x1874669228393728/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829519 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829494.517041:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829494.517044:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829494.517047:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228393728:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787829494.517068:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228393856:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829494.517089:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829494.517100:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829494/real 1787829494] req@ffff9856140d8b40 x1874669228393856/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829519 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829494.517108:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8b40 x1874669228393856/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829519 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829494.517113:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829494.517115:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829494.517117:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228393856:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787829494.517121:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228393984:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829494.517133:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829494.517139:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829494/real 1787829494] req@ffff9856140d9680 x1874669228393984/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829519 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829494.517154:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d9680 x1874669228393984/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829519 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829494.517162:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829494.517164:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829494.517166:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228393984:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829494.580806:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829495.476770:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829495.604896:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829496.500792:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829496.628856:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829497.524748:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829497.652859:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829498.548770:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829498.676769:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829499.572798:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787829499.636811:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829499.636817:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829499.636820:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829499.636829:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5645 00000100:00080000:1.0:1787829499.636838:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829499.636844:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787829499.636880:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a03c0 x1874669228394112/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829499.636900:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829499.636902:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829499.636903:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829499.636906:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5645 00000100:00080000:1.0:1787829499.636908:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829499.636911:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787829499.636918:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829499.636930:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3c00 x1874669228394240/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829499.636937:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829499.636940:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829499.636941:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829499.636946:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5645 00000100:00080000:1.0:1787829499.636952:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787829499.636954:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228394112:192.168.203.114@tcp:250:kworker.0 00000100:00080000:1.0:1787829499.636957:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787829499.636968:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829499.636976:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0780 x1874669228394368/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829499.637041:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829499.637081:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829499/real 1787829499] req@ffff9856085a03c0 x1874669228394112/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829524 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829499.637104:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a03c0 x1874669228394112/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829524 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829499.637118:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829499.637122:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829499.637125:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228394112:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787829499.637150:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228394240:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829499.637170:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829499.637180:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829499/real 1787829499] req@ffff9856085a3c00 x1874669228394240/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829524 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829499.637190:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3c00 x1874669228394240/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829524 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829499.637199:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829499.637203:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829499.637207:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228394240:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787829499.637214:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228394368:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829499.637245:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829499.637255:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829499/real 1787829499] req@ffff9856085a0780 x1874669228394368/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829524 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829499.637264:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0780 x1874669228394368/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829524 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829499.637270:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829499.637272:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829499.637275:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228394368:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829499.700801:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829500.596774:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829500.725091:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829501.620791:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829501.748793:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829502.644772:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829502.772922:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829503.668839:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829503.796844:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829504.692759:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787829504.756743:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829504.756750:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829504.756753:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829504.756761:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5650 00000100:00080000:1.0:1787829504.756768:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829504.756773:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787829504.756801:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a1680 x1874669228394496/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829504.756820:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829504.756824:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829504.756825:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829504.756829:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5650 00000100:00080000:1.0:1787829504.756832:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829504.756835:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787829504.756844:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829504.756854:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0b40 x1874669228394624/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829504.756856:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228394496:192.168.203.114@tcp:250:kworker.0 00000100:00080000:1.0:1787829504.756868:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829504.756871:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829504.756872:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829504.756875:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5650 00000100:00080000:1.0:1787829504.756878:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829504.756881:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787829504.756885:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829504.756892:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0000 x1874669228394752/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829504.756909:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829504.756937:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829504/real 1787829504] req@ffff9856085a1680 x1874669228394496/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829529 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829504.756948:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a1680 x1874669228394496/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829529 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829504.756955:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829504.756957:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829504.756960:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228394496:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787829504.756978:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228394624:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829504.756995:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829504.757002:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829504/real 1787829504] req@ffff9856085a0b40 x1874669228394624/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829529 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829504.757009:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0b40 x1874669228394624/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829529 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829504.757014:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829504.757016:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829504.757017:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228394624:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787829504.757020:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228394752:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829504.757029:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829504.757034:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829504/real 1787829504] req@ffff9856085a0000 x1874669228394752/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829529 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829504.757040:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0000 x1874669228394752/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829529 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829504.757045:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829504.757046:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829504.757048:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228394752:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829504.820873:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829505.716768:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829505.844812:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829506.740749:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829506.868823:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829507.764830:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829507.892800:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829508.788737:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829508.916794:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829509.812774:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787829509.876803:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829509.876813:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829509.876816:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829509.876825:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5655 00000100:00080000:1.0:1787829509.876836:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829509.876843:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787829509.876875:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2d00 x1874669228394880/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829509.876905:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829509.876909:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829509.876910:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829509.876914:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5655 00000100:00080000:1.0:1787829509.876918:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829509.876922:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787829509.876931:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829509.876946:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0000 x1874669228395008/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829509.876949:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228394880:192.168.203.114@tcp:250:kworker.0 00000100:00080000:1.0:1787829509.876963:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829509.876967:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829509.876969:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829509.876973:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5655 00000100:00080000:1.0:1787829509.876976:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829509.876978:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787829509.876982:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829509.876990:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0b40 x1874669228395136/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829509.877000:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829509.877027:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829509/real 1787829509] req@ffff9856085a2d00 x1874669228394880/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829534 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829509.877039:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2d00 x1874669228394880/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829534 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829509.877046:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829509.877048:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829509.877050:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228394880:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787829509.877065:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228395008:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829509.877078:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829509.877084:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829509/real 1787829509] req@ffff9856085a0000 x1874669228395008/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829534 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829509.877091:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0000 x1874669228395008/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829534 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829509.877096:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829509.877097:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829509.877099:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228395008:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787829509.877102:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228395136:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829509.877111:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829509.877116:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829509/real 1787829509] req@ffff9856085a0b40 x1874669228395136/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829534 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829509.877122:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0b40 x1874669228395136/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829534 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829509.877130:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829509.877132:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829509.877134:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228395136:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829509.940793:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829510.836791:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829510.964931:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829511.860816:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829511.988892:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829512.884800:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829513.012897:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829513.908780:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829514.036871:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829514.932793:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787829514.996820:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829514.996831:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829514.996836:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829514.996846:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5660 00000100:00080000:0.0:1787829514.996858:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829514.996866:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787829514.996905:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df52c0 x1874669228395264/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829514.996928:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829514.996932:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829514.996934:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829514.996938:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5660 00000100:00080000:0.0:1787829514.996942:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829514.996947:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787829514.996966:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829514.996972:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228395264:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829514.996984:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7840 x1874669228395392/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829514.996994:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829514.996997:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829514.996998:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829514.997002:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5660 00000100:00080000:0.0:1787829514.997005:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829514.997008:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787829514.997018:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829514.997027:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4f00 x1874669228395520/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829514.997051:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829514.997097:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829514/real 1787829514] req@ffff985607df52c0 x1874669228395264/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829539 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829514.997116:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df52c0 x1874669228395264/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829539 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829514.997128:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829514.997133:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829514.997138:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228395264:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787829514.997166:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228395392:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829514.997191:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829514.997206:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829514/real 1787829514] req@ffff985607df7840 x1874669228395392/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829539 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829514.997221:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7840 x1874669228395392/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829539 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829514.997230:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829514.997233:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829514.997236:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228395392:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787829514.997240:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228395520:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829514.997257:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829514.997264:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829514/real 1787829514] req@ffff985607df4f00 x1874669228395520/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829539 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829514.997272:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4f00 x1874669228395520/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829539 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829514.997278:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829514.997280:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829514.997282:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228395520:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829515.060917:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829515.956822:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829516.084876:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829516.980802:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829517.108907:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829518.004786:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829518.132767:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829519.028782:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829519.156813:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829520.052769:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787829520.116787:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829520.116794:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829520.116798:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829520.116806:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5665 00000100:00080000:2.0:1787829520.116815:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829520.116821:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787829520.116848:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d9a40 x1874669228395648/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829520.116867:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829520.116871:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829520.116872:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829520.116876:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5665 00000100:00080000:2.0:1787829520.116880:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829520.116884:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787829520.116893:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829520.116908:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db480 x1874669228395776/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829520.116919:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829520.116923:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829520.116925:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829520.116929:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5665 00000100:00080000:2.0:1787829520.116934:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829520.116938:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787829520.116944:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228395648:192.168.203.114@tcp:250:kworker.0 00000008:00100000:2.0:1787829520.116946:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829520.116963:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d83c0 x1874669228395904/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787829520.117010:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829520.117038:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829520/real 1787829520] req@ffff9856140d9a40 x1874669228395648/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829545 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829520.117055:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d9a40 x1874669228395648/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829545 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829520.117065:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829520.117069:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829520.117074:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228395648:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829520.117082:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228395776:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829520.117116:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829520.117129:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829520/real 1787829520] req@ffff9856140db480 x1874669228395776/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829545 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829520.117148:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db480 x1874669228395776/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829545 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829520.117156:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829520.117160:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829520.117163:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228395776:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829520.117187:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228395904:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829520.117210:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829520.117224:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829520/real 1787829520] req@ffff9856140d83c0 x1874669228395904/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829545 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829520.117236:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d83c0 x1874669228395904/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829545 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829520.117244:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829520.117248:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829520.117251:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228395904:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829520.180810:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829521.076786:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829521.204848:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829522.100766:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829522.228812:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829523.124817:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829523.252836:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829524.148753:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829524.276897:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829525.172778:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787829525.236769:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829525.236776:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829525.236779:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829525.236787:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5670 00000100:00080000:3.0:1787829525.236795:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829525.236801:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787829525.236832:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80b40 x1874669228396032/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829525.236855:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829525.236858:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829525.236860:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829525.236866:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5670 00000100:00080000:3.0:1787829525.236870:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00100000:0.0:1787829525.236880:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228396032:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787829525.236881:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829525.236891:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829525.236903:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80000 x1874669228396160/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829525.236910:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829525.236913:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829525.236914:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829525.236917:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5670 00000100:00080000:3.0:1787829525.236919:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829525.236922:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829525.236924:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829525.236931:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f830c0 x1874669228396288/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787829525.236944:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829525.236969:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829525/real 1787829525] req@ffff985610f80b40 x1874669228396032/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829550 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829525.236984:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80b40 x1874669228396032/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829550 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829525.236991:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829525.236994:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829525.236996:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228396032:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829525.237015:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228396160:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829525.237036:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829525.237047:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829525/real 1787829525] req@ffff985610f80000 x1874669228396160/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829550 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829525.237058:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80000 x1874669228396160/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829550 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829525.237063:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829525.237065:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829525.237067:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228396160:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829525.237071:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228396288:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829525.237084:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829525.237091:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829525/real 1787829525] req@ffff985610f830c0 x1874669228396288/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829550 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829525.237097:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f830c0 x1874669228396288/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829550 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829525.237105:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829525.237107:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829525.237109:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228396288:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829525.300781:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829526.196756:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829526.324836:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829527.220774:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829527.348784:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829528.244752:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829528.372828:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829529.268780:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829529.396827:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829530.292817:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787829530.356817:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829530.356827:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829530.356830:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829530.356838:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5675 00000100:00080000:0.0:1787829530.356848:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829530.356855:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787829530.356887:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6d00 x1874669228396416/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829530.356911:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829530.356913:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829530.356915:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829530.356917:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5675 00000100:00080000:0.0:1787829530.356919:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829530.356922:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787829530.356928:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829530.356935:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5e00 x1874669228396544/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829530.356941:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829530.356943:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829530.356944:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829530.356946:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5675 00000100:00080000:0.0:1787829530.356948:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829530.356950:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787829530.356952:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829530.356959:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5680 x1874669228396672/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829530.356973:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228396416:192.168.203.114@tcp:250:kworker.0 00000800:00000100:3.0:1787829530.357020:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829530.357051:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829530/real 1787829530] req@ffff985607df6d00 x1874669228396416/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829555 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829530.357068:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6d00 x1874669228396416/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829555 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829530.357078:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829530.357082:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829530.357086:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228396416:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787829530.357093:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228396544:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829530.357111:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829530.357120:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829530/real 1787829530] req@ffff985607df5e00 x1874669228396544/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829555 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829530.357127:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5e00 x1874669228396544/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829555 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829530.357132:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829530.357134:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829530.357136:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228396544:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787829530.357152:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228396672:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829530.357162:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829530.357168:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829530/real 1787829530] req@ffff985607df5680 x1874669228396672/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829555 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829530.357174:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5680 x1874669228396672/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829555 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829530.357179:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829530.357180:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829530.357182:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228396672:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829530.420821:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829531.316775:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829531.444810:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829532.340807:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829532.468903:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829533.364796:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829533.492824:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829534.388802:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829534.516823:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829535.412850:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787829535.476866:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829535.476876:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829535.476880:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829535.476892:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5681 00000100:00080000:3.0:1787829535.476903:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829535.476910:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787829535.476950:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83c00 x1874669228396800/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829535.476981:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829535.476984:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829535.476985:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829535.476988:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5681 00000100:00080000:3.0:1787829535.476991:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829535.476994:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829535.477003:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829535.477013:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80780 x1874669228396928/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829535.477021:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829535.477024:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829535.477026:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829535.477028:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5681 00000100:00080000:3.0:1787829535.477031:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829535.477036:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829535.477040:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829535.477042:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228396800:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787829535.477051:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f830c0 x1874669228397056/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787829535.477128:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829535.477173:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829535/real 1787829535] req@ffff985610f83c00 x1874669228396800/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829560 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829535.477191:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83c00 x1874669228396800/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829560 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829535.477200:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829535.477204:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829535.477208:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228396800:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829535.477216:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228396928:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829535.477240:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829535.477251:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829535/real 1787829535] req@ffff985610f80780 x1874669228396928/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829560 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829535.477259:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80780 x1874669228396928/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829560 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829535.477264:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829535.477266:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829535.477268:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228396928:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829535.477322:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228397056:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829535.477344:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829535.477355:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829535/real 1787829535] req@ffff985610f830c0 x1874669228397056/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829560 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829535.477378:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f830c0 x1874669228397056/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829560 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829535.477385:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829535.477389:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829535.477392:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228397056:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829535.540805:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829536.436756:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829536.564880:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829537.460799:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829537.588834:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829538.484843:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829538.612842:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829539.508839:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829539.636848:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829540.532832:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787829540.596753:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829540.596760:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829540.596763:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829540.596771:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5686 00000100:00080000:3.0:1787829540.596782:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829540.596788:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787829540.596815:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80000 x1874669228397184/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829540.596836:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829540.596838:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829540.596840:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829540.596842:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5686 00000100:00080000:3.0:1787829540.596844:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829540.596847:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829540.596855:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829540.596866:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80b40 x1874669228397312/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829540.596876:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829540.596880:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829540.596883:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829540.596887:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5686 00000100:00080000:3.0:1787829540.596892:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829540.596895:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829540.596899:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829540.596906:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81a40 x1874669228397440/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787829540.596917:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228397184:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829540.597032:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228397312:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829540.597077:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228397440:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829540.660863:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829541.556830:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829541.684907:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829542.580816:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829542.708797:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829543.604791:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000400:00000100:1.0:1787829543.669006:0:2370:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.114:988 00000400:00000100:1.0:1787829543.669058:0:2370:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.114@tcp at host 192.168.203.114 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1787829543.669077:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:1.0:1787829543.669118:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:1.0:1787829543.669125:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000100:00100000:0.0:1787829543.669175:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829540/real 1787829543] req@ffff985610f81a40 x1874669228397440/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829565 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829543.669214:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81a40 x1874669228397440/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829565 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829543.669231:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829543.669238:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829543.669242:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228397440:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829543.669250:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829540/real 1787829543] req@ffff985610f80000 x1874669228397184/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829565 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829543.669265:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80000 x1874669228397184/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829565 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829543.669275:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829543.669279:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829543.669283:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228397184:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829543.669291:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829540/real 1787829543] req@ffff985610f80b40 x1874669228397312/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829565 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829543.669315:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80b40 x1874669228397312/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829565 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829543.669325:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829543.669330:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829543.669334:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228397312:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829543.732855:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829544.628810:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829544.756794:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829545.652808:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787829545.716844:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829545.716855:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829545.716860:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829545.716870:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5691 00000100:00080000:0.0:1787829545.716883:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829545.716891:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787829545.716930:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df61c0 x1874669228397568/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829545.716962:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829545.716973:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829545.716976:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829545.716980:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5691 00000100:00080000:0.0:1787829545.716985:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829545.716990:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787829545.716999:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829545.717004:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228397568:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829545.717016:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df43c0 x1874669228397696/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829545.717028:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829545.717032:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829545.717035:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829545.717039:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5691 00000100:00080000:0.0:1787829545.717043:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829545.717047:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787829545.717052:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829545.717065:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df70c0 x1874669228397824/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829545.717080:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829545.717120:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829545/real 1787829545] req@ffff985607df61c0 x1874669228397568/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829570 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829545.717140:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df61c0 x1874669228397568/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829570 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829545.717152:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829545.717156:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829545.717160:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228397568:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787829545.717191:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228397696:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829545.717223:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829545.717237:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829545/real 1787829545] req@ffff985607df43c0 x1874669228397696/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829570 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829545.717250:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df43c0 x1874669228397696/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829570 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829545.717258:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829545.717262:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829545.717266:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228397696:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787829545.717273:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228397824:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829545.717309:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829545.717323:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829545/real 1787829545] req@ffff985607df70c0 x1874669228397824/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829570 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829545.717337:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df70c0 x1874669228397824/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829570 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829545.717344:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829545.717348:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829545.717351:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228397824:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829545.780840:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829546.676827:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829546.804860:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829547.700766:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829547.828920:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829548.724847:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829548.852827:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829549.748802:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829549.876861:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829550.772780:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787829550.836852:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829550.836861:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829550.836864:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829550.836872:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5696 00000100:00080000:3.0:1787829550.836881:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829550.836886:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787829550.836918:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f812c0 x1874669228397952/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829550.836943:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829550.836946:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829550.836947:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829550.836950:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5696 00000100:00080000:3.0:1787829550.836953:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829550.836957:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829550.836964:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829550.836974:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81680 x1874669228398080/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829550.836982:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829550.836984:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829550.836986:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829550.836988:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5696 00000100:00080000:3.0:1787829550.836990:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829550.836992:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829550.836995:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829550.837004:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83840 x1874669228398208/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787829550.837014:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228397952:192.168.203.114@tcp:250:kworker.0 00000800:00000100:0.0:1787829550.837113:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829550.837151:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829550/real 1787829550] req@ffff985610f812c0 x1874669228397952/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829575 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829550.837167:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f812c0 x1874669228397952/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829575 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829550.837176:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829550.837180:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829550.837183:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228397952:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829550.837189:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228398080:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829550.837208:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829550.837216:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829550/real 1787829550] req@ffff985610f81680 x1874669228398080/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829575 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829550.837225:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81680 x1874669228398080/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829575 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829550.837231:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829550.837233:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829550.837235:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228398080:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829550.837272:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228398208:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829550.837287:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829550.837294:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829550/real 1787829550] req@ffff985610f83840 x1874669228398208/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829575 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829550.837302:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83840 x1874669228398208/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829575 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829550.837308:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829550.837310:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829550.837312:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228398208:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829550.900852:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829551.796796:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829551.924905:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829552.820812:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829552.948863:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829553.844840:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829553.972865:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829554.868810:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829554.996837:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829555.892800:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787829555.956804:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829555.956819:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829555.956824:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829555.956836:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5701 00000100:00080000:0.0:1787829555.956850:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829555.956860:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787829555.956919:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5a40 x1874669228398336/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829555.956946:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829555.956949:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829555.956951:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829555.956954:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5701 00000100:00080000:0.0:1787829555.956958:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829555.956962:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787829555.956972:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829555.956984:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228398336:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829555.956986:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6580 x1874669228398464/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829555.956999:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829555.957001:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829555.957002:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829555.957005:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5701 00000100:00080000:0.0:1787829555.957014:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829555.957017:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787829555.957021:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829555.957032:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df70c0 x1874669228398592/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829555.957062:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829555.957100:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829555/real 1787829555] req@ffff985607df5a40 x1874669228398336/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829580 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829555.957120:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5a40 x1874669228398336/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829580 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829555.957130:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829555.957135:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829555.957139:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228398336:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787829555.957163:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228398464:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829555.957181:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829555.957190:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829555/real 1787829555] req@ffff985607df6580 x1874669228398464/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829580 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829555.957200:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6580 x1874669228398464/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829580 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829555.957206:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829555.957209:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829555.957211:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228398464:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787829555.957215:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228398592:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829555.957246:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829555.957257:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829555/real 1787829555] req@ffff985607df70c0 x1874669228398592/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829580 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829555.957266:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df70c0 x1874669228398592/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829580 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829555.957273:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829555.957276:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829555.957278:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228398592:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829556.020875:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829556.916782:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829557.044899:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829557.940832:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829558.068835:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829558.964801:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829559.092874:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829559.988798:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829560.116863:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829561.012811:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787829561.076858:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829561.076869:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829561.076873:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829561.076885:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5706 00000100:00080000:2.0:1787829561.076898:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829561.076906:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787829561.076951:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8780 x1874669228398720/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829561.076978:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829561.076981:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829561.076983:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829561.076987:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5706 00000100:00080000:2.0:1787829561.076990:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829561.076993:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787829561.077004:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829561.077015:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228398720:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787829561.077018:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d83c0 x1874669228398848/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829561.077029:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829561.077032:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829561.077033:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829561.077037:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5706 00000100:00080000:2.0:1787829561.077040:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829561.077042:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787829561.077045:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829561.077053:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db480 x1874669228398976/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829561.077093:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829561.077130:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829561/real 1787829561] req@ffff9856140d8780 x1874669228398720/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829586 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829561.077146:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8780 x1874669228398720/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829586 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829561.077155:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829561.077159:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829561.077162:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228398720:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787829561.077198:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228398848:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829561.077218:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829561.077228:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829561/real 1787829561] req@ffff9856140d83c0 x1874669228398848/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829586 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829561.077237:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d83c0 x1874669228398848/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829586 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829561.077243:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829561.077246:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829561.077248:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228398848:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787829561.077253:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228398976:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829561.077264:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829561.077270:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829561/real 1787829561] req@ffff9856140db480 x1874669228398976/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829586 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829561.077283:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db480 x1874669228398976/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829586 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829561.077288:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829561.077290:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829561.077292:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228398976:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829561.140875:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829562.036835:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829562.164904:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829563.060797:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829563.188858:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829564.084810:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829564.212932:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829565.108763:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829565.236863:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829566.132836:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787829566.196807:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829566.196819:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829566.196824:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829566.196836:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5711 00000100:00080000:0.0:1787829566.196850:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829566.196859:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787829566.196906:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df43c0 x1874669228399104/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829566.196938:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829566.196943:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829566.196945:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829566.196950:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5711 00000100:00080000:0.0:1787829566.196956:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829566.196962:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787829566.196975:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829566.196979:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228399104:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829566.196993:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df61c0 x1874669228399232/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829566.197013:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829566.197017:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829566.197020:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829566.197024:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5711 00000100:00080000:0.0:1787829566.197029:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829566.197034:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787829566.197040:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829566.197053:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5680 x1874669228399360/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829566.197062:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829566.197099:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829566/real 1787829566] req@ffff985607df43c0 x1874669228399104/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829591 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829566.197116:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df43c0 x1874669228399104/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829591 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829566.197124:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829566.197128:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829566.197137:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228399104:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787829566.197178:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228399232:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829566.197213:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829566.197225:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829566/real 1787829566] req@ffff985607df61c0 x1874669228399232/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829591 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829566.197235:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df61c0 x1874669228399232/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829591 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829566.197246:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829566.197248:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829566.197251:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228399232:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787829566.197255:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228399360:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829566.197270:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829566.197279:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829566/real 1787829566] req@ffff985607df5680 x1874669228399360/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829591 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829566.197288:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5680 x1874669228399360/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829591 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829566.197294:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829566.197296:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829566.197298:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228399360:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829566.260804:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829567.156795:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829567.284935:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829568.180794:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829568.308858:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829569.204772:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829569.332861:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829570.228826:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787829570.356829:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829571.252809:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787829571.316783:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829571.316794:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829571.316798:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829571.316809:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5716 00000100:00080000:0.0:1787829571.316822:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829571.316829:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787829571.316869:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5e00 x1874669228399488/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829571.316896:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829571.316899:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829571.316900:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829571.316904:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5716 00000100:00080000:0.0:1787829571.316907:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829571.316911:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787829571.316920:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829571.316930:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6d00 x1874669228399616/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829571.316938:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228399488:192.168.203.114@tcp:250:kworker.0 00000100:00080000:0.0:1787829571.316942:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829571.316947:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829571.316948:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829571.316951:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5716 00000100:00080000:0.0:1787829571.316954:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829571.316956:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787829571.316959:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829571.316968:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4f00 x1874669228399744/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829571.317024:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829571.317060:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829571/real 1787829571] req@ffff985607df5e00 x1874669228399488/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829596 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829571.317077:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5e00 x1874669228399488/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829596 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829571.317086:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829571.317089:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829571.317093:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228399488:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787829571.317130:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228399616:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829571.317150:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829571.317160:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829571/real 1787829571] req@ffff985607df6d00 x1874669228399616/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829596 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829571.317169:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6d00 x1874669228399616/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829596 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829571.317175:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829571.317178:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829571.317180:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228399616:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787829571.317184:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228399744:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829571.317196:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829571.317202:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829571/real 1787829571] req@ffff985607df4f00 x1874669228399744/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829596 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829571.317210:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4f00 x1874669228399744/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829596 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829571.317215:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829571.317217:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829571.317219:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228399744:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829571.380896:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829572.276813:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829572.404879:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829573.300860:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829573.428842:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829574.324823:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829574.452802:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829575.348821:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787829575.476850:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829576.372798:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787829576.436805:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829576.436814:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829576.436818:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829576.436826:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5722 00000100:00080000:2.0:1787829576.436835:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829576.436840:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787829576.436871:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d9a40 x1874669228399872/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829576.436894:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829576.436897:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829576.436898:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829576.436901:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5722 00000100:00080000:2.0:1787829576.436903:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829576.436906:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787829576.436913:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829576.436922:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d9680 x1874669228400000/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829576.436931:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829576.436933:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829576.436936:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829576.436939:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5722 00000100:00080000:2.0:1787829576.436943:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829576.436946:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787829576.436949:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829576.436961:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8b40 x1874669228400128/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829576.436969:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228399872:192.168.203.114@tcp:250:kworker.0 00000800:00000100:3.0:1787829576.437070:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829576.437172:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829576/real 1787829576] req@ffff9856140d9a40 x1874669228399872/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829601 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829576.437201:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d9a40 x1874669228399872/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829601 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829576.437216:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829576.437221:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829576.437226:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228399872:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787829576.437236:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228400000:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829576.437265:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829576.437290:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829576/real 1787829576] req@ffff9856140d9680 x1874669228400000/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829601 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829576.437305:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d9680 x1874669228400000/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829601 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829576.437315:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829576.437319:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829576.437323:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228400000:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787829576.437356:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228400128:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829576.437380:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829576.437403:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829576/real 1787829576] req@ffff9856140d8b40 x1874669228400128/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829601 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829576.437425:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8b40 x1874669228400128/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829601 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829576.437434:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829576.437438:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829576.437442:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228400128:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829576.500767:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829577.396808:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829577.524881:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829578.420809:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829578.548848:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829579.444757:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829579.572881:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829580.468762:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787829580.596910:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829581.492786:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787829581.556777:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829581.556788:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829581.556792:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829581.556803:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5727 00000100:00080000:2.0:1787829581.556815:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829581.556823:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787829581.556865:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140da940 x1874669228400256/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829581.556892:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829581.556895:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829581.556897:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829581.556899:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5727 00000100:00080000:2.0:1787829581.556902:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829581.556906:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787829581.556915:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829581.556932:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db0c0 x1874669228400384/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829581.556941:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829581.556945:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:3.0:1787829581.556945:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228400256:192.168.203.114@tcp:250:kworker.0 00000100:00080000:2.0:1787829581.556949:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829581.556960:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5727 00000100:00080000:2.0:1787829581.556963:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829581.556966:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787829581.556970:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829581.556980:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140dad00 x1874669228400512/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829581.557026:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829581.557079:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829581/real 1787829581] req@ffff9856140da940 x1874669228400256/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829606 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829581.557102:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140da940 x1874669228400256/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829606 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829581.557112:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829581.557115:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829581.557118:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228400256:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787829581.557144:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228400384:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829581.557162:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829581.557171:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829581/real 1787829581] req@ffff9856140db0c0 x1874669228400384/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829606 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829581.557179:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db0c0 x1874669228400384/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829606 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829581.557190:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829581.557192:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829581.557194:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228400384:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787829581.557198:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228400512:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829581.557209:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829581.557215:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829581/real 1787829581] req@ffff9856140dad00 x1874669228400512/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829606 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829581.557222:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140dad00 x1874669228400512/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829606 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829581.557227:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829581.557229:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829581.557231:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228400512:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829581.620813:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829582.516793:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829582.644901:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829583.540828:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829583.668845:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829584.564791:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829584.692894:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829585.588833:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787829585.716864:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829586.612797:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787829586.676867:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829586.676879:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829586.676884:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829586.676894:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5732 00000100:00080000:0.0:1787829586.676907:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829586.676914:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787829586.676956:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7840 x1874669228400640/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829586.676982:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829586.676985:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829586.676986:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829586.676989:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5732 00000100:00080000:0.0:1787829586.676991:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829586.676994:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787829586.677015:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829586.677027:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df52c0 x1874669228400768/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829586.677038:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829586.677043:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829586.677045:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829586.677049:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5732 00000100:00080000:0.0:1787829586.677054:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787829586.677059:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228400640:192.168.203.114@tcp:250:kworker.0 00000100:00080000:0.0:1787829586.677062:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787829586.677073:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829586.677086:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7480 x1874669228400896/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829586.677153:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829586.677196:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829586/real 1787829586] req@ffff985607df7840 x1874669228400640/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829611 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829586.677218:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7840 x1874669228400640/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829611 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829586.677229:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829586.677234:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829586.677239:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228400640:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787829586.677247:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228400768:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829586.677272:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829586.677286:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829586/real 1787829586] req@ffff985607df52c0 x1874669228400768/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829611 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829586.677299:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df52c0 x1874669228400768/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829611 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829586.677307:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829586.677310:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829586.677314:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228400768:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787829586.677341:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228400896:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829586.677362:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829586.677374:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829586/real 1787829586] req@ffff985607df7480 x1874669228400896/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829611 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829586.677387:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7480 x1874669228400896/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829611 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829586.677396:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829586.677399:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829586.677403:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228400896:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829586.740908:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829587.636848:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829587.764858:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829588.660808:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829588.788898:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829589.684832:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829589.812819:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829590.708852:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787829590.836908:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829591.732862:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787829591.796870:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829591.796886:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829591.796891:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829591.796904:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5737 00000100:00080000:2.0:1787829591.796918:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829591.796933:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787829591.796994:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8f00 x1874669228401024/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829591.797027:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829591.797032:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829591.797035:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829591.797040:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5737 00000100:00080000:2.0:1787829591.797046:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829591.797052:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787829591.797064:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829591.797074:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228401024:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787829591.797083:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db840 x1874669228401152/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829591.797102:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829591.797107:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829591.797109:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829591.797114:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5737 00000100:00080000:2.0:1787829591.797120:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829591.797124:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787829591.797130:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829591.797143:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140dbc00 x1874669228401280/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787829591.797169:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829591.797212:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829591/real 1787829591] req@ffff9856140d8f00 x1874669228401024/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829616 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829591.797232:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8f00 x1874669228401024/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829616 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829591.797244:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829591.797249:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829591.797253:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228401024:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829591.797284:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228401152:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829591.797312:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829591.797328:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829591/real 1787829591] req@ffff9856140db840 x1874669228401152/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829616 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829591.797342:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db840 x1874669228401152/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829616 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829591.797351:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829591.797355:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829591.797359:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228401152:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829591.797370:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228401280:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829591.797394:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829591.797410:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829591/real 1787829591] req@ffff9856140dbc00 x1874669228401280/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829616 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829591.797423:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140dbc00 x1874669228401280/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829616 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829591.797433:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829591.797437:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829591.797441:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228401280:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829591.860824:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829592.756758:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829592.884934:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829593.780809:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829593.908846:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829594.804825:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829594.932867:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829595.828831:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787829595.956849:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829596.852827:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787829596.916832:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829596.916843:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829596.916847:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829596.916858:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5742 00000100:00080000:3.0:1787829596.916869:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829596.916877:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787829596.916919:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f803c0 x1874669228401408/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829596.916960:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829596.916964:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829596.916965:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829596.916969:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5742 00000100:00080000:3.0:1787829596.916972:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829596.916975:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829596.916986:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829596.916998:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228401408:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787829596.917000:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81e00 x1874669228401536/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829596.917013:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829596.917016:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829596.917018:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829596.917021:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5742 00000100:00080000:3.0:1787829596.917023:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829596.917026:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829596.917029:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829596.917039:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82d00 x1874669228401664/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787829596.917077:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829596.917123:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829596/real 1787829596] req@ffff985610f803c0 x1874669228401408/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829621 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829596.917140:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f803c0 x1874669228401408/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829621 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829596.917149:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829596.917152:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829596.917155:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228401408:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829596.917178:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228401536:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829596.917198:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829596.917207:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829596/real 1787829596] req@ffff985610f81e00 x1874669228401536/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829621 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829596.917215:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81e00 x1874669228401536/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829621 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829596.917221:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829596.917223:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829596.917226:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228401536:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829596.917230:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228401664:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829596.917241:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829596.917247:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829596/real 1787829596] req@ffff985610f82d00 x1874669228401664/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829621 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829596.917254:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82d00 x1874669228401664/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829621 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829596.917260:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829596.917261:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829596.917263:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228401664:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829596.980900:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829597.876804:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829598.004955:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829598.900834:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829599.028863:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829599.924795:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829600.052887:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829600.948826:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787829601.076888:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829601.972768:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787829602.036808:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829602.036815:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829602.036818:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829602.036826:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5747 00000100:00080000:0.0:1787829602.036834:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829602.036839:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787829602.036867:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4b40 x1874669228401792/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829602.036887:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829602.036890:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829602.036891:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829602.036893:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5747 00000100:00080000:0.0:1787829602.036896:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829602.036908:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787829602.036917:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829602.036925:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228401792:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829602.036930:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6940 x1874669228401920/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829602.036938:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829602.036940:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829602.036941:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829602.036944:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5747 00000100:00080000:0.0:1787829602.036947:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829602.036949:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787829602.036953:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829602.036960:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4780 x1874669228402048/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829602.036977:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829602.037003:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829602/real 1787829602] req@ffff985607df4b40 x1874669228401792/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829627 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829602.037014:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4b40 x1874669228401792/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829627 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829602.037022:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829602.037024:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829602.037027:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228401792:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787829602.037042:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228401920:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829602.037055:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829602.037061:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829602/real 1787829602] req@ffff985607df6940 x1874669228401920/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829627 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829602.037068:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6940 x1874669228401920/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829627 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829602.037073:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829602.037074:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829602.037076:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228401920:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787829602.037080:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228402048:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829602.037089:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829602.037094:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829602/real 1787829602] req@ffff985607df4780 x1874669228402048/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829627 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829602.037100:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4780 x1874669228402048/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829627 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829602.037105:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829602.037106:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829602.037108:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228402048:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829602.100856:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829602.996832:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829603.124872:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829604.020801:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829604.148860:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829605.044866:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829605.172789:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829606.068774:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787829606.196888:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829607.092849:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787829607.156797:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829607.156808:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829607.156812:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829607.156823:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5752 00000100:00080000:2.0:1787829607.156834:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829607.156842:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787829607.156894:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140da1c0 x1874669228402176/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829607.156922:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829607.156925:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829607.156927:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829607.156930:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5752 00000100:00080000:2.0:1787829607.156933:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829607.156936:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787829607.156946:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829607.156964:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140dbc00 x1874669228402304/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829607.156973:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228402176:192.168.203.114@tcp:250:kworker.0 00000100:00080000:2.0:1787829607.156982:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829607.156988:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829607.156990:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829607.156996:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5752 00000100:00080000:2.0:1787829607.157000:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829607.157003:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787829607.157008:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829607.157021:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db840 x1874669228402432/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829607.157081:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228402304:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787829607.157107:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228402432:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829607.220845:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829608.116793:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829608.244976:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829609.140812:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829609.268823:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829610.164771:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000400:00000100:3.0:1787829610.229055:0:2371:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.114:988 00000400:00000100:3.0:1787829610.229132:0:2371:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.114@tcp at host 192.168.203.114 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:3.0:1787829610.229153:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:3.0:1787829610.229212:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:3.0:1787829610.229224:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000100:00100000:0.0:1787829610.229370:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829607/real 1787829610] req@ffff9856140dbc00 x1874669228402304/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829632 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829610.229406:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140dbc00 x1874669228402304/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829632 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829610.229421:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829610.229428:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829610.229434:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228402304:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829610.229444:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829607/real 1787829610] req@ffff9856140db840 x1874669228402432/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829632 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829610.229459:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db840 x1874669228402432/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829632 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829610.229468:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829610.229472:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829610.229476:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228402432:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829610.229483:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829607/real 1787829610] req@ffff9856140da1c0 x1874669228402176/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829632 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829610.229497:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140da1c0 x1874669228402176/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829632 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829610.229507:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829610.229511:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829610.229515:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228402176:192.168.203.114@tcp:250:kworker.0 00010000:00010000:0.0:1787829610.292889:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829611.188842:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787829611.316820:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829612.212854:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787829612.276905:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829612.276920:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829612.276926:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829612.276941:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5757 00000100:00080000:0.0:1787829612.276955:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829612.276965:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787829612.277012:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4000 x1874669228402560/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829612.277053:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829612.277058:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829612.277060:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829612.277065:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5757 00000100:00080000:0.0:1787829612.277068:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829612.277073:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787829612.277085:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829612.277101:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7c00 x1874669228402688/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829612.277113:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1787829612.277122:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228402560:192.168.203.114@tcp:250:kworker.0 00000100:00080000:0.0:1787829612.277123:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829612.277126:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829612.277136:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5757 00000100:00080000:0.0:1787829612.277141:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829612.277146:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787829612.277152:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829612.277169:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4780 x1874669228402816/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829612.277242:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829612.277300:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829612/real 1787829612] req@ffff985607df4000 x1874669228402560/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829637 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829612.277345:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4000 x1874669228402560/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829637 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829612.277359:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829612.277363:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829612.277369:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228402560:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787829612.277404:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228402688:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829612.277431:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829612.277446:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829612/real 1787829612] req@ffff985607df7c00 x1874669228402688/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829637 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829612.278292:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7c00 x1874669228402688/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829637 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829612.278303:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829612.278309:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829612.278313:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228402688:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787829612.278324:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228402816:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829612.278372:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829612.278386:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829612/real 1787829612] req@ffff985607df4780 x1874669228402816/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829637 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829612.278396:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4780 x1874669228402816/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829637 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829612.278407:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829612.278411:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829612.278415:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228402816:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829612.340821:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829613.236810:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829613.364879:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829614.260767:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829614.388885:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829615.284784:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829615.412879:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829616.308847:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829616.436912:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829617.332851:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787829617.396850:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829617.396861:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829617.396865:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829617.396876:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5763 00000100:00080000:2.0:1787829617.396888:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829617.396895:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787829617.396937:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8f00 x1874669228402944/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829617.396962:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829617.396965:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829617.396967:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829617.396970:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5763 00000100:00080000:2.0:1787829617.396972:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829617.396975:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787829617.396984:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829617.396999:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140dad00 x1874669228403072/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829617.397007:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829617.397010:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829617.397011:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829617.397014:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5763 00000100:00080000:2.0:1787829617.397017:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829617.397020:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787829617.397025:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829617.397033:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db0c0 x1874669228403200/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829617.397074:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228402944:192.168.203.114@tcp:250:kworker.0 00000800:00000100:3.0:1787829617.397148:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829617.397196:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829617/real 1787829617] req@ffff9856140d8f00 x1874669228402944/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829642 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829617.397217:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8f00 x1874669228402944/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829642 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829617.397229:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829617.397234:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829617.397239:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228402944:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787829617.397247:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228403072:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829617.397270:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829617.397282:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829617/real 1787829617] req@ffff9856140dad00 x1874669228403072/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829642 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829617.397294:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140dad00 x1874669228403072/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829642 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829617.397310:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829617.397314:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829617.397317:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228403072:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787829617.397324:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228403200:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829617.397341:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829617.397352:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829617/real 1787829617] req@ffff9856140db0c0 x1874669228403200/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829642 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829617.397363:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db0c0 x1874669228403200/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829642 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829617.397371:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829617.397374:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829617.397378:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228403200:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829617.460828:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829618.356833:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829618.484870:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829619.380808:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829619.508898:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829620.404846:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829620.532840:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829621.428776:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829621.556888:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829622.452767:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787829622.516820:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829622.516832:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829622.516836:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829622.516849:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5768 00000100:00080000:3.0:1787829622.516862:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829622.516871:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787829622.516914:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80f00 x1874669228403328/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829622.516945:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829622.516948:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829622.516950:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829622.516954:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5768 00000100:00080000:3.0:1787829622.516959:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829622.516964:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829622.516975:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829622.516991:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82940 x1874669228403456/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829622.517005:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1787829622.517009:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228403328:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787829622.517010:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829622.517014:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829622.517022:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5768 00000100:00080000:3.0:1787829622.517025:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829622.517030:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829622.517036:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829622.517051:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82580 x1874669228403584/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787829622.517104:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829622.517148:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829622/real 1787829622] req@ffff985610f80f00 x1874669228403328/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829647 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829622.517165:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80f00 x1874669228403328/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829647 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829622.517174:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829622.517178:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829622.517181:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228403328:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829622.517204:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228403456:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829622.517234:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829622.517244:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829622/real 1787829622] req@ffff985610f82940 x1874669228403456/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829647 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829622.517253:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82940 x1874669228403456/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829647 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829622.517259:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829622.517261:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829622.517263:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228403456:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829622.517267:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228403584:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829622.517279:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829622.517286:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829622/real 1787829622] req@ffff985610f82580 x1874669228403584/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829647 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829622.517293:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82580 x1874669228403584/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829647 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829622.517298:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829622.517300:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829622.517302:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228403584:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829622.580835:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829623.476765:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829623.604935:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829624.500801:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829624.628837:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829625.524807:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829625.652854:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829626.548833:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829626.676925:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829627.572779:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787829627.636849:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829627.636858:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829627.636863:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829627.636874:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5773 00000100:00080000:3.0:1787829627.636884:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829627.636892:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787829627.636928:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f821c0 x1874669228403712/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829627.636969:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829627.636976:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829627.636980:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829627.636988:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5773 00000100:00080000:3.0:1787829627.636995:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829627.637004:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829627.637030:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829627.637035:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f821c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228403712:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787829627.637049:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83480 x1874669228403840/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829627.637062:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829627.637067:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829627.637069:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829627.637073:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5773 00000100:00080000:3.0:1787829627.637077:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829627.637082:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829627.637087:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829627.637102:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82580 x1874669228403968/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829627.637151:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787829627.637217:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829627/real 1787829627] req@ffff985610f821c0 x1874669228403712/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829652 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829627.637241:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f821c0 x1874669228403712/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829652 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829627.637252:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829627.637256:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787829627.637262:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f821c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228403712:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787829627.637292:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228403840:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787829627.637330:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787829627.637346:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829627/real 1787829627] req@ffff985610f83480 x1874669228403840/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829652 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829627.637365:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83480 x1874669228403840/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829652 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829627.637375:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829627.637380:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787829627.637384:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228403840:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787829627.637394:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228403968:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787829627.637421:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787829627.637446:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829627/real 1787829627] req@ffff985610f82580 x1874669228403968/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829652 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829627.637458:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82580 x1874669228403968/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829652 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829627.637470:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829627.637476:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787829627.637482:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228403968:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829627.700887:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829628.596829:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829628.724888:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829629.620768:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829629.748894:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829630.644835:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829630.772840:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829631.668763:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829631.796888:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829632.692798:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787829632.756757:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829632.756768:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829632.756772:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829632.756782:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5778 00000100:00080000:3.0:1787829632.756794:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829632.756802:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787829632.756843:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82940 x1874669228404096/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829632.756868:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829632.756871:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829632.756873:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829632.756875:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5778 00000100:00080000:3.0:1787829632.756878:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829632.756880:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829632.756889:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829632.756898:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80f00 x1874669228404224/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829632.756906:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829632.756909:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829632.756911:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829632.756914:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5778 00000100:00080000:3.0:1787829632.756916:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829632.756919:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829632.756922:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829632.756929:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82d00 x1874669228404352/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829632.756939:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228404096:192.168.203.114@tcp:250:kworker.0 00000800:00000100:2.0:1787829632.757027:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787829632.757066:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829632/real 1787829632] req@ffff985610f82940 x1874669228404096/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829657 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829632.757088:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82940 x1874669228404096/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829657 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829632.757101:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829632.757106:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787829632.757111:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228404096:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787829632.757120:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228404224:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787829632.757157:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787829632.757174:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829632/real 1787829632] req@ffff985610f80f00 x1874669228404224/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829657 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829632.757188:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80f00 x1874669228404224/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829657 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829632.757198:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829632.757201:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787829632.757213:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228404224:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787829632.757245:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228404352:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787829632.757270:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787829632.757284:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829632/real 1787829632] req@ffff985610f82d00 x1874669228404352/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829657 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829632.757298:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82d00 x1874669228404352/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829657 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829632.757314:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829632.757318:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787829632.757322:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228404352:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829632.820797:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829633.716919:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829633.844851:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829634.740802:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829634.868944:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829635.764832:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829635.892896:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829636.788785:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829636.916940:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829637.812852:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787829637.876762:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829637.876773:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829637.876779:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829637.876789:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5783 00000100:00080000:3.0:1787829637.876802:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829637.876810:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787829637.876852:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81e00 x1874669228404480/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829637.876878:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829637.876881:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829637.876882:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829637.876885:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5783 00000100:00080000:3.0:1787829637.876888:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829637.876890:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829637.876899:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829637.876909:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f803c0 x1874669228404608/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829637.876917:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829637.876920:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829637.876921:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829637.876924:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5783 00000100:00080000:3.0:1787829637.876926:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829637.876928:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829637.876932:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829637.876939:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228404480:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787829637.876942:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83840 x1874669228404736/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829637.877030:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787829637.877074:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829637/real 1787829637] req@ffff985610f81e00 x1874669228404480/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829662 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829637.877105:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81e00 x1874669228404480/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829662 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829637.877118:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829637.877123:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787829637.877128:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228404480:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787829637.877137:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228404608:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787829637.877164:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787829637.877190:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829637/real 1787829637] req@ffff985610f803c0 x1874669228404608/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829662 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829637.877204:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f803c0 x1874669228404608/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829662 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829637.877214:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829637.877217:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787829637.877221:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228404608:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787829637.877255:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228404736:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787829637.877289:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787829637.877304:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829637/real 1787829637] req@ffff985610f83840 x1874669228404736/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829662 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829637.877318:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83840 x1874669228404736/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829662 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829637.877327:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829637.877331:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787829637.877334:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228404736:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829637.940817:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829638.836780:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829638.964886:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829639.860810:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829639.988837:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829640.884770:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829641.012886:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829641.908824:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829642.036873:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829642.932829:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787829642.996801:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829642.996810:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829642.996813:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829642.996821:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5788 00000100:00080000:3.0:1787829642.996836:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829642.996842:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787829642.996873:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81680 x1874669228404864/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829642.996895:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829642.996898:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829642.996899:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829642.996902:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5788 00000100:00080000:3.0:1787829642.996904:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829642.996906:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829642.996913:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829642.996921:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f812c0 x1874669228404992/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829642.996928:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829642.996930:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829642.996931:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829642.996934:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5788 00000100:00080000:3.0:1787829642.996935:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829642.996938:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829642.996941:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829642.996947:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80b40 x1874669228405120/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829642.996998:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228404864:192.168.203.114@tcp:250:kworker.0 00000800:00000100:2.0:1787829642.997079:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787829642.997119:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829642/real 1787829642] req@ffff985610f81680 x1874669228404864/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829667 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829642.997135:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81680 x1874669228404864/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829667 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829642.997146:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829642.997149:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787829642.997153:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228404864:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787829642.997158:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228404992:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787829642.997173:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787829642.997181:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829642/real 1787829642] req@ffff985610f812c0 x1874669228404992/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829667 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829642.997190:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f812c0 x1874669228404992/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829667 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829642.997195:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829642.997197:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787829642.997200:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228404992:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787829642.997204:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228405120:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787829642.997216:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787829642.997227:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829642/real 1787829642] req@ffff985610f80b40 x1874669228405120/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829667 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829642.997239:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80b40 x1874669228405120/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829667 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829642.997248:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829642.997252:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787829642.997256:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228405120:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829643.060921:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829643.956781:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829644.084904:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829644.980756:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829645.108952:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829646.004775:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829646.132912:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829647.028783:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829647.156919:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829648.052806:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787829648.116813:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829648.116821:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829648.116824:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829648.116832:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5793 00000100:00080000:3.0:1787829648.116841:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829648.116847:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787829648.116879:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80000 x1874669228405248/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829648.116903:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829648.116905:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829648.116906:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829648.116909:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5793 00000100:00080000:3.0:1787829648.116912:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829648.116914:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829648.116921:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829648.116930:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81a40 x1874669228405376/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829648.116937:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829648.116941:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829648.116943:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829648.116946:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5793 00000100:00080000:3.0:1787829648.116949:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829648.116951:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829648.116954:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829648.116960:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f830c0 x1874669228405504/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829648.116981:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228405248:192.168.203.114@tcp:250:kworker.0 00000800:00000100:2.0:1787829648.117081:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787829648.117121:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829648/real 1787829648] req@ffff985610f80000 x1874669228405248/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829673 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829648.117137:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80000 x1874669228405248/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829673 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829648.117147:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829648.117150:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787829648.117154:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228405248:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787829648.117159:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228405376:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787829648.117175:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787829648.117183:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829648/real 1787829648] req@ffff985610f81a40 x1874669228405376/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829673 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829648.117191:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81a40 x1874669228405376/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829673 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829648.117196:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829648.117198:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787829648.117200:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228405376:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787829648.117226:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228405504:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787829648.117238:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787829648.117245:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829648/real 1787829648] req@ffff985610f830c0 x1874669228405504/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829673 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829648.117253:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f830c0 x1874669228405504/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829673 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829648.117258:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829648.117260:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787829648.117261:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228405504:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829648.180802:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829649.076747:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829649.204839:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829650.100843:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829650.228832:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829651.124841:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829651.252862:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829652.148792:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829652.276924:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829653.172841:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787829653.236782:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829653.236796:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829653.236801:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829653.236815:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5798 00000100:00080000:3.0:1787829653.236829:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829653.236838:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787829653.236882:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80780 x1874669228405632/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829653.236908:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829653.236911:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829653.236913:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829653.236916:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5798 00000100:00080000:3.0:1787829653.236918:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829653.236921:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829653.236929:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829653.236939:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83c00 x1874669228405760/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829653.236953:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829653.236955:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829653.236957:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829653.236961:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5798 00000100:00080000:3.0:1787829653.236964:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00100000:2.0:1787829653.236966:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228405632:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787829653.236968:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829653.236975:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829653.236984:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f830c0 x1874669228405888/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829653.237071:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787829653.237116:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829653/real 1787829653] req@ffff985610f80780 x1874669228405632/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829678 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829653.237139:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80780 x1874669228405632/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829678 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829653.237161:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829653.237167:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787829653.237172:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228405632:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787829653.237205:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228405760:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787829653.237233:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787829653.237249:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829653/real 1787829653] req@ffff985610f83c00 x1874669228405760/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829678 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829653.237263:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83c00 x1874669228405760/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829678 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829653.237273:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829653.237276:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787829653.237288:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228405760:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787829653.237296:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228405888:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787829653.237319:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787829653.237333:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829653/real 1787829653] req@ffff985610f830c0 x1874669228405888/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829678 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829653.237346:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f830c0 x1874669228405888/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829678 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829653.237355:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829653.237359:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787829653.237363:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228405888:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829653.300856:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829654.196781:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787829654.324801:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829655.220826:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829655.348855:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829656.244788:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829656.372845:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829657.268844:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829657.396971:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829658.292742:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787829658.356802:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829658.356810:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829658.356813:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829658.356824:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5803 00000100:00080000:3.0:1787829658.356834:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829658.356840:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787829658.356873:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81a40 x1874669228406016/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829658.356895:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829658.356897:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829658.356899:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829658.356901:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5803 00000100:00080000:3.0:1787829658.356904:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829658.356907:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829658.356915:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829658.356924:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80000 x1874669228406144/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829658.356933:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1787829658.356940:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228406016:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787829658.356941:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829658.356946:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829658.356949:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5803 00000100:00080000:3.0:1787829658.356958:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829658.356962:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829658.356965:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829658.356973:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80b40 x1874669228406272/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829658.357051:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787829658.357093:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829658/real 1787829658] req@ffff985610f81a40 x1874669228406016/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829683 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829658.357109:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81a40 x1874669228406016/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829683 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829658.357119:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829658.357122:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787829658.357125:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228406016:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787829658.357150:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228406144:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787829658.357175:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787829658.357190:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829658/real 1787829658] req@ffff985610f80000 x1874669228406144/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829683 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829658.357201:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80000 x1874669228406144/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829683 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829658.357209:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829658.357213:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787829658.357216:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228406144:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787829658.357222:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228406272:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787829658.357243:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787829658.357255:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829658/real 1787829658] req@ffff985610f80b40 x1874669228406272/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829683 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829658.357267:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80b40 x1874669228406272/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829683 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829658.357276:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829658.357280:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787829658.357283:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228406272:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829658.420890:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829659.316815:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787829659.444834:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829660.340790:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829660.468829:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829661.364777:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829661.492875:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829662.388792:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829662.516920:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829663.412820:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787829663.476787:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829663.476802:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829663.476809:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829663.476832:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5809 00000100:00080000:3.0:1787829663.476849:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829663.476860:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787829663.476910:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f812c0 x1874669228406400/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829663.476957:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829663.476962:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829663.476965:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829663.476971:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5809 00000100:00080000:3.0:1787829663.476978:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829663.476984:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829663.476998:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829663.477025:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81680 x1874669228406528/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787829663.477026:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228406400:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787829663.477047:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829663.477052:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829663.477055:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829663.477061:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5809 00000800:00000100:0.0:1787829663.477118:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00080000:3.0:1787829663.477150:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00100000:0.0:1787829663.477153:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829663/real 1787829663] req@ffff985610f812c0 x1874669228406400/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829688 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829663.477161:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829663.477171:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829663.477177:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f812c0 x1874669228406400/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829688 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829663.477191:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83840 x1874669228406656/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829663.477193:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829663.477204:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829663.477213:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228406400:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829663.477246:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228406528:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829663.477271:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829663.477282:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829663/real 1787829663] req@ffff985610f81680 x1874669228406528/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829688 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829663.477293:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81680 x1874669228406528/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829688 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829663.477304:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829663.477308:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829663.477312:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228406528:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829663.477319:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228406656:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829663.477344:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829663.477357:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829663/real 1787829663] req@ffff985610f83840 x1874669228406656/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829688 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829663.477368:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83840 x1874669228406656/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829688 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829663.477376:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829663.477380:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829663.477383:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228406656:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829663.540836:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829664.436764:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787829664.564883:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829665.460820:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829665.588872:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829666.484790:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829666.612845:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829667.508837:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829667.636858:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829668.532768:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787829668.596787:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829668.596798:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829668.596802:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829668.596813:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5814 00000100:00080000:3.0:1787829668.596826:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829668.596841:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787829668.596887:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f803c0 x1874669228406784/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829668.596911:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829668.596915:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829668.596917:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829668.596919:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5814 00000100:00080000:3.0:1787829668.596922:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829668.596925:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829668.596935:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829668.596945:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81e00 x1874669228406912/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787829668.596954:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228406784:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787829668.596955:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829668.596960:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829668.596963:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829668.596969:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5814 00000100:00080000:3.0:1787829668.596971:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829668.596974:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829668.596977:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829668.596985:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82d00 x1874669228407040/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787829668.597042:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829668.597079:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829668/real 1787829668] req@ffff985610f803c0 x1874669228406784/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829693 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829668.597094:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f803c0 x1874669228406784/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829693 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829668.597103:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829668.597106:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829668.597109:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228406784:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829668.597133:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228406912:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829668.597151:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829668.597160:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829668/real 1787829668] req@ffff985610f81e00 x1874669228406912/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829693 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829668.597169:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81e00 x1874669228406912/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829693 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829668.597175:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829668.597177:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829668.597179:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228406912:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829668.597183:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228407040:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829668.597193:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829668.597198:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829668/real 1787829668] req@ffff985610f82d00 x1874669228407040/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829693 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829668.597210:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82d00 x1874669228407040/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829693 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829668.597216:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829668.597217:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829668.597219:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228407040:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829668.660863:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829669.556786:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787829669.684834:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829670.580895:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829670.708862:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829671.604829:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829671.732885:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829672.628876:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829672.756871:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829673.652782:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787829673.716844:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829673.716855:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829673.716860:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829673.716872:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5819 00000100:00080000:3.0:1787829673.716886:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829673.716895:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787829673.716962:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80f00 x1874669228407168/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829673.716990:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829673.716993:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829673.716995:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829673.716998:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5819 00000100:00080000:3.0:1787829673.717000:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829673.717003:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829673.717013:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829673.717024:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82940 x1874669228407296/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829673.717032:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829673.717035:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829673.717037:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829673.717042:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5819 00000100:00100000:0.0:1787829673.717045:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228407168:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787829673.717046:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829673.717052:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829673.717058:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829673.717064:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82580 x1874669228407424/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787829673.717157:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228407296:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829673.717186:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228407424:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829673.780869:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829674.676823:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787829674.804876:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829675.700780:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829675.828873:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829676.724816:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000400:00000100:1.0:1787829676.788992:0:2372:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.114:988 00000400:00000100:1.0:1787829676.789055:0:2372:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.114@tcp at host 192.168.203.114 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1787829676.789077:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:1.0:1787829676.789124:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:1.0:1787829676.789135:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000100:00100000:0.0:1787829676.789182:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829673/real 1787829676] req@ffff985610f82940 x1874669228407296/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829698 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829676.789213:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82940 x1874669228407296/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829698 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829676.789228:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829676.789234:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829676.789239:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228407296:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829676.789248:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829673/real 1787829676] req@ffff985610f82580 x1874669228407424/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829698 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829676.789261:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82580 x1874669228407424/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829698 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829676.789269:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829676.789273:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829676.789276:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228407424:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829676.789283:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829673/real 1787829676] req@ffff985610f80f00 x1874669228407168/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829698 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829676.789296:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80f00 x1874669228407168/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829698 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829676.789304:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829676.789307:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829676.789310:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228407168:192.168.203.114@tcp:250:kworker.0 00010000:00010000:0.0:1787829676.852854:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829677.748784:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829677.876893:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829678.772824:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787829678.836822:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829678.836832:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829678.836836:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829678.836847:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5824 00000100:00080000:2.0:1787829678.836858:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829678.836866:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787829678.836906:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140da940 x1874669228407552/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829678.836947:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829678.836951:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829678.836953:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829678.836958:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5824 00000100:00080000:2.0:1787829678.836962:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829678.836968:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787829678.836980:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829678.836997:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8b40 x1874669228407680/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829678.837011:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1787829678.837013:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228407552:192.168.203.114@tcp:250:kworker.0 00000100:00080000:2.0:1787829678.837016:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829678.837020:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829678.837027:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5824 00000100:00080000:2.0:1787829678.837031:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829678.837036:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787829678.837043:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829678.837055:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d9680 x1874669228407808/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787829678.837119:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829678.837165:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829678/real 1787829678] req@ffff9856140da940 x1874669228407552/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829703 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829678.837187:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140da940 x1874669228407552/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829703 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829678.837199:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829678.837204:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829678.837209:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228407552:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829678.837240:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228407680:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829678.837269:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829678.837285:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829678/real 1787829678] req@ffff9856140d8b40 x1874669228407680/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829703 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829678.837299:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8b40 x1874669228407680/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829703 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829678.837307:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829678.837311:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829678.837315:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228407680:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829678.837321:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228407808:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829678.837345:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829678.837358:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829678/real 1787829678] req@ffff9856140d9680 x1874669228407808/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829703 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829678.837386:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d9680 x1874669228407808/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829703 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829678.837395:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829678.837400:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829678.837403:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228407808:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829678.900805:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829679.796833:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787829679.924860:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829680.820785:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829680.948838:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829681.844769:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829681.972888:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829682.868808:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829682.996854:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829683.892763:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787829683.956764:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829683.956772:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829683.956775:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829683.956783:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5829 00000100:00080000:2.0:1787829683.956793:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829683.956799:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787829683.956830:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d9a40 x1874669228407936/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829683.956852:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829683.956855:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829683.956856:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829683.956859:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5829 00000100:00080000:2.0:1787829683.956861:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829683.956863:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787829683.956871:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829683.956880:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db480 x1874669228408064/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829683.956889:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829683.956891:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829683.956892:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829683.956897:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5829 00000100:00100000:0.0:1787829683.956900:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228407936:192.168.203.114@tcp:250:kworker.0 00000100:00080000:2.0:1787829683.956900:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829683.956906:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787829683.956909:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829683.956915:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d83c0 x1874669228408192/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787829683.956965:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829683.956994:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829683/real 1787829683] req@ffff9856140d9a40 x1874669228407936/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829708 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829683.957007:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d9a40 x1874669228407936/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829708 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829683.957014:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829683.957017:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829683.957020:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228407936:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829683.957039:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228408064:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829683.957054:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829683.957062:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829683/real 1787829683] req@ffff9856140db480 x1874669228408064/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829708 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829683.957078:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db480 x1874669228408064/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829708 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829683.957084:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829683.957086:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829683.957088:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228408064:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829683.957092:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228408192:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829683.957104:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829683.957110:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829683/real 1787829683] req@ffff9856140d83c0 x1874669228408192/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829708 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829683.957118:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d83c0 x1874669228408192/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829708 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829683.957123:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829683.957124:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829683.957126:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228408192:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829684.020812:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829684.916789:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787829685.044809:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829685.940792:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829686.068828:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829686.964772:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829687.092798:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829687.988755:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829688.116854:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829689.012779:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787829689.076814:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829689.076822:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829689.076825:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829689.076834:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5834 00000100:00080000:3.0:1787829689.076844:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829689.076850:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787829689.076879:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83480 x1874669228408320/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829689.076906:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829689.076909:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829689.076911:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829689.076915:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5834 00000100:00080000:3.0:1787829689.076919:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829689.076923:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829689.076933:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829689.076947:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f821c0 x1874669228408448/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829689.076958:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1787829689.076960:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228408320:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787829689.076961:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829689.076964:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829689.076971:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5834 00000100:00080000:3.0:1787829689.076974:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829689.076977:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829689.076980:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829689.076992:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80f00 x1874669228408576/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787829689.077035:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829689.077072:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829689/real 1787829689] req@ffff985610f83480 x1874669228408320/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829714 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829689.077106:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83480 x1874669228408320/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829714 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829689.077119:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829689.077123:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829689.077127:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228408320:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829689.077150:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f821c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228408448:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829689.077170:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829689.077180:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829689/real 1787829689] req@ffff985610f821c0 x1874669228408448/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829714 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829689.077193:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f821c0 x1874669228408448/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829714 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829689.077199:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829689.077201:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829689.077203:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f821c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228408448:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829689.077207:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228408576:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829689.077218:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829689.077224:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829689/real 1787829689] req@ffff985610f80f00 x1874669228408576/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829714 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829689.077231:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80f00 x1874669228408576/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829714 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829689.077236:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829689.077238:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829689.077240:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228408576:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829689.140804:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829690.036818:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787829690.164917:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829691.060771:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829691.188933:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829692.084778:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829692.212801:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829693.108785:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829693.236952:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829694.132804:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787829694.196821:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829694.196839:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829694.196846:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829694.196857:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5839 00000100:00080000:3.0:1787829694.196869:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829694.196878:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787829694.196919:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82580 x1874669228408704/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829694.196947:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829694.196950:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829694.196952:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829694.196955:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5839 00000100:00080000:3.0:1787829694.196957:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829694.196960:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829694.196970:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829694.196979:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82940 x1874669228408832/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829694.196987:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829694.196990:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829694.196991:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829694.196995:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5839 00000100:00080000:3.0:1787829694.196997:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829694.196999:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829694.197002:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829694.197009:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82d00 x1874669228408960/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787829694.197018:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228408704:192.168.203.114@tcp:250:kworker.0 00000800:00000100:0.0:1787829694.197108:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829694.197154:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829694/real 1787829694] req@ffff985610f82580 x1874669228408704/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829719 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829694.197178:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82580 x1874669228408704/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829719 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829694.197187:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829694.197190:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829694.197193:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228408704:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829694.197200:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228408832:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829694.197216:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829694.197225:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829694/real 1787829694] req@ffff985610f82940 x1874669228408832/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829719 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829694.197233:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82940 x1874669228408832/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829719 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829694.197239:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829694.197241:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829694.197243:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228408832:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829694.197280:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228408960:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829694.197294:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829694.197302:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829694/real 1787829694] req@ffff985610f82d00 x1874669228408960/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829719 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829694.197310:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82d00 x1874669228408960/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829719 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829694.197320:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829694.197322:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829694.197325:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228408960:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829694.260816:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829695.156757:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787829695.284923:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829696.180799:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829696.308819:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829697.204804:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829697.332878:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829698.228824:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829698.356901:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829699.252822:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787829699.316792:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829699.316807:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829699.316812:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829699.316825:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5844 00000100:00080000:3.0:1787829699.316849:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829699.316857:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787829699.316899:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81e00 x1874669228409088/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829699.316918:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829699.316921:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829699.316923:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829699.316927:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5844 00000100:00080000:3.0:1787829699.316941:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829699.316944:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829699.316953:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829699.316957:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228409088:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787829699.316967:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f803c0 x1874669228409216/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829699.316976:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829699.316979:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829699.316980:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829699.316983:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5844 00000100:00080000:3.0:1787829699.316986:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829699.316989:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829699.316994:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829699.317015:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83840 x1874669228409344/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787829699.317050:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829699.317108:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829699/real 1787829699] req@ffff985610f81e00 x1874669228409088/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829724 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829699.317139:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81e00 x1874669228409088/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829724 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829699.317149:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829699.317154:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829699.317159:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228409088:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829699.317193:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228409216:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829699.317246:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829699.317258:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829699/real 1787829699] req@ffff985610f803c0 x1874669228409216/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829724 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829699.317268:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f803c0 x1874669228409216/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829724 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829699.317279:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829699.317282:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829699.317285:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228409216:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829699.317289:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228409344:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829699.317305:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829699.317312:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829699/real 1787829699] req@ffff985610f83840 x1874669228409344/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829724 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829699.317319:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83840 x1874669228409344/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829724 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829699.317325:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829699.317327:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829699.317329:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228409344:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829699.380839:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829700.276817:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787829700.404866:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829701.300798:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829701.428895:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829702.324804:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829702.452829:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829703.348770:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829703.477011:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829704.372768:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787829704.436764:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829704.436795:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829704.436800:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829704.436825:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5850 00000100:00080000:3.0:1787829704.436840:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829704.436851:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787829704.436899:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81680 x1874669228409472/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829704.436935:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829704.436940:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829704.436943:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829704.436949:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5850 00000100:00080000:3.0:1787829704.436954:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829704.436960:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829704.436988:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829704.436995:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228409472:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787829704.437006:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f812c0 x1874669228409600/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829704.437021:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829704.437026:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829704.437045:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829704.437051:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5850 00000100:00080000:3.0:1787829704.437057:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829704.437064:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829704.437070:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829704.437084:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80b40 x1874669228409728/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787829704.437093:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829704.437134:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829704/real 1787829704] req@ffff985610f81680 x1874669228409472/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829729 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829704.437151:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81680 x1874669228409472/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829729 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829704.437166:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829704.437170:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829704.437173:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228409472:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829704.437214:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228409600:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829704.437237:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829704.437248:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829704/real 1787829704] req@ffff985610f812c0 x1874669228409600/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829729 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829704.437258:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f812c0 x1874669228409600/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829729 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829704.437265:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829704.437267:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829704.437270:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228409600:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829704.437274:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228409728:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829704.437287:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829704.437294:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829704/real 1787829704] req@ffff985610f80b40 x1874669228409728/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829729 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829704.437302:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80b40 x1874669228409728/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829729 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829704.437308:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829704.437310:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829704.437312:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228409728:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829704.500818:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829705.396754:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787829705.524931:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829706.420786:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829706.548813:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829707.444831:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829707.572812:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829708.468809:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829708.596857:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829709.492783:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787829709.556841:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829709.556851:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829709.556854:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829709.556865:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5855 00000100:00080000:2.0:1787829709.556877:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829709.556884:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787829709.556928:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8780 x1874669228409856/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829709.556956:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829709.556959:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829709.556960:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829709.556963:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5855 00000100:00080000:2.0:1787829709.556965:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829709.556968:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787829709.556977:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829709.556989:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d83c0 x1874669228409984/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829709.556997:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829709.556999:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829709.557001:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829709.557004:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5855 00000100:00080000:2.0:1787829709.557007:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829709.557009:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787829709.557012:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829709.557019:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db480 x1874669228410112/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787829709.557030:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228409856:192.168.203.114@tcp:250:kworker.0 00000800:00000100:0.0:1787829709.557116:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829709.557169:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829709/real 1787829709] req@ffff9856140d8780 x1874669228409856/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829734 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829709.557186:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8780 x1874669228409856/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829734 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829709.557195:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829709.557198:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829709.557201:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228409856:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829709.557207:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228409984:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829709.557225:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829709.557234:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829709/real 1787829709] req@ffff9856140d83c0 x1874669228409984/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829734 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829709.557242:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d83c0 x1874669228409984/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829734 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829709.557248:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829709.557250:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829709.557252:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228409984:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829709.557278:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228410112:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829709.557301:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829709.557314:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829709/real 1787829709] req@ffff9856140db480 x1874669228410112/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829734 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829709.557327:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db480 x1874669228410112/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829734 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829709.557337:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829709.557340:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829709.557344:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228410112:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829709.620863:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829710.516781:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829710.644861:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829711.540818:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829711.668906:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829712.564808:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829712.692918:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829713.588816:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829713.716906:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829714.612763:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787829714.676851:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829714.676863:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829714.676868:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829714.676880:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5860 00000100:00080000:3.0:1787829714.676895:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829714.676904:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787829714.676966:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80000 x1874669228410240/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829714.676999:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829714.677004:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829714.677006:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829714.677011:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5860 00000100:00080000:3.0:1787829714.677016:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829714.677022:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829714.677034:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829714.677045:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228410240:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787829714.677053:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81a40 x1874669228410368/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829714.677068:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829714.677073:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829714.677075:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829714.677080:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5860 00000100:00080000:3.0:1787829714.677085:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829714.677090:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829714.677096:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829714.677132:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f830c0 x1874669228410496/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787829714.677134:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829714.677176:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829714/real 1787829714] req@ffff985610f80000 x1874669228410240/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829739 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829714.677192:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80000 x1874669228410240/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829739 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829714.677201:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829714.677204:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829714.677208:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228410240:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829714.677235:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228410368:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829714.677261:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829714.677277:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829714/real 1787829714] req@ffff985610f81a40 x1874669228410368/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829739 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829714.677288:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81a40 x1874669228410368/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829739 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829714.677294:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829714.677297:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829714.677299:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228410368:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829714.677304:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228410496:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829714.677323:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829714.677335:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829714/real 1787829714] req@ffff985610f830c0 x1874669228410496/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829739 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829714.677348:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f830c0 x1874669228410496/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829739 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829714.677358:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829714.677361:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829714.677365:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228410496:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829714.740893:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829715.636792:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829715.764852:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829716.660752:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829716.788884:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829717.684831:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829717.812804:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829718.708820:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829718.836925:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829719.732823:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787829719.796811:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829719.796821:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829719.796826:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829719.796836:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5865 00000100:00080000:3.0:1787829719.796850:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829719.796857:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787829719.796905:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83c00 x1874669228410624/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829719.796940:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829719.796945:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829719.796948:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829719.796952:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5865 00000100:00080000:3.0:1787829719.796957:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829719.796962:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829719.796974:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829719.796988:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80780 x1874669228410752/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829719.796997:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829719.797000:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829719.797001:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829719.797004:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5865 00000100:00080000:3.0:1787829719.797007:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829719.797011:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829719.797015:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829719.797022:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228410624:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787829719.797031:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f830c0 x1874669228410880/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787829719.797108:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829719.797142:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829719/real 1787829719] req@ffff985610f83c00 x1874669228410624/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829744 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829719.797156:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83c00 x1874669228410624/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829744 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829719.797165:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829719.797169:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829719.797172:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228410624:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829719.797178:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228410752:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829719.797193:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829719.797201:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829719/real 1787829719] req@ffff985610f80780 x1874669228410752/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829744 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829719.797208:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80780 x1874669228410752/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829744 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829719.797214:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829719.797216:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829719.797218:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228410752:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829719.797237:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228410880:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829719.797249:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829719.797255:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829719/real 1787829719] req@ffff985610f830c0 x1874669228410880/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829744 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829719.797263:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f830c0 x1874669228410880/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829744 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829719.797272:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829719.797274:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829719.797276:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228410880:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829719.860813:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829720.756804:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829720.884925:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829721.780842:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829721.908850:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829722.804816:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829722.932855:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829723.828820:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829723.956927:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829724.852787:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787829724.916802:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829724.916810:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829724.916814:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829724.916822:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5870 00000100:00080000:3.0:1787829724.916831:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829724.916836:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787829724.916866:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81a40 x1874669228411008/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829724.916889:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829724.916891:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829724.916893:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829724.916895:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5870 00000100:00080000:3.0:1787829724.916898:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829724.916901:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829724.916908:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829724.916918:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80000 x1874669228411136/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829724.916925:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829724.916928:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829724.916929:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829724.916932:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5870 00000100:00080000:3.0:1787829724.916934:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829724.916937:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829724.916940:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829724.916947:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80b40 x1874669228411264/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787829724.916965:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228411008:192.168.203.114@tcp:250:kworker.0 00000800:00000100:0.0:1787829724.917071:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829724.917120:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829724/real 1787829724] req@ffff985610f81a40 x1874669228411008/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829749 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829724.917142:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81a40 x1874669228411008/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829749 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829724.917156:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829724.917161:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829724.917166:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228411008:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829724.917175:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228411136:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829724.917201:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829724.917215:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829724/real 1787829724] req@ffff985610f80000 x1874669228411136/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829749 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829724.917229:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80000 x1874669228411136/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829749 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829724.917244:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829724.917248:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829724.917252:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228411136:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829724.917282:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228411264:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829724.917304:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829724.917315:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829724/real 1787829724] req@ffff985610f80b40 x1874669228411264/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829749 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829724.917328:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80b40 x1874669228411264/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829749 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829724.917338:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829724.917342:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829724.917345:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228411264:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829724.980880:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829725.876801:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829726.004889:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829726.900818:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829727.028842:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829727.924770:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829728.052808:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829728.948837:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829729.076883:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829729.972782:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787829730.036825:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829730.036837:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829730.036841:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829730.036852:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5875 00000100:00080000:3.0:1787829730.036864:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829730.036871:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787829730.036915:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f812c0 x1874669228411392/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829730.036942:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829730.036946:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829730.036947:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829730.036950:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5875 00000100:00080000:3.0:1787829730.036953:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829730.036955:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829730.036964:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829730.036979:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81680 x1874669228411520/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829730.036991:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1787829730.036994:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228411392:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787829730.036996:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829730.037001:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829730.037021:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5875 00000100:00080000:3.0:1787829730.037026:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829730.037031:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829730.037037:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829730.037052:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83840 x1874669228411648/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787829730.037096:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829730.037138:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829730/real 1787829730] req@ffff985610f812c0 x1874669228411392/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829755 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829730.037153:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f812c0 x1874669228411392/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829755 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829730.037167:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829730.037171:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829730.037174:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228411392:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829730.037198:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228411520:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829730.037216:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829730.037224:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829730/real 1787829730] req@ffff985610f81680 x1874669228411520/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829755 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829730.037232:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81680 x1874669228411520/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829755 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829730.037238:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829730.037240:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829730.037242:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228411520:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829730.037246:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228411648:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829730.037256:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829730.037261:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829730/real 1787829730] req@ffff985610f83840 x1874669228411648/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829755 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829730.037269:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83840 x1874669228411648/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829755 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829730.037274:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829730.037276:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829730.037278:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228411648:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829730.100901:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829730.996817:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829731.124899:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829732.020815:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829732.148900:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829733.044818:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829733.172849:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829734.068779:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829734.196941:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829735.092807:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787829735.156810:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829735.156820:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829735.156824:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829735.156835:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5880 00000100:00080000:3.0:1787829735.156847:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829735.156857:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787829735.156905:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f803c0 x1874669228411776/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829735.156932:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829735.156937:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829735.156940:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829735.156945:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5880 00000100:00080000:3.0:1787829735.156950:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829735.156955:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787829735.156965:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228411776:192.168.203.114@tcp:250:kworker.0 00000008:00100000:3.0:1787829735.156968:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829735.157004:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81e00 x1874669228411904/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829735.157018:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829735.157023:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829735.157026:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829735.157030:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5880 00000100:00080000:3.0:1787829735.157035:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829735.157041:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829735.157047:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000800:00000100:0.0:1787829735.157074:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829735.157118:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829735/real 1787829735] req@ffff985610f803c0 x1874669228411776/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829760 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829735.157143:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82d00 x1874669228412032/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787829735.157144:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f803c0 x1874669228411776/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829760 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829735.157176:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829735.157183:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829735.157189:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228411776:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829735.157220:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228411904:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829735.157264:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829735.157280:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829735/real 1787829735] req@ffff985610f81e00 x1874669228411904/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829760 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829735.157295:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81e00 x1874669228411904/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829760 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829735.157305:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829735.157310:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829735.157314:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228411904:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829735.157322:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228412032:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829735.157351:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829735.157367:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829735/real 1787829735] req@ffff985610f82d00 x1874669228412032/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829760 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829735.157383:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82d00 x1874669228412032/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829760 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829735.157393:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829735.157398:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829735.157402:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228412032:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829735.220820:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829736.116782:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829736.244954:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829737.140811:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829737.268831:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829738.164798:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829738.292839:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829739.188820:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829739.316853:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829740.212781:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787829740.276822:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829740.276836:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829740.276842:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829740.276855:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5885 00000100:00080000:2.0:1787829740.276875:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829740.276884:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787829740.276934:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d9a40 x1874669228412160/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829740.276996:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829740.277001:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829740.277003:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829740.277008:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5885 00000100:00080000:2.0:1787829740.277013:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829740.277019:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787829740.277031:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829740.277047:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228412160:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787829740.277050:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d9680 x1874669228412288/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829740.277067:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829740.277071:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829740.277073:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829740.277078:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5885 00000100:00080000:2.0:1787829740.277083:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829740.277087:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787829740.277093:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829740.277108:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8b40 x1874669228412416/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787829740.277169:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228412288:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829740.277219:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228412416:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829740.340812:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829741.236821:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829741.364857:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829742.260775:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829742.388848:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829743.284813:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000400:00000100:2.0:1787829743.349053:0:2373:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.114:988 00000400:00000100:2.0:1787829743.349106:0:2373:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.114@tcp at host 192.168.203.114 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:2.0:1787829743.349129:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:2.0:1787829743.349181:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:2.0:1787829743.349192:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000100:00100000:0.0:1787829743.349349:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829740/real 1787829743] req@ffff9856140d9680 x1874669228412288/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829765 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829743.349382:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d9680 x1874669228412288/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829765 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829743.349397:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829743.349414:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829743.349421:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228412288:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829743.349431:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829740/real 1787829743] req@ffff9856140d8b40 x1874669228412416/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829765 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829743.349446:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8b40 x1874669228412416/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829765 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829743.349456:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829743.349460:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829743.349464:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228412416:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829743.349473:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829740/real 1787829743] req@ffff9856140d9a40 x1874669228412160/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829765 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829743.349487:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d9a40 x1874669228412160/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829765 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829743.349495:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829743.349499:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829743.349502:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228412160:192.168.203.114@tcp:250:kworker.0 00010000:00010000:0.0:1787829743.412807:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829744.308828:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829744.436970:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829745.332834:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787829745.396839:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829745.396850:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829745.396854:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829745.396866:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5891 00000100:00080000:0.0:1787829745.396878:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829745.396886:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787829745.396941:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6940 x1874669228412544/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829745.396972:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829745.396976:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829745.396978:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829745.396983:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5891 00000100:00080000:0.0:1787829745.396988:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829745.396993:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787829745.397006:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829745.397015:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228412544:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829745.397024:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4b40 x1874669228412672/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829745.397044:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829745.397049:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829745.397051:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829745.397055:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5891 00000100:00080000:0.0:1787829745.397060:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829745.397065:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787829745.397071:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829745.397085:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7480 x1874669228412800/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829745.397114:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787829745.397164:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829745/real 1787829745] req@ffff985607df6940 x1874669228412544/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829770 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829745.397192:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6940 x1874669228412544/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829770 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829745.397203:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829745.397209:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787829745.397214:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228412544:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787829745.397247:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228412672:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787829745.397275:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787829745.397291:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829745/real 1787829745] req@ffff985607df4b40 x1874669228412672/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829770 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829745.397305:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4b40 x1874669228412672/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829770 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829745.397315:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829745.397318:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787829745.397322:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228412672:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787829745.397329:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228412800:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787829745.397352:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787829745.397365:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829745/real 1787829745] req@ffff985607df7480 x1874669228412800/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829770 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829745.397378:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7480 x1874669228412800/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829770 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829745.397394:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829745.397398:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787829745.397401:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228412800:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829745.460817:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829746.356772:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829746.485002:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829747.380817:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829747.508801:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829748.404817:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829748.532846:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829749.428791:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829749.557003:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829750.452836:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787829750.516954:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829750.516976:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829750.516981:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829750.517010:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5896 00000100:00080000:2.0:1787829750.517026:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829750.517036:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787829750.517090:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140da940 x1874669228412928/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829750.517139:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829750.517151:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829750.517153:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829750.517162:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5896 00000100:00080000:2.0:1787829750.517166:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829750.517170:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787829750.517182:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829750.517196:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db0c0 x1874669228413056/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829750.517209:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829750.517213:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829750.517216:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829750.517220:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5896 00000100:00080000:2.0:1787829750.517223:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829750.517227:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787829750.517232:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829750.517260:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228412928:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787829750.517260:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140dad00 x1874669228413184/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787829750.517347:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829750.517387:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829750/real 1787829750] req@ffff9856140da940 x1874669228412928/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829775 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829750.517404:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140da940 x1874669228412928/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829775 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829750.517414:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829750.517418:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829750.517422:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228412928:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829750.517428:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228413056:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829750.517445:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829750.517455:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829750/real 1787829750] req@ffff9856140db0c0 x1874669228413056/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829775 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829750.517464:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db0c0 x1874669228413056/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829775 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829750.517470:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829750.517472:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829750.517475:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228413056:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829750.517534:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228413184:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829750.517571:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829750.517601:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829750/real 1787829750] req@ffff9856140dad00 x1874669228413184/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829775 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829750.517634:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140dad00 x1874669228413184/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829775 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829750.517640:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829750.517643:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829750.517645:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228413184:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829750.580914:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829751.476797:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829751.605020:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829752.500791:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829752.628813:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829753.524782:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829753.652879:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829754.548860:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829754.676861:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829755.572851:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787829755.636815:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829755.636826:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829755.636847:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829755.636859:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5901 00000100:00080000:2.0:1787829755.636871:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829755.636879:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787829755.636923:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8f00 x1874669228413312/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829755.636951:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829755.636954:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829755.636957:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829755.636960:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5901 00000100:00080000:2.0:1787829755.636963:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829755.636966:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787829755.636975:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829755.636985:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140da1c0 x1874669228413440/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829755.636993:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829755.636996:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829755.636997:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829755.637000:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5901 00000100:00080000:2.0:1787829755.637002:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829755.637005:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787829755.637008:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829755.637012:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228413312:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787829755.637018:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db840 x1874669228413568/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787829755.637103:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829755.637153:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829755/real 1787829755] req@ffff9856140d8f00 x1874669228413312/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829780 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829755.637173:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8f00 x1874669228413312/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829780 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829755.637185:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829755.637189:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829755.637193:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228413312:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829755.637201:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228413440:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829755.637226:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829755.637240:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829755/real 1787829755] req@ffff9856140da1c0 x1874669228413440/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829780 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829755.637258:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140da1c0 x1874669228413440/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829780 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829755.637266:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829755.637270:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829755.637273:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228413440:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829755.637307:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228413568:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829755.637324:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829755.637333:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829755/real 1787829755] req@ffff9856140db840 x1874669228413568/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829780 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829755.637341:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db840 x1874669228413568/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829780 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829755.637347:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829755.637349:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829755.637351:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228413568:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829755.700882:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829756.596850:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829756.724962:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829757.620778:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829757.748942:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829758.644802:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829758.772799:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829759.668782:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787829759.796923:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829760.692792:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787829760.756778:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829760.756789:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829760.756806:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829760.756817:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5906 00000100:00080000:2.0:1787829760.756829:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829760.756837:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787829760.756881:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140dbc00 x1874669228413696/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829760.756903:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829760.756906:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829760.756907:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829760.756910:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5906 00000100:00080000:2.0:1787829760.756913:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829760.756915:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787829760.756923:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829760.756933:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db840 x1874669228413824/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829760.756941:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829760.756944:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829760.756945:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829760.756948:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5906 00000100:00080000:2.0:1787829760.756950:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829760.756952:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787829760.756955:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829760.756961:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228413696:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787829760.756964:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140da1c0 x1874669228413952/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787829760.757029:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829760.757065:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829760/real 1787829760] req@ffff9856140dbc00 x1874669228413696/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829785 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829760.757092:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140dbc00 x1874669228413696/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829785 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829760.757103:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829760.757107:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829760.757110:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228413696:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829760.757116:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228413824:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829760.757134:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829760.757143:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829760/real 1787829760] req@ffff9856140db840 x1874669228413824/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829785 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829760.757151:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db840 x1874669228413824/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829785 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829760.757157:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829760.757159:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829760.757161:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228413824:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829760.757178:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228413952:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829760.757189:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829760.757195:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829760/real 1787829760] req@ffff9856140da1c0 x1874669228413952/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829785 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829760.757203:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140da1c0 x1874669228413952/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829785 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829760.757212:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829760.757214:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829760.757216:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228413952:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829760.820827:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829761.716762:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829761.844850:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829762.740807:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829762.868854:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829763.764801:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829763.892884:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829764.788804:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787829764.916892:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829765.812827:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787829765.876803:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829765.876816:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829765.876821:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829765.876834:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5911 00000100:00080000:1.0:1787829765.876848:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829765.876857:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787829765.876903:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a1680 x1874669228414080/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829765.876932:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829765.876936:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829765.876938:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829765.876942:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5911 00000100:00080000:1.0:1787829765.876946:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829765.876950:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787829765.876962:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829765.876977:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0780 x1874669228414208/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787829765.876979:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228414080:192.168.203.114@tcp:250:kworker.0 00000100:00080000:1.0:1787829765.876996:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829765.877000:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829765.877002:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829765.877007:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5911 00000100:00080000:1.0:1787829765.877011:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829765.877015:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787829765.877019:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829765.877031:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3c00 x1874669228414336/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787829765.877074:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829765.877109:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829765/real 1787829765] req@ffff9856085a1680 x1874669228414080/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829790 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829765.877125:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a1680 x1874669228414080/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829790 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829765.877133:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829765.877137:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829765.877140:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228414080:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829765.877164:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228414208:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829765.877181:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829765.877194:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829765/real 1787829765] req@ffff9856085a0780 x1874669228414208/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829790 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829765.877207:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0780 x1874669228414208/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829790 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829765.877223:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829765.877227:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829765.877231:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228414208:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829765.877238:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228414336:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829765.877258:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829765.877266:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829765/real 1787829765] req@ffff9856085a3c00 x1874669228414336/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829790 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829765.877274:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3c00 x1874669228414336/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829790 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829765.877280:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829765.877283:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829765.877285:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228414336:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829765.940871:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829766.836830:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829766.965004:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829767.860832:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829767.988857:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829768.884801:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829769.012901:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829769.908802:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787829770.036898:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829770.932829:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787829770.996875:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829770.996884:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829770.996889:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829770.996902:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5916 00000100:00080000:3.0:1787829770.996915:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829770.996924:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787829770.996964:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82940 x1874669228414464/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829770.996989:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829770.996992:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829770.996993:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829770.996997:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5916 00000100:00080000:3.0:1787829770.997000:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829770.997003:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829770.997013:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829770.997026:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82580 x1874669228414592/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787829770.997033:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228414464:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787829770.997037:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829770.997042:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829770.997045:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829770.997052:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5916 00000100:00080000:3.0:1787829770.997055:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829770.997059:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829770.997062:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829770.997070:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80f00 x1874669228414720/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787829770.997116:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829770.997149:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829770/real 1787829770] req@ffff985610f82940 x1874669228414464/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829795 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829770.997165:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82940 x1874669228414464/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829795 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829770.997179:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829770.997182:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829770.997186:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228414464:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829770.997210:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228414592:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829770.997228:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829770.997242:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829770/real 1787829770] req@ffff985610f82580 x1874669228414592/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829795 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829770.997258:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82580 x1874669228414592/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829795 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829770.997264:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829770.997267:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829770.997270:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228414592:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829770.997274:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228414720:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829770.997290:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829770.997299:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829770/real 1787829770] req@ffff985610f80f00 x1874669228414720/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829795 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829770.997308:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80f00 x1874669228414720/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829795 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829770.997314:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829770.997317:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829770.997320:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228414720:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829771.060917:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829771.956853:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829772.084841:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829772.980865:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829773.108924:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829774.004830:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829774.132851:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829775.028797:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787829775.156895:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829776.052827:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787829776.116782:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829776.116791:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829776.116795:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829776.116804:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5921 00000100:00080000:3.0:1787829776.116814:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829776.116820:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787829776.116851:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f821c0 x1874669228414848/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829776.116876:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829776.116879:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829776.116881:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829776.116885:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5921 00000100:00080000:3.0:1787829776.116889:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829776.116893:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787829776.116901:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f821c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228414848:192.168.203.114@tcp:250:kworker.0 00000008:00100000:3.0:1787829776.116902:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829776.116918:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83480 x1874669228414976/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829776.116929:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829776.116932:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829776.116935:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829776.116939:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5921 00000100:00080000:3.0:1787829776.116943:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829776.116946:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000800:00000100:0.0:1787829776.116951:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000008:00100000:3.0:1787829776.116951:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829776.116965:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80f00 x1874669228415104/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787829776.116978:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829776/real 1787829776] req@ffff985610f821c0 x1874669228414848/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829801 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829776.116992:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f821c0 x1874669228414848/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829801 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829776.117001:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829776.117006:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829776.117011:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f821c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228414848:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829776.117032:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228414976:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829776.117055:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829776.117067:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829776/real 1787829776] req@ffff985610f83480 x1874669228414976/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829801 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829776.117077:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83480 x1874669228414976/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829801 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829776.117083:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829776.117085:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829776.117087:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228414976:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829776.117091:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228415104:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829776.117105:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829776.117111:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829776/real 1787829776] req@ffff985610f80f00 x1874669228415104/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829801 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829776.117117:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80f00 x1874669228415104/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829801 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829776.117122:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829776.117124:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829776.117126:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228415104:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829776.180815:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829777.076798:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829777.204864:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829778.100792:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829778.228899:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829779.124820:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829779.252848:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829780.148935:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787829780.276911:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829781.172768:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787829781.236827:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829781.236837:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829781.236842:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829781.236852:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5926 00000100:00080000:3.0:1787829781.236863:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829781.236872:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787829781.236918:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82580 x1874669228415232/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829781.236952:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829781.236956:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829781.236958:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829781.236963:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5926 00000100:00080000:3.0:1787829781.236967:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829781.236972:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829781.236983:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829781.236994:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82940 x1874669228415360/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829781.237004:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829781.237008:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:0.0:1787829781.237010:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228415232:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787829781.237010:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829781.237018:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5926 00000100:00080000:3.0:1787829781.237023:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829781.237027:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829781.237031:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829781.237043:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82d00 x1874669228415488/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787829781.237098:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829781.237137:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829781/real 1787829781] req@ffff985610f82580 x1874669228415232/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829806 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829781.237157:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82580 x1874669228415232/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829806 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829781.237168:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829781.237173:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829781.237177:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228415232:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829781.237204:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228415360:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829781.237229:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829781.237243:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829781/real 1787829781] req@ffff985610f82940 x1874669228415360/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829806 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829781.237256:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82940 x1874669228415360/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829806 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829781.237265:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829781.237268:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829781.237271:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228415360:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829781.237277:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228415488:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829781.237296:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829781.237307:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829781/real 1787829781] req@ffff985610f82d00 x1874669228415488/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829806 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829781.237319:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82d00 x1874669228415488/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829806 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829781.237327:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829781.237330:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829781.237333:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228415488:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829781.300819:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829782.196781:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829782.324837:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829783.220787:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829783.348845:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829784.244791:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829784.372822:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829785.268859:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787829785.396810:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829786.292802:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787829786.356833:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829786.356843:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829786.356847:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829786.356858:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5931 00000100:00080000:2.0:1787829786.356869:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829786.356876:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787829786.356916:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8f00 x1874669228415616/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829786.356940:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829786.356942:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829786.356944:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829786.356946:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5931 00000100:00080000:2.0:1787829786.356948:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829786.356951:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787829786.356959:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829786.356967:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140dad00 x1874669228415744/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829786.356974:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829786.356976:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829786.356977:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829786.356980:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5931 00000100:00080000:2.0:1787829786.356981:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829786.356984:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787829786.356987:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829786.356994:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db0c0 x1874669228415872/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787829786.357036:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228415616:192.168.203.114@tcp:250:kworker.0 00000800:00000100:0.0:1787829786.357123:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829786.357166:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829786/real 1787829786] req@ffff9856140d8f00 x1874669228415616/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829811 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829786.357187:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8f00 x1874669228415616/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829811 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829786.357199:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829786.357204:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829786.357208:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228415616:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829786.357217:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228415744:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829786.357241:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829786.357254:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829786/real 1787829786] req@ffff9856140dad00 x1874669228415744/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829811 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829786.357266:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140dad00 x1874669228415744/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829811 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829786.357276:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829786.357280:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829786.357283:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228415744:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829786.357290:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228415872:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829786.357310:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829786.357320:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829786/real 1787829786] req@ffff9856140db0c0 x1874669228415872/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829811 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829786.357332:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db0c0 x1874669228415872/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829811 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829786.357357:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829786.357362:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829786.357365:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228415872:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829786.420844:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829787.316818:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829787.444860:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829788.340773:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829788.468865:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829789.364855:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829789.492848:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829790.388759:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787829790.516869:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829791.412785:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787829791.476781:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829791.476793:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829791.476797:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829791.476807:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5937 00000100:00080000:0.0:1787829791.476819:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829791.476826:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787829791.476875:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df52c0 x1874669228416000/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829791.476900:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829791.476903:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829791.476904:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829791.476907:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5937 00000100:00080000:0.0:1787829791.476910:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829791.476913:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787829791.476927:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829791.476938:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228416000:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829791.476939:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7840 x1874669228416128/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829791.476959:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829791.476962:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829791.476963:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829791.476966:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5937 00000100:00080000:0.0:1787829791.476969:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829791.476971:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787829791.476974:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829791.476984:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4f00 x1874669228416256/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829791.477020:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787829791.477058:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829791/real 1787829791] req@ffff985607df52c0 x1874669228416000/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829816 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829791.477073:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df52c0 x1874669228416000/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829816 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829791.477082:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829791.477086:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787829791.477089:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228416000:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787829791.477111:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228416128:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787829791.477128:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787829791.477136:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829791/real 1787829791] req@ffff985607df7840 x1874669228416128/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829816 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829791.477154:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7840 x1874669228416128/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829816 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829791.477160:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829791.477162:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787829791.477165:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228416128:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787829791.477170:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228416256:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787829791.477182:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787829791.477193:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829791/real 1787829791] req@ffff985607df4f00 x1874669228416256/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829816 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829791.477200:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4f00 x1874669228416256/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829816 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829791.477206:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829791.477208:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787829791.477210:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228416256:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829791.540800:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829792.436804:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829792.564829:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829793.460795:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829793.588782:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829794.484779:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829794.612895:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829795.508864:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787829795.636887:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829796.532802:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787829796.596826:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829796.596834:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829796.596837:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829796.596845:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5942 00000100:00080000:2.0:1787829796.596854:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829796.596860:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787829796.596894:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140da940 x1874669228416384/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829796.596933:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829796.596937:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829796.596939:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829796.596945:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5942 00000100:00080000:2.0:1787829796.596950:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829796.596955:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787829796.596964:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829796.596980:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d9a40 x1874669228416512/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787829796.596992:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228416384:192.168.203.114@tcp:250:kworker.0 00000100:00080000:2.0:1787829796.596998:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829796.597009:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829796.597012:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829796.597017:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5942 00000100:00080000:2.0:1787829796.597023:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829796.597030:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787829796.597036:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829796.597051:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8b40 x1874669228416640/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787829796.597091:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829796.597142:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829796/real 1787829796] req@ffff9856140da940 x1874669228416384/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829821 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829796.597159:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140da940 x1874669228416384/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829821 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829796.597169:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829796.597173:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829796.597176:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228416384:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829796.597203:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228416512:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829796.597226:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829796.597240:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829796/real 1787829796] req@ffff9856140d9a40 x1874669228416512/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829821 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829796.597254:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d9a40 x1874669228416512/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829821 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829796.597264:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829796.597268:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829796.597271:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228416512:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829796.597279:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228416640:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829796.597299:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829796.597307:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829796/real 1787829796] req@ffff9856140d8b40 x1874669228416640/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829821 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829796.597315:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8b40 x1874669228416640/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829821 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829796.597321:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829796.597323:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829796.597325:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228416640:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829796.660877:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829797.556826:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829797.684952:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829798.580779:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829798.708883:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829799.604840:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829799.732801:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829800.628773:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787829800.756869:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829801.652802:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787829801.716812:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829801.716822:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829801.716828:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829801.716838:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5947 00000100:00080000:2.0:1787829801.716851:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829801.716858:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787829801.716903:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d9680 x1874669228416768/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829801.716931:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829801.716934:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829801.716935:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829801.716938:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5947 00000100:00080000:2.0:1787829801.716941:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829801.716943:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787829801.716953:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829801.716962:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db480 x1874669228416896/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829801.716970:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829801.716973:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829801.716974:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829801.716977:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5947 00000100:00080000:2.0:1787829801.716979:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829801.716981:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787829801.716986:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829801.716994:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228416768:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787829801.716995:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d83c0 x1874669228417024/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787829801.717086:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829801.717122:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829801/real 1787829801] req@ffff9856140d9680 x1874669228416768/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829826 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829801.717143:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d9680 x1874669228416768/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829826 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829801.717152:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829801.717156:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829801.717159:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228416768:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829801.717165:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228416896:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829801.717182:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829801.717190:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829801/real 1787829801] req@ffff9856140db480 x1874669228416896/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829826 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829801.717198:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db480 x1874669228416896/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829826 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829801.717204:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829801.717206:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829801.717209:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228416896:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829801.717231:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228417024:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829801.717243:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829801.717248:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829801/real 1787829801] req@ffff9856140d83c0 x1874669228417024/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829826 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829801.717256:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d83c0 x1874669228417024/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829826 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829801.717265:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829801.717267:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829801.717269:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228417024:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829801.780804:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829802.676773:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829802.804858:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829803.700796:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829803.828802:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829804.724817:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829804.852824:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829805.748786:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787829805.876853:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829806.772792:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787829806.836807:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829806.836817:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829806.836822:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829806.836831:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5952 00000100:00080000:2.0:1787829806.836843:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829806.836851:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787829806.836889:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8780 x1874669228417152/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829806.836918:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829806.836922:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829806.836925:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829806.836930:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5952 00000100:00080000:2.0:1787829806.836935:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829806.836941:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787829806.836952:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228417152:192.168.203.114@tcp:250:kworker.0 00000008:00100000:2.0:1787829806.836952:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829806.836974:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d83c0 x1874669228417280/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829806.836990:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829806.836994:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829806.836997:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829806.837002:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5952 00000100:00080000:2.0:1787829806.837018:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829806.837023:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787829806.837028:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829806.837043:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db480 x1874669228417408/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787829806.837062:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228417280:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829806.837100:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228417408:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829806.900809:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829807.796799:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829807.924931:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829808.820805:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829808.948794:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829809.844820:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000400:00000100:0.0:1787829809.909021:0:2370:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.114:988 00000400:00000100:0.0:1787829809.909066:0:2370:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.114@tcp at host 192.168.203.114 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:0.0:1787829809.909085:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:0.0:1787829809.909138:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:0.0:1787829809.909150:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000100:00100000:2.0:1787829809.909205:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829806/real 1787829809] req@ffff9856140d83c0 x1874669228417280/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829831 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829809.909239:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d83c0 x1874669228417280/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829831 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829809.909257:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829809.909263:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787829809.909269:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228417280:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787829809.909280:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829806/real 1787829809] req@ffff9856140db480 x1874669228417408/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829831 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829809.909298:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db480 x1874669228417408/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829831 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829809.909325:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829809.909330:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787829809.909335:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228417408:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787829809.909344:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829806/real 1787829809] req@ffff9856140d8780 x1874669228417152/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829831 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829809.909380:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8780 x1874669228417152/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829831 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829809.909392:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829809.909397:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787829809.909401:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228417152:192.168.203.114@tcp:250:kworker.0 00010000:00010000:0.0:1787829809.972824:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829810.868791:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787829810.996826:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829811.892825:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787829811.956878:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829811.956889:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829811.956893:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829811.956904:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5957 00000100:00080000:1.0:1787829811.956923:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829811.956933:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787829811.956966:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a03c0 x1874669228417536/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829811.956994:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829811.956998:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829811.957000:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829811.957004:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5957 00000100:00080000:1.0:1787829811.957008:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829811.957013:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787829811.957023:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829811.957036:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3840 x1874669228417664/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829811.957041:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228417536:192.168.203.114@tcp:250:kworker.0 00000100:00080000:1.0:1787829811.957055:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829811.957059:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829811.957061:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829811.957075:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5957 00000100:00080000:1.0:1787829811.957080:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829811.957084:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787829811.957091:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829811.957103:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2580 x1874669228417792/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829811.957113:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787829811.957174:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829811/real 1787829811] req@ffff9856085a03c0 x1874669228417536/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829836 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829811.957193:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a03c0 x1874669228417536/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829836 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829811.957204:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829811.957208:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787829811.957212:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228417536:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787829811.957236:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228417664:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787829811.957260:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787829811.957273:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829811/real 1787829811] req@ffff9856085a3840 x1874669228417664/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829836 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829811.957285:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3840 x1874669228417664/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829836 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829811.957294:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829811.957298:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787829811.957301:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228417664:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787829811.957307:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228417792:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787829811.957327:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787829811.957339:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829811/real 1787829811] req@ffff9856085a2580 x1874669228417792/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829836 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829811.957350:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2580 x1874669228417792/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829836 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829811.957357:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829811.957360:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787829811.957363:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228417792:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829812.020889:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829812.916850:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829813.044916:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829813.940821:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829814.068942:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829814.964821:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829815.092875:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829815.988774:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787829816.116920:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829817.012827:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787829817.076868:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829817.076883:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829817.076889:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829817.076902:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5962 00000100:00080000:1.0:1787829817.076918:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829817.076927:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787829817.076981:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a1a40 x1874669228417920/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829817.077016:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829817.077020:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829817.077022:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829817.077027:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5962 00000100:00080000:1.0:1787829817.077031:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829817.077035:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787829817.077046:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829817.077061:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a21c0 x1874669228418048/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829817.077072:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228417920:192.168.203.114@tcp:250:kworker.0 00000100:00080000:1.0:1787829817.077076:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829817.077082:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829817.077084:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829817.077089:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5962 00000100:00080000:1.0:1787829817.077093:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829817.077097:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787829817.077102:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829817.077112:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2940 x1874669228418176/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829817.077158:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787829817.077204:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829817/real 1787829817] req@ffff9856085a1a40 x1874669228417920/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829842 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829817.077224:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a1a40 x1874669228417920/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829842 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829817.077238:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829817.077243:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787829817.077248:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228417920:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787829817.077277:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228418048:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787829817.077306:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787829817.077321:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829817/real 1787829817] req@ffff9856085a21c0 x1874669228418048/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829842 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829817.077335:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a21c0 x1874669228418048/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829842 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829817.077345:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829817.077349:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787829817.077353:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228418048:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787829817.077360:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228418176:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787829817.077380:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787829817.077391:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829817/real 1787829817] req@ffff9856085a2940 x1874669228418176/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829842 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829817.077422:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2940 x1874669228418176/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829842 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829817.077432:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829817.077436:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787829817.077439:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228418176:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829817.140836:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829818.036806:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829818.164927:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829819.060835:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829819.188858:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829820.084768:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829820.212856:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829821.108839:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787829821.236854:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829822.132837:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787829822.196843:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829822.196856:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829822.196861:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829822.196874:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5967 00000100:00080000:1.0:1787829822.196888:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829822.196897:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787829822.196943:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a12c0 x1874669228418304/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829822.196977:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829822.196981:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829822.196983:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829822.196987:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5967 00000100:00080000:1.0:1787829822.196990:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829822.196995:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787829822.197007:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829822.197019:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3480 x1874669228418432/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829822.197031:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1787829822.197036:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228418304:192.168.203.114@tcp:250:kworker.0 00000100:00080000:1.0:1787829822.197037:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829822.197040:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829822.197049:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5967 00000100:00080000:1.0:1787829822.197052:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829822.197056:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787829822.197060:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829822.197072:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a30c0 x1874669228418560/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829822.197130:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787829822.197178:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829822/real 1787829822] req@ffff9856085a12c0 x1874669228418304/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829847 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829822.197199:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a12c0 x1874669228418304/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829847 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829822.197211:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829822.197216:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787829822.197220:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228418304:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787829822.197247:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228418432:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787829822.197278:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787829822.197294:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829822/real 1787829822] req@ffff9856085a3480 x1874669228418432/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829847 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829822.197308:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3480 x1874669228418432/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829847 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829822.197318:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829822.197321:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787829822.197325:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228418432:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787829822.197332:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228418560:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787829822.197352:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787829822.197363:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829822/real 1787829822] req@ffff9856085a30c0 x1874669228418560/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829847 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829822.197390:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a30c0 x1874669228418560/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829847 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829822.197399:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829822.197403:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787829822.197407:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228418560:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829822.260992:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829823.156789:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829823.284904:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829824.180805:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829824.308843:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829825.204781:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829825.332829:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829826.228787:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787829826.356823:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829827.252798:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787829827.316837:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829827.316846:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829827.316850:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829827.316859:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5972 00000100:00080000:1.0:1787829827.316879:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829827.316886:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787829827.316917:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0f00 x1874669228418688/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829827.316944:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829827.316947:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829827.316949:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829827.316952:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5972 00000100:00080000:1.0:1787829827.316955:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829827.316958:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787829827.316965:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829827.316977:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a30c0 x1874669228418816/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829827.316987:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829827.316993:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829827.316996:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829827.317000:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5972 00000100:00080000:1.0:1787829827.317003:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829827.317006:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787829827.317010:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829827.317021:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3480 x1874669228418944/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829827.317034:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228418688:192.168.203.114@tcp:250:kworker.0 00000800:00000100:2.0:1787829827.317124:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787829827.317169:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829827/real 1787829827] req@ffff9856085a0f00 x1874669228418688/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829852 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829827.317188:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0f00 x1874669228418688/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829852 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829827.317201:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829827.317205:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787829827.317210:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228418688:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787829827.317218:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228418816:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787829827.317243:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787829827.317256:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829827/real 1787829827] req@ffff9856085a30c0 x1874669228418816/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829852 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829827.317268:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a30c0 x1874669228418816/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829852 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829827.317276:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829827.317280:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787829827.317283:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228418816:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787829827.317312:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228418944:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787829827.317441:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787829827.317454:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829827/real 1787829827] req@ffff9856085a3480 x1874669228418944/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829852 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829827.317473:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3480 x1874669228418944/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829852 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829827.317482:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829827.317485:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787829827.317489:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228418944:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829827.380835:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829828.276808:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829828.404885:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829829.300812:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829829.428890:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829830.324790:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829830.452856:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829831.348826:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787829831.476940:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829832.372804:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787829832.436806:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829832.436819:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829832.436825:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829832.436836:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5978 00000100:00080000:0.0:1787829832.436848:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829832.436856:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787829832.436897:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6d00 x1874669228419072/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829832.436923:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829832.436926:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829832.436928:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829832.436931:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5978 00000100:00080000:0.0:1787829832.436935:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829832.436939:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787829832.436949:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829832.436963:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5e00 x1874669228419200/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829832.436964:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228419072:192.168.203.114@tcp:250:kworker.0 00000100:00080000:0.0:1787829832.436981:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829832.436984:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829832.436986:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829832.436991:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5978 00000100:00080000:0.0:1787829832.436994:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829832.436998:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787829832.437002:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829832.437013:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5680 x1874669228419328/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829832.437047:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787829832.437086:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829832/real 1787829832] req@ffff985607df6d00 x1874669228419072/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829857 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829832.437104:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6d00 x1874669228419072/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829857 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829832.437115:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829832.437120:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787829832.437124:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228419072:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787829832.437160:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228419200:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787829832.437185:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787829832.437198:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829832/real 1787829832] req@ffff985607df5e00 x1874669228419200/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829857 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829832.437210:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5e00 x1874669228419200/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829857 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829832.437218:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829832.437221:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787829832.437225:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228419200:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787829832.437231:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228419328:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787829832.437247:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787829832.437257:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829832/real 1787829832] req@ffff985607df5680 x1874669228419328/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829857 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829832.437268:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5680 x1874669228419328/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829857 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829832.437276:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829832.437279:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787829832.437282:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228419328:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829832.500889:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829833.396776:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829833.524839:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829834.420749:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829834.548870:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829835.444803:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829835.572812:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829836.468795:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787829836.596929:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829837.492792:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787829837.556800:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829837.556808:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829837.556811:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829837.556820:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5983 00000100:00080000:1.0:1787829837.556830:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829837.556838:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787829837.556871:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a12c0 x1874669228419456/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829837.556891:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829837.556894:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829837.556896:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829837.556900:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5983 00000100:00080000:1.0:1787829837.556903:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829837.556907:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787829837.556915:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829837.556927:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2940 x1874669228419584/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829837.556932:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228419456:192.168.203.114@tcp:250:kworker.0 00000100:00080000:1.0:1787829837.556936:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829837.556939:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829837.556940:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829837.556944:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5983 00000100:00080000:1.0:1787829837.556947:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829837.556950:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787829837.556954:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829837.556961:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a21c0 x1874669228419712/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829837.556995:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787829837.557029:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829837/real 1787829837] req@ffff9856085a12c0 x1874669228419456/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829862 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829837.557040:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a12c0 x1874669228419456/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829862 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829837.557046:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829837.557052:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787829837.557054:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228419456:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787829837.557071:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228419584:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787829837.557085:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787829837.557092:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829837/real 1787829837] req@ffff9856085a2940 x1874669228419584/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829862 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829837.557100:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2940 x1874669228419584/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829862 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829837.557107:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829837.557110:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787829837.557119:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228419584:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787829837.557125:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228419712:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787829837.557143:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787829837.557153:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829837/real 1787829837] req@ffff9856085a21c0 x1874669228419712/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829862 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829837.557168:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a21c0 x1874669228419712/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829862 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829837.557174:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829837.557177:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787829837.557180:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228419712:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829837.620803:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829838.516802:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829838.644840:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829839.540805:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829839.668849:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829840.564797:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829840.692789:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829841.588809:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787829841.716854:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829842.612784:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787829842.676930:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829842.676944:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829842.676951:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829842.676965:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5988 00000100:00080000:1.0:1787829842.676980:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829842.676989:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787829842.677038:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a1a40 x1874669228419840/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829842.677067:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829842.677070:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829842.677071:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829842.677075:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5988 00000100:00080000:1.0:1787829842.677079:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829842.677083:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787829842.677093:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829842.677103:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2580 x1874669228419968/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829842.677111:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1787829842.677116:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228419840:192.168.203.114@tcp:250:kworker.0 00000100:00080000:1.0:1787829842.677116:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829842.677118:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829842.677126:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5988 00000100:00080000:1.0:1787829842.677130:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829842.677134:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787829842.677139:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829842.677147:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3840 x1874669228420096/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787829842.677198:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829842.677240:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829842/real 1787829842] req@ffff9856085a1a40 x1874669228419840/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829867 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829842.677254:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a1a40 x1874669228419840/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829867 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829842.677263:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829842.677267:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829842.677270:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228419840:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829842.677293:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228419968:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829842.677312:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829842.677325:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829842/real 1787829842] req@ffff9856085a2580 x1874669228419968/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829867 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829842.677338:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2580 x1874669228419968/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829867 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829842.677345:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829842.677348:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829842.677351:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228419968:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829842.677357:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228420096:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829842.677377:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829842.677387:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829842/real 1787829842] req@ffff9856085a3840 x1874669228420096/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829867 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829842.677397:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3840 x1874669228420096/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829867 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829842.677404:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829842.677407:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829842.677410:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228420096:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829842.740827:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829843.636867:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829843.764869:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829844.660775:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829844.788833:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829845.684832:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829845.812852:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829846.708763:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787829846.836913:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829847.732845:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787829847.796794:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829847.796808:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829847.796814:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829847.796824:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5993 00000100:00080000:1.0:1787829847.796836:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829847.796842:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787829847.796882:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a03c0 x1874669228420224/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829847.796911:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829847.796915:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829847.796916:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829847.796919:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5993 00000100:00080000:1.0:1787829847.796922:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829847.796925:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787829847.796934:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829847.796943:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228420224:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787829847.796944:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3c00 x1874669228420352/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829847.796955:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829847.796957:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829847.796958:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829847.796961:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5993 00000100:00080000:1.0:1787829847.796963:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829847.796965:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787829847.796967:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829847.796974:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0780 x1874669228420480/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787829847.797021:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829847.797055:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829847/real 1787829847] req@ffff9856085a03c0 x1874669228420224/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829872 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829847.797073:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a03c0 x1874669228420224/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829872 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829847.797081:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829847.797085:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829847.797087:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228420224:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829847.797107:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228420352:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829847.797120:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829847.797128:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829847/real 1787829847] req@ffff9856085a3c00 x1874669228420352/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829872 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829847.797135:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3c00 x1874669228420352/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829872 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829847.797139:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829847.797141:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829847.797143:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228420352:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829847.797145:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228420480:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829847.797155:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829847.797160:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829847/real 1787829847] req@ffff9856085a0780 x1874669228420480/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829872 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829847.797167:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0780 x1874669228420480/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829872 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829847.797176:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829847.797177:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829847.797179:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228420480:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829847.860862:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829848.756801:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787829848.884903:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829849.780820:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829849.908863:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829850.804819:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829850.932881:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829851.828825:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787829851.956936:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829852.852801:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787829852.916848:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829852.916862:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829852.916867:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829852.916880:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 5998 00000100:00080000:1.0:1787829852.916894:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829852.916903:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787829852.916948:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a1680 x1874669228420608/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829852.916987:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829852.916992:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829852.916994:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829852.916998:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5998 00000100:00080000:1.0:1787829852.917001:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829852.917005:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787829852.917017:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829852.917029:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0b40 x1874669228420736/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829852.917039:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829852.917042:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829852.917044:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829852.917048:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 5998 00000100:00080000:1.0:1787829852.917052:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829852.917055:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787829852.917060:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829852.917070:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0000 x1874669228420864/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787829852.917102:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228420608:192.168.203.114@tcp:250:kworker.0 00000800:00000100:0.0:1787829852.917199:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829852.917240:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829852/real 1787829852] req@ffff9856085a1680 x1874669228420608/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829877 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829852.917260:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a1680 x1874669228420608/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829877 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829852.917273:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829852.917277:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829852.917282:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228420608:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829852.917299:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228420736:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829852.917320:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829852.917329:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829852/real 1787829852] req@ffff9856085a0b40 x1874669228420736/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829877 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829852.917338:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0b40 x1874669228420736/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829877 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829852.917349:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829852.917351:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829852.917353:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228420736:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829852.917357:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228420864:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829852.917368:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829852.917374:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829852/real 1787829852] req@ffff9856085a0000 x1874669228420864/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829877 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829852.917381:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0000 x1874669228420864/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829877 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829852.917386:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829852.917388:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829852.917390:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228420864:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829852.980889:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829853.876828:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829854.004917:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829854.900783:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829855.028880:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829855.924797:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829856.052845:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829856.948803:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787829857.076961:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829857.972841:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787829858.036893:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829858.036904:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829858.036908:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829858.036919:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6003 00000100:00080000:0.0:1787829858.036932:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829858.036939:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787829858.036986:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df61c0 x1874669228420992/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829858.037017:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829858.037020:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829858.037022:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829858.037024:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6003 00000100:00080000:0.0:1787829858.037027:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829858.037029:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787829858.037039:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829858.037048:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df43c0 x1874669228421120/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829858.037056:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829858.037060:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829858.037062:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:2.0:1787829858.037065:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228420992:192.168.203.114@tcp:250:kworker.0 00000100:00080000:0.0:1787829858.037067:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6003 00000100:00080000:0.0:1787829858.037073:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829858.037076:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787829858.037079:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829858.037087:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df70c0 x1874669228421248/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829858.037166:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787829858.037213:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829858/real 1787829858] req@ffff985607df61c0 x1874669228420992/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829883 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829858.037230:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df61c0 x1874669228420992/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829883 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829858.037240:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829858.037244:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787829858.037247:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228420992:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787829858.037271:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228421120:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787829858.037290:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787829858.037299:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829858/real 1787829858] req@ffff985607df43c0 x1874669228421120/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829883 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829858.037307:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df43c0 x1874669228421120/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829883 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829858.037313:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829858.037316:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787829858.037318:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228421120:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787829858.037322:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228421248:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787829858.037332:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787829858.037338:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829858/real 1787829858] req@ffff985607df70c0 x1874669228421248/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829883 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829858.037345:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df70c0 x1874669228421248/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829883 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829858.037351:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829858.037353:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787829858.037355:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228421248:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829858.100842:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829858.996793:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829859.124861:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829860.020847:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829860.148848:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829861.044835:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829861.172851:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829862.068769:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787829862.196856:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829863.092782:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787829863.156786:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829863.156794:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829863.156797:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829863.156806:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6008 00000100:00080000:1.0:1787829863.156818:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829863.156825:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787829863.156868:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2d00 x1874669228421376/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829863.156890:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829863.156893:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829863.156894:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829863.156897:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6008 00000100:00080000:1.0:1787829863.156900:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829863.156903:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787829863.156911:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829863.156927:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0000 x1874669228421504/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829863.156940:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829863.156946:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:2.0:1787829863.156946:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228421376:192.168.203.114@tcp:250:kworker.0 00000100:00080000:1.0:1787829863.156950:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829863.156959:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6008 00000100:00080000:1.0:1787829863.156962:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829863.156964:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787829863.156969:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829863.156979:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0b40 x1874669228421632/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829863.157014:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787829863.157059:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829863/real 1787829863] req@ffff9856085a2d00 x1874669228421376/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829888 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829863.157076:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2d00 x1874669228421376/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829888 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829863.157098:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829863.157103:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787829863.157107:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228421376:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787829863.157131:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228421504:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787829863.157153:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787829863.157166:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829863/real 1787829863] req@ffff9856085a0000 x1874669228421504/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829888 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829863.157178:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0000 x1874669228421504/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829888 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829863.157187:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829863.157190:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787829863.157194:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228421504:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787829863.157200:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228421632:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787829863.157218:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787829863.157228:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829863/real 1787829863] req@ffff9856085a0b40 x1874669228421632/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829888 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829863.157246:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0b40 x1874669228421632/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829888 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829863.157254:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829863.157257:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787829863.157260:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228421632:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829863.220800:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829864.116761:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829864.244847:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829865.140795:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829865.268789:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829866.164812:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829866.292811:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829867.188791:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787829867.316807:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829868.212773:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787829868.276795:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829868.276803:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829868.276807:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829868.276815:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6013 00000100:00080000:2.0:1787829868.276824:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829868.276829:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787829868.276859:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8780 x1874669228421760/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829868.276884:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829868.276886:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829868.276887:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829868.276890:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6013 00000100:00080000:2.0:1787829868.276892:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829868.276894:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787829868.276902:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829868.276911:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db480 x1874669228421888/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829868.276919:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829868.276921:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829868.276922:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829868.276925:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6013 00000100:00080000:2.0:1787829868.276927:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829868.276929:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787829868.276932:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829868.276940:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228421760:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787829868.276940:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d83c0 x1874669228422016/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787829868.277005:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829868.277036:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829868/real 1787829868] req@ffff9856140d8780 x1874669228421760/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829893 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829868.277070:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8780 x1874669228421760/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829893 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829868.277078:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829868.277081:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829868.277084:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228421760:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829868.277089:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228421888:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829868.277111:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829868.277124:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829868/real 1787829868] req@ffff9856140db480 x1874669228421888/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829893 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829868.277138:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db480 x1874669228421888/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829893 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829868.277147:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829868.277151:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829868.277153:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228421888:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829868.277172:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228422016:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829868.277193:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829868.277205:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829868/real 1787829868] req@ffff9856140d83c0 x1874669228422016/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829893 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829868.277218:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d83c0 x1874669228422016/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829893 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829868.277227:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829868.277230:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829868.277232:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228422016:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829868.340838:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829869.236799:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829869.364795:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829870.260770:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829870.388818:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829871.284810:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829871.412823:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829872.308762:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787829872.436816:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829873.332777:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787829873.396822:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829873.396833:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829873.396838:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829873.396849:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6019 00000100:00080000:3.0:1787829873.396861:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829873.396869:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787829873.396907:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81e00 x1874669228422144/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829873.396937:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829873.396941:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829873.396943:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829873.396948:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6019 00000100:00080000:3.0:1787829873.396953:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829873.396958:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829873.396969:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829873.396976:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228422144:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787829873.396987:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f803c0 x1874669228422272/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829873.397000:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829873.397004:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829873.397006:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829873.397022:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6019 00000100:00080000:3.0:1787829873.397028:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829873.397033:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829873.397039:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829873.397053:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83840 x1874669228422400/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787829873.397094:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228422272:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829873.397124:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228422400:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829873.460809:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829874.356789:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829874.484836:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829875.380791:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829875.508776:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829876.404791:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000400:00000100:1.0:1787829876.468936:0:2371:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.114:988 00000400:00000100:1.0:1787829876.468973:0:2371:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.114@tcp at host 192.168.203.114 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1787829876.469001:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:1.0:1787829876.469045:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:1.0:1787829876.469056:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000100:00100000:0.0:1787829876.469132:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829873/real 1787829876] req@ffff985610f803c0 x1874669228422272/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829898 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829876.469157:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f803c0 x1874669228422272/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829898 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829876.469189:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829876.469194:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829876.469198:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228422272:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829876.469206:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829873/real 1787829876] req@ffff985610f83840 x1874669228422400/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829898 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829876.469220:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83840 x1874669228422400/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829898 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829876.469228:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829876.469232:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829876.469235:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228422400:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829876.469241:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829873/real 1787829876] req@ffff985610f81e00 x1874669228422144/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829898 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829876.469252:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81e00 x1874669228422144/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829898 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829876.469259:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829876.469262:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829876.469265:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228422144:192.168.203.114@tcp:250:kworker.0 00010000:00010000:0.0:1787829876.532857:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829877.428803:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787829877.556848:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829878.452801:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787829878.516836:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829878.516843:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829878.516846:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829878.516855:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6024 00000100:00080000:1.0:1787829878.516864:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829878.516869:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787829878.516895:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a1680 x1874669228422528/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829878.516916:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829878.516918:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829878.516920:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829878.516923:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6024 00000100:00080000:1.0:1787829878.516925:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829878.516927:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787829878.516934:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829878.516945:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0780 x1874669228422656/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829878.516957:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829878.516961:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829878.516966:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1787829878.516967:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228422528:192.168.203.114@tcp:250:kworker.0 00000100:00080000:1.0:1787829878.516984:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6024 00000100:00080000:1.0:1787829878.516991:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829878.516995:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787829878.517002:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829878.517022:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3c00 x1874669228422784/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787829878.517050:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829878.517088:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829878/real 1787829878] req@ffff9856085a1680 x1874669228422528/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829903 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829878.517111:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a1680 x1874669228422528/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829903 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829878.517124:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829878.517129:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829878.517134:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228422528:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829878.517160:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228422656:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829878.517192:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829878.517208:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829878/real 1787829878] req@ffff9856085a0780 x1874669228422656/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829903 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829878.517223:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0780 x1874669228422656/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829903 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829878.517233:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829878.517236:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829878.517240:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228422656:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829878.517247:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228422784:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829878.517270:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829878.517284:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829878/real 1787829878] req@ffff9856085a3c00 x1874669228422784/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829903 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829878.517297:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3c00 x1874669228422784/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829903 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829878.517311:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829878.517322:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829878.517326:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228422784:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829878.580894:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829879.476788:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829879.604812:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829880.500797:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829880.628815:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829881.524776:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829881.652795:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829882.548768:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787829882.676835:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829883.572805:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787829883.636809:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829883.636821:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829883.636825:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829883.636835:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6029 00000100:00080000:3.0:1787829883.636846:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829883.636852:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787829883.636885:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81680 x1874669228422912/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829883.636906:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829883.636909:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829883.636910:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829883.636913:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6029 00000100:00080000:3.0:1787829883.636917:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829883.636921:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829883.636930:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829883.636964:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228422912:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787829883.636965:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f812c0 x1874669228423040/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829883.636983:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829883.636985:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829883.636987:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829883.636991:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6029 00000100:00080000:3.0:1787829883.636996:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829883.637000:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829883.637006:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829883.637020:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80b40 x1874669228423168/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787829883.637036:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829883.637077:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829883/real 1787829883] req@ffff985610f81680 x1874669228422912/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829908 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829883.637097:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81680 x1874669228422912/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829908 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829883.637107:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829883.637111:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829883.637115:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228422912:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829883.637139:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228423040:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829883.637163:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829883.637177:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829883/real 1787829883] req@ffff985610f812c0 x1874669228423040/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829908 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829883.637190:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f812c0 x1874669228423040/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829908 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829883.637205:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829883.637209:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829883.637212:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228423040:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829883.637218:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228423168:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829883.637238:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829883.637250:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829883/real 1787829883] req@ffff985610f80b40 x1874669228423168/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829908 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829883.637262:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80b40 x1874669228423168/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829908 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829883.637270:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829883.637273:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829883.637277:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228423168:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829883.700828:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829884.596830:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829884.724872:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829885.620780:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829885.748853:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829886.644807:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829886.772796:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829887.668811:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787829887.796955:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829888.692813:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787829888.756757:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829888.756766:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829888.756770:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829888.756780:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6034 00000100:00080000:3.0:1787829888.756791:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829888.756799:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787829888.756833:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80000 x1874669228423296/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829888.756861:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829888.756866:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829888.756868:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829888.756872:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6034 00000100:00080000:3.0:1787829888.756876:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829888.756880:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829888.756889:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829888.756912:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81a40 x1874669228423424/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829888.756925:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829888.756930:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829888.756932:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829888.756937:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6034 00000100:00080000:3.0:1787829888.756940:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829888.756945:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829888.756951:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829888.756965:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f830c0 x1874669228423552/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787829888.756970:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228423296:192.168.203.114@tcp:250:kworker.0 00000800:00000100:0.0:1787829888.757087:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829888.757137:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829888/real 1787829888] req@ffff985610f80000 x1874669228423296/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829913 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829888.757173:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80000 x1874669228423296/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829913 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829888.757193:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829888.757199:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829888.757204:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228423296:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829888.757214:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228423424:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829888.757243:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829888.757259:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829888/real 1787829888] req@ffff985610f81a40 x1874669228423424/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829913 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829888.757274:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81a40 x1874669228423424/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829913 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829888.757284:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829888.757288:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829888.757292:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228423424:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829888.757323:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228423552:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829888.757358:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829888.757373:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829888/real 1787829888] req@ffff985610f830c0 x1874669228423552/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829913 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829888.757386:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f830c0 x1874669228423552/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829913 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829888.757396:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829888.757400:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829888.757403:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228423552:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829888.820852:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829889.716806:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829889.844877:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829890.740788:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829890.868904:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829891.764798:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829891.892876:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829892.788843:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787829892.916954:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829893.812815:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787829893.876848:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829893.876860:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829893.876865:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829893.876890:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6039 00000100:00080000:1.0:1787829893.876904:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829893.876912:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787829893.876957:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a03c0 x1874669228423680/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829893.876994:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829893.876999:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829893.877001:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829893.877006:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6039 00000100:00080000:1.0:1787829893.877009:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829893.877013:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787829893.877024:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829893.877036:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3840 x1874669228423808/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829893.877046:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1787829893.877047:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228423680:192.168.203.114@tcp:250:kworker.0 00000100:00080000:1.0:1787829893.877056:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829893.877059:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829893.877069:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6039 00000100:00080000:1.0:1787829893.877073:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829893.877076:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787829893.877080:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829893.877090:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2580 x1874669228423936/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787829893.877128:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829893.877172:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829893/real 1787829893] req@ffff9856085a03c0 x1874669228423680/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829918 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829893.877192:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a03c0 x1874669228423680/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829918 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829893.877204:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829893.877208:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829893.877212:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228423680:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829893.877239:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228423808:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829893.877266:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829893.877279:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829893/real 1787829893] req@ffff9856085a3840 x1874669228423808/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829918 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829893.877292:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3840 x1874669228423808/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829918 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829893.877302:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829893.877306:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829893.877308:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228423808:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829893.877315:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228423936:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829893.877341:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829893.877350:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829893/real 1787829893] req@ffff9856085a2580 x1874669228423936/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829918 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829893.877358:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2580 x1874669228423936/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829918 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829893.877364:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829893.877366:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829893.877369:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228423936:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829893.940921:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829894.836813:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829894.964848:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829895.860792:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829895.988896:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829896.884808:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829897.012817:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829897.908797:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787829898.036962:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829898.932992:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787829898.996867:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829898.996883:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829898.996888:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829898.996900:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6044 00000100:00080000:1.0:1787829898.996913:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829898.996920:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787829898.996962:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a1a40 x1874669228424064/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829898.996989:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829898.996992:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829898.996994:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829898.996998:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6044 00000100:00080000:1.0:1787829898.997001:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829898.997005:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787829898.997017:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829898.997030:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a21c0 x1874669228424192/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829898.997041:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829898.997044:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829898.997045:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1787829898.997050:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228424064:192.168.203.114@tcp:250:kworker.0 00000100:00080000:1.0:1787829898.997051:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6044 00000100:00080000:1.0:1787829898.997056:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829898.997061:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787829898.997064:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829898.997073:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2940 x1874669228424320/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787829898.997141:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829898.997190:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829898/real 1787829898] req@ffff9856085a1a40 x1874669228424064/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829923 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829898.997208:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a1a40 x1874669228424064/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829923 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829898.997220:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829898.997224:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829898.997228:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228424064:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829898.997255:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228424192:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829898.997276:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829898.997294:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829898/real 1787829898] req@ffff9856085a21c0 x1874669228424192/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829923 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829898.997305:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a21c0 x1874669228424192/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829923 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829898.997313:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829898.997317:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829898.997320:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228424192:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829898.997325:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228424320:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829898.997339:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829898.997347:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829898/real 1787829898] req@ffff9856085a2940 x1874669228424320/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829923 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829898.997358:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2940 x1874669228424320/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829923 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829898.997365:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829898.997368:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829898.997371:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228424320:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829899.060915:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829899.956851:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829900.084876:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829900.980774:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829901.108833:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829902.004780:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829902.132791:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829903.028741:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787829903.156915:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829904.052825:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787829904.116763:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829904.116770:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829904.116773:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829904.116781:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6049 00000100:00080000:3.0:1787829904.116789:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829904.116804:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787829904.116833:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80780 x1874669228424448/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829904.116852:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829904.116855:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829904.116857:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829904.116861:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6049 00000100:00080000:3.0:1787829904.116864:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829904.116868:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829904.116876:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829904.116887:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83c00 x1874669228424576/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787829904.116892:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228424448:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787829904.116897:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829904.116900:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829904.116901:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829904.116904:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6049 00000100:00080000:3.0:1787829904.116907:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829904.116911:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829904.116915:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829904.116924:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f830c0 x1874669228424704/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787829904.116946:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829904.116983:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829904/real 1787829904] req@ffff985610f80780 x1874669228424448/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829929 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829904.116998:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80780 x1874669228424448/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829929 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829904.117008:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829904.117012:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829904.117016:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228424448:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829904.117037:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228424576:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829904.117054:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829904.117062:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829904/real 1787829904] req@ffff985610f83c00 x1874669228424576/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829929 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829904.117069:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83c00 x1874669228424576/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829929 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829904.117074:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829904.117076:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829904.117078:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228424576:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829904.117081:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228424704:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829904.117093:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829904.117109:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829904/real 1787829904] req@ffff985610f830c0 x1874669228424704/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829929 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829904.117126:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f830c0 x1874669228424704/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829929 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829904.117131:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829904.117133:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829904.117135:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228424704:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829904.180847:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829905.076760:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829905.204874:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829906.100820:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829906.228812:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829907.124830:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829907.252874:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829908.148853:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787829908.276918:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829909.172839:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787829909.236850:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829909.236864:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829909.236869:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829909.236881:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6054 00000100:00080000:2.0:1787829909.236895:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829909.236903:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787829909.236949:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d9680 x1874669228424832/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829909.236980:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829909.236985:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829909.236987:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829909.236991:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6054 00000100:00080000:2.0:1787829909.236995:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829909.237000:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787829909.237011:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829909.237024:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8b40 x1874669228424960/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829909.237035:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829909.237040:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829909.237046:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1787829909.237049:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228424832:192.168.203.114@tcp:250:kworker.0 00000100:00080000:2.0:1787829909.237053:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6054 00000100:00080000:2.0:1787829909.237060:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829909.237066:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787829909.237070:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829909.237084:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d9a40 x1874669228425088/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787829909.237127:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829909.237163:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829909/real 1787829909] req@ffff9856140d9680 x1874669228424832/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829934 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829909.237180:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d9680 x1874669228424832/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829934 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829909.237192:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829909.237197:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829909.237200:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228424832:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829909.237224:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228424960:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829909.237256:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829909.237267:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829909/real 1787829909] req@ffff9856140d8b40 x1874669228424960/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829934 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829909.237286:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8b40 x1874669228424960/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829934 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829909.237295:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829909.237298:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829909.237301:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228424960:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829909.237307:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228425088:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829909.237324:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829909.237334:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829909/real 1787829909] req@ffff9856140d9a40 x1874669228425088/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829934 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829909.237345:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d9a40 x1874669228425088/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829934 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829909.237353:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829909.237356:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829909.237360:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228425088:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829909.300807:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829910.196844:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829910.324867:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829911.220831:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829911.348911:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829912.244831:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829912.372940:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829913.268804:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787829913.396930:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829914.292830:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787829914.356877:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829914.356890:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829914.356896:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829914.356910:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6059 00000100:00080000:2.0:1787829914.356924:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829914.356933:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787829914.356980:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140da940 x1874669228425216/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829914.357017:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829914.357022:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829914.357024:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829914.357028:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6059 00000100:00080000:2.0:1787829914.357032:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829914.357037:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787829914.357049:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829914.357062:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228425216:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787829914.357068:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db0c0 x1874669228425344/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829914.357081:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829914.357085:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829914.357087:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829914.357091:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6059 00000100:00080000:2.0:1787829914.357096:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829914.357100:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787829914.357107:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829914.357130:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140dad00 x1874669228425472/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787829914.357138:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829914.357184:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829914/real 1787829914] req@ffff9856140da940 x1874669228425216/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829939 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829914.357211:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140da940 x1874669228425216/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829939 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829914.357230:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829914.357235:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829914.357239:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228425216:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829914.357264:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228425344:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829914.357293:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829914.357311:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829914/real 1787829914] req@ffff9856140db0c0 x1874669228425344/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829939 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829914.357324:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db0c0 x1874669228425344/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829939 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829914.357331:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829914.357334:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829914.357336:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228425344:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829914.357341:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228425472:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829914.357357:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829914.357366:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829914/real 1787829914] req@ffff9856140dad00 x1874669228425472/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829939 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829914.357379:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140dad00 x1874669228425472/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829939 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829914.357396:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829914.357403:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829914.357407:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228425472:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829914.420831:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829915.316811:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829915.444880:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829916.340847:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829916.468852:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829917.364818:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829917.492863:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829918.388784:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787829918.516846:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829919.412818:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787829919.476839:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829919.476857:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829919.476862:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829919.476875:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6065 00000100:00080000:2.0:1787829919.476888:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829919.476899:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787829919.476941:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8f00 x1874669228425600/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829919.476970:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829919.476974:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829919.476976:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829919.476982:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6065 00000100:00080000:2.0:1787829919.476987:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829919.476991:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787829919.477002:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829919.477012:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228425600:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787829919.477022:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140da1c0 x1874669228425728/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829919.477035:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829919.477040:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829919.477041:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829919.477046:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6065 00000100:00080000:2.0:1787829919.477051:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829919.477056:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787829919.477062:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829919.477076:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db840 x1874669228425856/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787829919.477083:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829919.477124:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829919/real 1787829919] req@ffff9856140d8f00 x1874669228425600/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829944 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829919.477137:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8f00 x1874669228425600/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829944 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829919.477145:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829919.477147:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829919.477156:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228425600:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829919.477177:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228425728:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829919.477191:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829919.477198:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829919/real 1787829919] req@ffff9856140da1c0 x1874669228425728/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829944 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829919.477205:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140da1c0 x1874669228425728/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829944 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829919.477215:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829919.477217:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829919.477219:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228425728:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829919.477222:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228425856:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829919.477232:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829919.477237:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829919/real 1787829919] req@ffff9856140db840 x1874669228425856/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829944 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829919.477243:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db840 x1874669228425856/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829944 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829919.477247:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829919.477249:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829919.477250:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228425856:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829919.540842:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829920.436814:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829920.564799:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829921.460837:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829921.588864:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829922.484782:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829922.612894:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829923.508825:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787829923.636893:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829924.532752:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787829924.596823:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829924.596833:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829924.596837:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829924.596846:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6070 00000100:00080000:3.0:1787829924.596858:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829924.596864:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787829924.596903:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81a40 x1874669228425984/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829924.596924:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829924.596927:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829924.596928:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829924.596931:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6070 00000100:00080000:3.0:1787829924.596934:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829924.596938:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829924.596948:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829924.596961:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80000 x1874669228426112/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829924.596971:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1787829924.596974:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228425984:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787829924.596976:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829924.596981:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829924.596986:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6070 00000100:00080000:3.0:1787829924.596989:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829924.596993:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787829924.596997:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829924.597009:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80b40 x1874669228426240/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787829924.597049:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829924.597083:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829924/real 1787829924] req@ffff985610f81a40 x1874669228425984/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829949 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829924.597097:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81a40 x1874669228425984/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829949 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829924.597109:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829924.597112:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829924.597115:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228425984:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829924.597136:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228426112:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829924.597149:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829924.597157:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829924/real 1787829924] req@ffff985610f80000 x1874669228426112/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829949 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829924.597164:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80000 x1874669228426112/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829949 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829924.597169:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829924.597171:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829924.597173:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228426112:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829924.597176:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228426240:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829924.597191:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829924.597200:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829924/real 1787829924] req@ffff985610f80b40 x1874669228426240/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829949 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829924.597212:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80b40 x1874669228426240/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829949 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829924.597220:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829924.597223:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829924.597226:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228426240:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829924.660892:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829925.556782:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829925.684836:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829926.580778:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829926.708802:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829927.604806:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829927.732877:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829928.628826:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787829928.756856:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829929.652828:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787829929.716810:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829929.716819:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829929.716823:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829929.716833:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6075 00000100:00080000:2.0:1787829929.716844:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829929.716851:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787829929.716896:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140dbc00 x1874669228426368/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829929.716933:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829929.716936:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829929.716937:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829929.716940:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6075 00000100:00080000:2.0:1787829929.716943:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829929.716947:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787829929.716956:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829929.716968:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db840 x1874669228426496/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787829929.716975:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228426368:192.168.203.114@tcp:250:kworker.0 00000100:00080000:2.0:1787829929.716978:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829929.716984:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829929.716986:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829929.716988:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6075 00000100:00080000:2.0:1787829929.716990:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829929.716992:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787829929.716995:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829929.717002:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140da1c0 x1874669228426624/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787829929.717055:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829929.717098:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829929/real 1787829929] req@ffff9856140dbc00 x1874669228426368/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829954 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829929.717121:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140dbc00 x1874669228426368/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829954 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829929.717133:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829929.717137:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829929.717141:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228426368:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829929.717175:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228426496:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829929.717198:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829929.717211:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829929/real 1787829929] req@ffff9856140db840 x1874669228426496/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829954 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829929.717222:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db840 x1874669228426496/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829954 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829929.717238:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829929.717241:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829929.717244:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228426496:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829929.717253:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228426624:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829929.717270:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829929.717280:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829929/real 1787829929] req@ffff9856140da1c0 x1874669228426624/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829954 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829929.717292:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140da1c0 x1874669228426624/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829954 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829929.717299:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829929.717302:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829929.717305:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228426624:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829929.780850:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829930.676830:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829930.804899:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829931.700818:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829931.828851:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829932.724823:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829932.852875:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829933.748845:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787829933.876936:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829934.772904:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787829934.836795:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829934.836806:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829934.836811:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829934.836820:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6080 00000100:00080000:0.0:1787829934.836833:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829934.836839:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787829934.836876:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6580 x1874669228426752/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829934.836899:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829934.836902:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829934.836904:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829934.836907:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6080 00000100:00080000:0.0:1787829934.836909:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829934.836912:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787829934.836921:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829934.836929:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5a40 x1874669228426880/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829934.836936:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829934.836940:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:2.0:1787829934.836940:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228426752:192.168.203.114@tcp:250:kworker.0 00000100:00080000:0.0:1787829934.836941:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829934.836945:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6080 00000100:00080000:0.0:1787829934.836950:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829934.836953:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787829934.836956:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829934.836964:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df70c0 x1874669228427008/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829934.837022:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787829934.837061:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829934/real 1787829934] req@ffff985607df6580 x1874669228426752/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829959 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829934.837080:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6580 x1874669228426752/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829959 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829934.837091:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829934.837096:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787829934.837100:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228426752:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787829934.837126:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228426880:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787829934.837149:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787829934.837162:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829934/real 1787829934] req@ffff985607df5a40 x1874669228426880/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829959 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829934.837174:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5a40 x1874669228426880/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829959 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829934.837184:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829934.837187:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787829934.837190:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228426880:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787829934.837196:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228427008:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787829934.837214:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787829934.837224:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829934/real 1787829934] req@ffff985607df70c0 x1874669228427008/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829959 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829934.837235:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df70c0 x1874669228427008/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829959 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829934.837243:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829934.837246:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787829934.837250:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228427008:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829934.900893:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829935.796820:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829935.924835:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829936.820784:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829936.948813:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829937.844828:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829937.972840:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829938.868802:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787829938.996890:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829939.892848:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787829939.956912:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829939.956926:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829939.956931:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829939.956944:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6085 00000100:00080000:0.0:1787829939.956959:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829939.956968:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787829939.957018:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df43c0 x1874669228427136/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829939.957052:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829939.957055:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829939.957056:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829939.957059:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6085 00000100:00080000:0.0:1787829939.957062:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829939.957065:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787829939.957075:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829939.957089:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df61c0 x1874669228427264/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829939.957100:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829939.957103:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829939.957105:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829939.957110:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6085 00000100:00080000:0.0:1787829939.957115:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829939.957120:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787829939.957126:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829939.957134:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228427136:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829939.957148:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5680 x1874669228427392/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829939.957254:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228427264:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787829939.957304:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228427392:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829940.020911:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829940.916780:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829941.044852:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829941.940765:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829942.068897:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829942.964779:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000400:00000100:1.0:1787829943.028926:0:2372:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.114:988 00000400:00000100:1.0:1787829943.028966:0:2372:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.114@tcp at host 192.168.203.114 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1787829943.028984:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:1.0:1787829943.029018:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:1.0:1787829943.029032:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000100:00100000:2.0:1787829943.029056:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829939/real 1787829943] req@ffff985607df5680 x1874669228427392/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829964 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829943.029078:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5680 x1874669228427392/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829964 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829943.029088:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829943.029093:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787829943.029097:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228427392:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787829943.029103:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829939/real 1787829943] req@ffff985607df43c0 x1874669228427136/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829964 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829943.029114:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df43c0 x1874669228427136/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829964 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829943.029121:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829943.029125:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787829943.029128:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228427136:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787829943.029134:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829939/real 1787829943] req@ffff985607df61c0 x1874669228427264/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829964 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829943.029144:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df61c0 x1874669228427264/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829964 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829943.029151:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829943.029154:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787829943.029157:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228427264:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829943.092836:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829943.988822:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787829944.116894:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829945.012863:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787829945.076817:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829945.076826:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829945.076831:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829945.076842:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6090 00000100:00080000:0.0:1787829945.076857:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829945.076866:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787829945.076912:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5e00 x1874669228427520/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829945.076935:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829945.076939:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829945.076941:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829945.076946:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6090 00000100:00080000:0.0:1787829945.076949:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829945.076954:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787829945.076965:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829945.076977:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228427520:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829945.076983:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6d00 x1874669228427648/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829945.077003:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829945.077007:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829945.077008:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829945.077013:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6090 00000100:00080000:0.0:1787829945.077017:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829945.077021:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787829945.077029:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829945.077042:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4f00 x1874669228427776/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829945.077070:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787829945.077105:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829945/real 1787829945] req@ffff985607df5e00 x1874669228427520/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829970 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829945.077129:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5e00 x1874669228427520/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829970 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829945.077140:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829945.077143:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787829945.077147:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228427520:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787829945.077171:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228427648:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787829945.077203:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787829945.077214:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829945/real 1787829945] req@ffff985607df6d00 x1874669228427648/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829970 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829945.077224:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6d00 x1874669228427648/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829970 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829945.077231:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829945.077234:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787829945.077237:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228427648:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787829945.077249:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228427776:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787829945.077265:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787829945.077273:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829945/real 1787829945] req@ffff985607df4f00 x1874669228427776/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829970 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829945.077283:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4f00 x1874669228427776/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829970 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829945.077294:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829945.077301:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787829945.077304:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228427776:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829945.140830:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829946.036824:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829946.164814:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829947.060833:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829947.188800:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829948.084827:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829948.212834:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829949.108841:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787829949.236842:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829950.132779:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787829950.196784:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829950.196793:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829950.196797:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829950.196806:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6095 00000100:00080000:0.0:1787829950.196816:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829950.196822:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787829950.196852:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7840 x1874669228427904/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829950.196873:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829950.196877:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829950.196878:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829950.196882:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6095 00000100:00080000:0.0:1787829950.196885:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829950.196888:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787829950.196895:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829950.196906:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df52c0 x1874669228428032/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829950.196917:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829950.196920:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829950.196922:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1787829950.196925:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228427904:192.168.203.114@tcp:250:kworker.0 00000100:00080000:0.0:1787829950.196926:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6095 00000100:00080000:0.0:1787829950.196931:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829950.196935:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787829950.196940:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829950.196950:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7480 x1874669228428160/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829950.196991:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829950.197088:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829950/real 1787829950] req@ffff985607df7840 x1874669228427904/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829975 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829950.197109:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7840 x1874669228427904/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829975 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829950.197118:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829950.197121:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829950.197124:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228427904:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787829950.197144:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228428032:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829950.197269:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829950.197337:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829950/real 1787829950] req@ffff985607df52c0 x1874669228428032/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829975 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829950.197349:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df52c0 x1874669228428032/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829975 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829950.197356:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829950.197359:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829950.197361:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228428032:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787829950.197366:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228428160:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829950.197382:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829950.197390:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829950/real 1787829950] req@ffff985607df7480 x1874669228428160/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829975 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829950.197397:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7480 x1874669228428160/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829975 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829950.197403:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829950.197405:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829950.197407:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228428160:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829950.260837:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829951.156803:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829951.284829:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829952.180793:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829952.308837:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829953.204799:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829953.332829:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829954.228803:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787829954.356918:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829955.252863:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787829955.316866:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829955.316877:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829955.316881:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829955.316891:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6100 00000100:00080000:2.0:1787829955.316902:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829955.316909:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787829955.316948:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8f00 x1874669228428288/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829955.316973:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829955.316975:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829955.316980:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829955.316983:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6100 00000100:00080000:2.0:1787829955.316986:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829955.316988:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787829955.316998:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829955.317006:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140dad00 x1874669228428416/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829955.317013:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829955.317016:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829955.317017:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829955.317019:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6100 00000100:00080000:2.0:1787829955.317020:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829955.317024:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787829955.317028:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228428288:192.168.203.114@tcp:250:kworker.0 00000008:00100000:2.0:1787829955.317029:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829955.317039:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db0c0 x1874669228428544/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829955.317096:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829955.317130:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829955/real 1787829955] req@ffff9856140d8f00 x1874669228428288/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829980 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829955.317143:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8f00 x1874669228428288/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829980 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829955.317151:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829955.317154:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829955.317158:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228428288:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787829955.317164:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228428416:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829955.317184:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829955.317196:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829955/real 1787829955] req@ffff9856140dad00 x1874669228428416/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829980 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829955.317209:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140dad00 x1874669228428416/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829980 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829955.317216:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829955.317218:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829955.317220:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228428416:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787829955.317246:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228428544:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829955.317259:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829955.317266:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829955/real 1787829955] req@ffff9856140db0c0 x1874669228428544/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829980 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829955.317276:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db0c0 x1874669228428544/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829980 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829955.317283:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829955.317286:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829955.317289:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228428544:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829955.380827:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829956.276819:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829956.404872:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829957.300843:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829957.428835:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829958.324843:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829958.452869:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829959.348780:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787829959.476871:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829960.372797:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787829960.436825:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829960.436835:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829960.436840:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829960.436853:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6106 00000100:00080000:0.0:1787829960.436869:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829960.436878:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787829960.436927:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4b40 x1874669228428672/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829960.436961:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829960.436964:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829960.436967:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829960.436971:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6106 00000100:00080000:0.0:1787829960.436974:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829960.436978:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787829960.436990:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829960.437005:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6940 x1874669228428800/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829960.437015:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829960.437018:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829960.437020:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829960.437023:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6106 00000100:00080000:0.0:1787829960.437026:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829960.437031:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787829960.437034:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228428672:192.168.203.114@tcp:250:kworker.0 00000008:00100000:0.0:1787829960.437037:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829960.437052:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4780 x1874669228428928/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829960.437134:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829960.437181:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829960/real 1787829960] req@ffff985607df4b40 x1874669228428672/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829985 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829960.437202:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4b40 x1874669228428672/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829985 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829960.437213:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829960.437218:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829960.437222:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228428672:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787829960.437230:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228428800:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829960.437260:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829960.437273:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829960/real 1787829960] req@ffff985607df6940 x1874669228428800/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829985 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829960.437282:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6940 x1874669228428800/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829985 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829960.437288:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829960.437290:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829960.437292:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228428800:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787829960.437320:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228428928:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829960.437338:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829960.437349:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829960/real 1787829960] req@ffff985607df4780 x1874669228428928/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829985 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829960.437359:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4780 x1874669228428928/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829985 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829960.437374:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829960.437377:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829960.437380:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228428928:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829960.501647:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829961.396824:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829961.525134:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829962.420835:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829962.548856:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829963.444845:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829963.573969:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829964.468835:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787829964.596905:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829965.492818:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787829965.556762:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829965.556774:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829965.556779:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829965.556792:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6111 00000100:00080000:0.0:1787829965.556806:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829965.556815:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787829965.556862:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7c00 x1874669228429056/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829965.556893:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829965.556901:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829965.556903:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829965.556908:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6111 00000100:00080000:0.0:1787829965.556912:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829965.556916:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787829965.556927:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829965.556930:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228429056:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829965.556939:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4000 x1874669228429184/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829965.556949:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829965.556952:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829965.556953:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829965.556956:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6111 00000100:00080000:0.0:1787829965.556959:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829965.556962:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787829965.556967:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829965.556976:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4780 x1874669228429312/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829965.557011:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829965.557044:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829965/real 1787829965] req@ffff985607df7c00 x1874669228429056/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829990 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829965.557058:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7c00 x1874669228429056/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829990 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829965.557065:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829965.557068:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829965.557071:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228429056:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787829965.557093:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228429184:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829965.557123:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829965.557137:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829965/real 1787829965] req@ffff985607df4000 x1874669228429184/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829990 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829965.557149:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4000 x1874669228429184/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829990 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829965.557158:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829965.557160:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829965.557162:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228429184:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787829965.557167:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228429312:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829965.557187:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829965.557198:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829965/real 1787829965] req@ffff985607df4780 x1874669228429312/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829990 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829965.557210:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4780 x1874669228429312/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829990 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829965.557218:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829965.557221:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829965.557224:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228429312:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829965.621010:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829966.516934:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829966.644869:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829967.540816:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829967.668872:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829968.564847:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829968.693086:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829969.589068:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787829969.717413:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829970.612827:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787829970.676832:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829970.676849:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829970.676857:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829970.676870:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6116 00000100:00080000:2.0:1787829970.676881:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829970.676888:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787829970.676928:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140da940 x1874669228429440/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829970.676950:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829970.676953:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829970.676954:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829970.676957:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6116 00000100:00080000:2.0:1787829970.676959:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829970.676962:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787829970.676972:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829970.676981:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d9a40 x1874669228429568/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829970.676989:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228429440:192.168.203.114@tcp:250:kworker.0 00000100:00080000:2.0:1787829970.676990:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829970.676996:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829970.676998:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829970.677002:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6116 00000100:00080000:2.0:1787829970.677004:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829970.677006:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787829970.677009:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829970.677018:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8b40 x1874669228429696/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829970.677071:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829970.677106:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829970/real 1787829970] req@ffff9856140da940 x1874669228429440/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829995 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829970.677120:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140da940 x1874669228429440/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829995 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829970.677130:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829970.677133:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829970.677136:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228429440:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787829970.677158:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228429568:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829970.677173:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829970.677180:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829970/real 1787829970] req@ffff9856140d9a40 x1874669228429568/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829995 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829970.677188:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d9a40 x1874669228429568/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829995 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829970.677193:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829970.677195:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829970.677197:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228429568:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787829970.677200:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228429696:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829970.677209:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829970.677214:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829970/real 1787829970] req@ffff9856140d8b40 x1874669228429696/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829995 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829970.677221:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8b40 x1874669228429696/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829995 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829970.677226:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829970.677227:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829970.677229:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228429696:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829970.740899:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829971.636926:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829971.764876:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829972.660925:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829972.788848:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829973.684852:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829973.812878:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829974.708813:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787829974.836925:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829975.732797:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787829975.796865:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829975.796879:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829975.796885:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829975.796899:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6121 00000100:00080000:2.0:1787829975.796913:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829975.796922:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787829975.796971:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d9680 x1874669228429824/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829975.797006:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829975.797011:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829975.797013:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829975.797018:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6121 00000100:00080000:2.0:1787829975.797022:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829975.797027:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787829975.797040:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829975.797056:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d83c0 x1874669228429952/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829975.797071:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228429824:192.168.203.114@tcp:250:kworker.0 00000100:00080000:2.0:1787829975.797074:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829975.797099:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829975.797107:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829975.797114:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6121 00000100:00080000:2.0:1787829975.797119:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829975.797126:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787829975.797131:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829975.797149:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db480 x1874669228430080/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829975.797169:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829975.797210:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829975/real 1787829975] req@ffff9856140d9680 x1874669228429824/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830000 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829975.797227:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d9680 x1874669228429824/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830000 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829975.797238:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829975.797241:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829975.797245:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228429824:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787829975.797269:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228429952:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829975.797289:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829975.797319:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829975/real 1787829975] req@ffff9856140d83c0 x1874669228429952/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830000 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829975.797332:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d83c0 x1874669228429952/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830000 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829975.797340:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829975.797343:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829975.797346:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228429952:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787829975.797353:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228430080:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829975.797370:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829975.797379:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829975/real 1787829975] req@ffff9856140db480 x1874669228430080/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830000 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829975.797390:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db480 x1874669228430080/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830000 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829975.797396:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829975.797400:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829975.797403:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228430080:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829975.860975:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829976.756836:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829976.884856:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829977.780845:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829977.909065:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829978.804881:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829978.932836:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829979.828847:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787829979.957046:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829980.852823:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787829980.917106:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829980.917117:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829980.917122:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829980.917134:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6126 00000100:00080000:0.0:1787829980.917146:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829980.917155:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787829980.917199:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6940 x1874669228430208/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829980.917229:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829980.917232:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829980.917234:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829980.917238:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6126 00000100:00080000:0.0:1787829980.917241:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829980.917245:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787829980.917255:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829980.917277:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4b40 x1874669228430336/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829980.917293:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829980.917297:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829980.917299:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829980.917302:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6126 00000100:00080000:0.0:1787829980.917306:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829980.917309:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787829980.917313:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829980.917322:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7480 x1874669228430464/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829980.918852:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228430208:192.168.203.114@tcp:250:kworker.0 00000800:00000100:3.0:1787829980.918917:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829980.918951:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829980/real 1787829980] req@ffff985607df6940 x1874669228430208/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830005 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829980.918968:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6940 x1874669228430208/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830005 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829980.918978:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829980.918982:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829980.918986:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228430208:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787829980.918994:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228430336:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829980.919011:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829980.919022:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829980/real 1787829980] req@ffff985607df4b40 x1874669228430336/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830005 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829980.919033:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4b40 x1874669228430336/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830005 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829980.919053:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829980.919057:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829980.919060:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228430336:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787829980.919066:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228430464:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829980.919082:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829980.919091:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829980/real 1787829980] req@ffff985607df7480 x1874669228430464/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830005 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829980.919101:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7480 x1874669228430464/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830005 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829980.919108:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829980.919111:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829980.919114:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228430464:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829980.980967:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829981.876810:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829982.005076:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829982.900829:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829983.028917:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829983.924826:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829984.052828:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829984.948837:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787829985.076942:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829985.972857:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787829986.037029:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829986.037038:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829986.037044:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829986.037057:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6131 00000100:00080000:2.0:1787829986.037070:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829986.037079:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787829986.037130:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8780 x1874669228430592/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829986.037168:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829986.037173:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829986.037175:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829986.037179:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6131 00000100:00080000:2.0:1787829986.037182:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829986.037187:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787829986.037198:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829986.037212:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db480 x1874669228430720/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829986.037223:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829986.037238:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829986.037240:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829986.037245:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6131 00000100:00080000:2.0:1787829986.037250:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829986.037254:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787829986.037261:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829986.037278:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228430592:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787829986.037279:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d83c0 x1874669228430848/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787829986.037365:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829986.037398:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829986/real 1787829986] req@ffff9856140d8780 x1874669228430592/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830011 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829986.037413:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8780 x1874669228430592/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830011 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829986.037424:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829986.037428:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829986.037432:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228430592:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787829986.037440:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228430720:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829986.037465:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829986.037481:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829986/real 1787829986] req@ffff9856140db480 x1874669228430720/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830011 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829986.037493:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db480 x1874669228430720/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830011 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829986.037501:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829986.037504:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829986.037507:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228430720:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787829986.037530:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228430848:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787829986.037563:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787829986.037574:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829986/real 1787829986] req@ffff9856140d83c0 x1874669228430848/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830011 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829986.037586:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d83c0 x1874669228430848/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830011 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829986.037593:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829986.037597:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787829986.037600:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228430848:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829986.100849:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829986.996877:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829987.124827:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829988.023341:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829988.148879:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829989.044899:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829989.172873:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829990.068829:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787829990.196896:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829991.093150:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787829991.156949:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829991.156959:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829991.156962:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829991.156971:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6136 00000100:00080000:0.0:1787829991.156980:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829991.156986:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787829991.157020:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df52c0 x1874669228430976/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829991.157054:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829991.157064:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829991.157066:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829991.157070:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6136 00000100:00080000:0.0:1787829991.157074:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829991.157077:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787829991.157086:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829991.157098:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7840 x1874669228431104/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829991.157107:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829991.157110:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829991.157111:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829991.157114:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6136 00000100:00080000:0.0:1787829991.157118:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829991.157122:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787829991.157126:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829991.157134:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4f00 x1874669228431232/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829991.157276:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228430976:192.168.203.114@tcp:250:kworker.0 00000800:00000100:3.0:1787829991.157338:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829991.157367:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829991/real 1787829991] req@ffff985607df52c0 x1874669228430976/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830016 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829991.157383:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df52c0 x1874669228430976/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830016 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829991.157392:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829991.157395:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829991.157399:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228430976:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787829991.157405:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228431104:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829991.157423:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829991.157433:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829991/real 1787829991] req@ffff985607df7840 x1874669228431104/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830016 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829991.157443:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7840 x1874669228431104/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830016 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829991.157451:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829991.157454:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829991.157457:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228431104:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787829991.157462:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228431232:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829991.157476:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829991.157486:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829991/real 1787829991] req@ffff985607df4f00 x1874669228431232/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830016 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829991.157496:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4f00 x1874669228431232/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830016 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829991.157504:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829991.157507:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829991.157510:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228431232:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829991.220825:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829992.116792:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829992.244835:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829993.142606:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829993.268873:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829994.164847:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829994.292835:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829995.188802:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787829995.316872:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829996.212774:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787829996.276787:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829996.276796:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829996.276800:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829996.276809:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6141 00000100:00080000:0.0:1787829996.276821:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829996.276827:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787829996.276863:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6d00 x1874669228431360/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829996.276891:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829996.276895:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829996.276897:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829996.276901:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6141 00000100:00080000:0.0:1787829996.276904:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829996.276908:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787829996.276918:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829996.276931:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5e00 x1874669228431488/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829996.276940:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1787829996.276941:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228431360:192.168.203.114@tcp:250:kworker.0 00000100:00080000:0.0:1787829996.276944:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829996.276948:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829996.276951:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6141 00000100:00080000:0.0:1787829996.276953:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829996.276955:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787829996.276959:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829996.276968:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df61c0 x1874669228431616/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829996.277010:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829996.277045:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829996/real 1787829996] req@ffff985607df6d00 x1874669228431360/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830021 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829996.277059:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6d00 x1874669228431360/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830021 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829996.277067:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829996.277070:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829996.277073:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228431360:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787829996.277095:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228431488:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829996.277109:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829996.277116:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829996/real 1787829996] req@ffff985607df5e00 x1874669228431488/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830021 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829996.277125:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5e00 x1874669228431488/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830021 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829996.277130:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829996.277139:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829996.277141:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228431488:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787829996.277145:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228431616:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787829996.277174:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787829996.277183:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829996/real 1787829996] req@ffff985607df61c0 x1874669228431616/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830021 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829996.277190:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df61c0 x1874669228431616/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830021 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829996.277195:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829996.277197:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787829996.277199:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228431616:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787829996.340849:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829997.236827:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829997.364819:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829998.260819:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829998.388857:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829999.284786:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787829999.412830:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830000.308820:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830000.436931:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830001.332803:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787830001.396867:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830001.396881:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787830001.396886:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830001.396900:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6147 00000100:00080000:3.0:1787830001.396914:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830001.396922:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787830001.396976:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f812c0 x1874669228431744/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830001.397009:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830001.397013:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787830001.397015:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830001.397020:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6147 00000100:00080000:3.0:1787830001.397024:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830001.397029:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830001.397040:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830001.397053:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228431744:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787830001.397056:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81680 x1874669228431872/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830001.397069:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830001.397071:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787830001.397073:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830001.397077:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6147 00000100:00080000:3.0:1787830001.397081:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830001.397086:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830001.397095:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830001.397108:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81e00 x1874669228432000/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787830001.397155:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830001.397197:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830001/real 1787830001] req@ffff985610f812c0 x1874669228431744/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830026 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830001.397215:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f812c0 x1874669228431744/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830026 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830001.397237:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830001.397242:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830001.397246:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228431744:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787830001.397278:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228431872:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830001.397302:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830001.397314:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830001/real 1787830001] req@ffff985610f81680 x1874669228431872/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830026 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830001.397333:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81680 x1874669228431872/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830026 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830001.397342:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830001.397345:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830001.397348:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228431872:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787830001.397354:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228432000:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830001.397373:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830001.397383:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830001/real 1787830001] req@ffff985610f81e00 x1874669228432000/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830026 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830001.397404:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81e00 x1874669228432000/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830026 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830001.397412:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830001.397416:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830001.397419:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228432000:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830001.460923:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830002.356841:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830002.484947:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830003.381041:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830003.508886:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830004.404854:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830004.534073:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830005.428924:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830005.557184:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830006.452816:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787830006.516811:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830006.516819:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787830006.516822:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830006.516829:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6152 00000100:00080000:3.0:1787830006.516837:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830006.516842:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787830006.516870:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83840 x1874669228432128/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830006.516892:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830006.516894:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787830006.516895:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830006.516897:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6152 00000100:00080000:3.0:1787830006.516900:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830006.516902:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830006.516909:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830006.516916:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f803c0 x1874669228432256/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830006.516922:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830006.516924:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787830006.516925:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830006.516927:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6152 00000100:00080000:3.0:1787830006.516928:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830006.516930:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830006.516933:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830006.516938:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82d00 x1874669228432384/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830006.517201:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228432128:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787830006.517336:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228432256:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787830006.517384:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228432384:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830006.580896:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830007.477223:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830007.604856:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830008.500803:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830008.628834:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830009.525233:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000400:00000100:2.0:1787830009.589031:0:2373:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.114:988 00000400:00000100:2.0:1787830009.589085:0:2373:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.114@tcp at host 192.168.203.114 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:2.0:1787830009.589100:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:2.0:1787830009.589135:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:2.0:1787830009.589143:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000100:00100000:0.0:1787830009.589226:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830006/real 1787830009] req@ffff985610f83840 x1874669228432128/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830031 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830009.589261:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83840 x1874669228432128/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830031 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830009.589274:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830009.589280:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830009.589285:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228432128:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787830009.589293:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830006/real 1787830009] req@ffff985610f803c0 x1874669228432256/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830031 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830009.589306:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f803c0 x1874669228432256/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830031 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830009.589314:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830009.589317:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830009.589321:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228432256:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787830009.589328:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830006/real 1787830009] req@ffff985610f82d00 x1874669228432384/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830031 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830009.589339:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82d00 x1874669228432384/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830031 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830009.589347:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830009.589351:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830009.589354:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228432384:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830009.652821:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830010.548812:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830010.677183:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830011.575153:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787830011.637107:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830011.637120:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830011.637125:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830011.637137:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6157 00000100:00080000:0.0:1787830011.637151:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830011.637158:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787830011.637198:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df43c0 x1874669228432512/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830011.637235:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830011.637239:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830011.637241:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830011.637245:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6157 00000100:00080000:0.0:1787830011.637248:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830011.637251:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787830011.637262:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830011.637272:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5680 x1874669228432640/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830011.637282:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830011.637285:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830011.637287:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830011.637291:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6157 00000100:00080000:0.0:1787830011.637294:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830011.637298:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787830011.637302:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830011.637310:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df70c0 x1874669228432768/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830011.637383:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228432512:192.168.203.114@tcp:250:kworker.0 00000800:00000100:3.0:1787830011.637473:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830011.637516:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830011/real 1787830011] req@ffff985607df43c0 x1874669228432512/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830036 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830011.637533:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df43c0 x1874669228432512/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830036 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830011.637545:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830011.637549:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830011.637554:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228432512:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787830011.637561:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228432640:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830011.637581:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830011.637592:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830011/real 1787830011] req@ffff985607df5680 x1874669228432640/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830036 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830011.637603:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5680 x1874669228432640/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830036 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830011.637611:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830011.637614:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830011.637617:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228432640:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787830011.637622:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228432768:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830011.637638:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830011.637647:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830011/real 1787830011] req@ffff985607df70c0 x1874669228432768/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830036 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830011.637658:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df70c0 x1874669228432768/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830036 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830011.637666:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830011.637669:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830011.637672:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228432768:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830011.700859:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830012.596886:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830012.726124:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830013.621121:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830013.748933:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830014.644856:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830014.774202:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830015.669503:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830015.796916:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830016.692894:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787830016.756868:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830016.756882:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830016.756887:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830016.756900:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6162 00000100:00080000:2.0:1787830016.756915:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830016.756924:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787830016.756986:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d9680 x1874669228432896/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830016.757030:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830016.757035:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830016.757037:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830016.757041:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6162 00000100:00080000:2.0:1787830016.757045:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830016.757049:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830016.757060:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830016.757074:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8b40 x1874669228433024/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830016.757090:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830016.757093:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830016.757095:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830016.757099:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6162 00000100:00080000:2.0:1787830016.757103:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830016.757107:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830016.757112:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830016.757123:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d9a40 x1874669228433152/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830016.757323:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228432896:192.168.203.114@tcp:250:kworker.0 00000800:00000100:3.0:1787830016.757399:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830016.757442:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830016/real 1787830016] req@ffff9856140d9680 x1874669228432896/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830041 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830016.757460:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d9680 x1874669228432896/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830041 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830016.757471:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830016.757475:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830016.757479:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228432896:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787830016.757485:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228433024:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830016.757509:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830016.757522:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830016/real 1787830016] req@ffff9856140d8b40 x1874669228433024/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830041 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830016.757535:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8b40 x1874669228433024/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830041 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830016.757544:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830016.757547:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830016.757551:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228433024:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787830016.757557:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228433152:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830016.757576:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830016.757588:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830016/real 1787830016] req@ffff9856140d9a40 x1874669228433152/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830041 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830016.757599:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d9a40 x1874669228433152/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830041 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830016.757606:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830016.757609:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830016.757612:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228433152:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830016.820931:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830017.716801:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830017.844879:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830018.741270:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830018.869033:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830019.764896:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830019.892882:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830020.790665:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830020.916856:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830021.814455:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787830021.876842:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830021.876856:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830021.876861:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830021.876874:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6167 00000100:00080000:2.0:1787830021.876887:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830021.876895:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787830021.876941:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140da940 x1874669228433280/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830021.876986:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830021.876990:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830021.876992:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830021.876996:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6167 00000100:00080000:2.0:1787830021.877000:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830021.877005:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830021.877016:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830021.877029:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db0c0 x1874669228433408/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830021.877038:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830021.877042:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830021.877043:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830021.877047:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6167 00000100:00080000:2.0:1787830021.877051:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830021.877056:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830021.877061:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830021.877071:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140dad00 x1874669228433536/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830021.877099:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228433280:192.168.203.114@tcp:250:kworker.0 00000800:00000100:3.0:1787830021.877163:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830021.877196:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830021/real 1787830021] req@ffff9856140da940 x1874669228433280/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830046 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830021.877211:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140da940 x1874669228433280/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830046 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830021.877219:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830021.877222:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830021.877224:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228433280:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787830021.877229:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228433408:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830021.877243:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830021.877250:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830021/real 1787830021] req@ffff9856140db0c0 x1874669228433408/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830046 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830021.877258:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db0c0 x1874669228433408/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830046 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830021.877263:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830021.877265:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830021.877267:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228433408:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787830021.877270:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228433536:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830021.877279:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830021.877284:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830021/real 1787830021] req@ffff9856140dad00 x1874669228433536/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830046 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830021.877291:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140dad00 x1874669228433536/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830046 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830021.877300:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830021.877306:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830021.877308:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228433536:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830021.940900:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830022.836848:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830022.965064:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830023.861127:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830023.989152:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830024.884853:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830025.012844:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830025.908827:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830026.037151:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830026.932860:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787830026.996978:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830026.996987:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830026.996991:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830026.997001:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6172 00000100:00080000:2.0:1787830026.997371:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830026.997380:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787830026.997428:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8f00 x1874669228433664/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830026.997462:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830026.997466:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830026.997469:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830026.997473:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6172 00000100:00080000:2.0:1787830026.997477:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830026.997481:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830026.997492:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830026.997505:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140da1c0 x1874669228433792/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830026.997515:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830026.997518:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830026.997520:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830026.997524:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6172 00000100:00080000:2.0:1787830026.997528:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830026.997533:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830026.997539:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830026.997553:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db840 x1874669228433920/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830026.997553:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228433664:192.168.203.114@tcp:250:kworker.0 00000800:00000100:3.0:1787830026.997621:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830026.997653:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830026/real 1787830026] req@ffff9856140d8f00 x1874669228433664/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830051 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830026.997677:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8f00 x1874669228433664/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830051 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830026.997832:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830026.997837:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830026.997840:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228433664:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787830026.997846:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228433792:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830026.997867:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830026.997877:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830026/real 1787830026] req@ffff9856140da1c0 x1874669228433792/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830051 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830026.997885:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140da1c0 x1874669228433792/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830051 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830026.997891:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830026.997893:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830026.997895:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228433792:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787830026.997921:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228433920:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830026.997932:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830026.997938:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830026/real 1787830026] req@ffff9856140db840 x1874669228433920/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830051 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830026.997961:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db840 x1874669228433920/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830051 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830026.997967:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830026.997969:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830026.997971:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228433920:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830027.060879:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830027.956842:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830028.084882:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830028.980848:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830029.108873:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830030.004854:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830030.132795:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830031.029244:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830031.156985:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830032.053070:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787830032.117005:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830032.117015:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830032.117019:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830032.117028:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6177 00000100:00080000:2.0:1787830032.117038:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830032.117045:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787830032.117079:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140dbc00 x1874669228434048/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830032.117106:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830032.117109:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830032.117111:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830032.117114:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6177 00000100:00080000:2.0:1787830032.117117:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830032.117121:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830032.117130:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830032.117142:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db840 x1874669228434176/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830032.117153:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228434048:192.168.203.114@tcp:250:kworker.0 00000100:00080000:2.0:1787830032.117155:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830032.117163:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830032.117164:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830032.117169:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6177 00000100:00080000:2.0:1787830032.117172:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830032.117176:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830032.117181:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830032.117192:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140da1c0 x1874669228434304/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787830032.117229:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830032.117260:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830032/real 1787830032] req@ffff9856140dbc00 x1874669228434048/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830057 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830032.117278:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140dbc00 x1874669228434048/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830057 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830032.117299:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830032.117304:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830032.117316:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228434048:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787830032.117340:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228434176:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830032.117363:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830032.117388:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830032/real 1787830032] req@ffff9856140db840 x1874669228434176/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830057 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830032.117402:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db840 x1874669228434176/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830057 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830032.117408:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830032.117410:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830032.117412:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228434176:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787830032.117416:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228434304:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830032.117431:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830032.117451:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830032/real 1787830032] req@ffff9856140da1c0 x1874669228434304/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830057 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830032.117463:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140da1c0 x1874669228434304/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830057 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830032.117470:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830032.117473:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830032.117476:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228434304:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830032.181068:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830033.076873:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830033.204925:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830034.100983:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830034.228887:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830035.124828:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830035.252826:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830036.148834:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830036.277458:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830037.172751:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787830037.236815:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830037.236823:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830037.236826:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830037.236832:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6182 00000100:00080000:2.0:1787830037.236841:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830037.236845:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787830037.236888:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8f00 x1874669228434432/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830037.236914:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830037.236918:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830037.236919:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830037.236923:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6182 00000100:00080000:2.0:1787830037.236927:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830037.236931:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830037.236939:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830037.236950:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140dad00 x1874669228434560/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830037.236971:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830037.236974:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830037.236976:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830037.236980:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6182 00000100:00080000:2.0:1787830037.236983:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830037.236987:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830037.236995:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830037.237009:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228434432:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787830037.237009:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db0c0 x1874669228434688/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787830037.237109:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830037.237153:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830037/real 1787830037] req@ffff9856140d8f00 x1874669228434432/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830062 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830037.237178:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8f00 x1874669228434432/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830062 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830037.237191:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830037.237195:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830037.237199:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228434432:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787830037.237207:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228434560:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830037.237228:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830037.237240:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830037/real 1787830037] req@ffff9856140dad00 x1874669228434560/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830062 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830037.237252:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140dad00 x1874669228434560/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830062 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830037.237259:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830037.237262:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830037.237266:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228434560:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787830037.237298:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228434688:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830037.237316:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830037.237326:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830037/real 1787830037] req@ffff9856140db0c0 x1874669228434688/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830062 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830037.237337:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db0c0 x1874669228434688/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830062 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830037.237350:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830037.237353:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830037.237356:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228434688:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830037.300869:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830038.196854:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830038.324837:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830039.220826:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830039.348889:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830040.244990:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830040.373013:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830041.270190:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830041.397071:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830042.292849:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787830042.356916:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830042.356929:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787830042.356935:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830042.356948:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6187 00000100:00080000:3.0:1787830042.356962:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830042.356971:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787830042.357021:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82940 x1874669228434816/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830042.357058:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830042.357061:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787830042.357063:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830042.357067:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6187 00000100:00080000:3.0:1787830042.357070:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830042.357075:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830042.357085:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830042.357098:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82580 x1874669228434944/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830042.357106:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830042.357110:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787830042.357111:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830042.357115:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6187 00000100:00080000:3.0:1787830042.357119:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830042.357122:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830042.357128:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830042.357150:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80f00 x1874669228435072/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830042.357356:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228434816:192.168.203.114@tcp:250:kworker.0 00000800:00000100:0.0:1787830042.357449:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830042.357497:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830042/real 1787830042] req@ffff985610f82940 x1874669228434816/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830067 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830042.357516:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82940 x1874669228434816/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830067 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830042.357527:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830042.357532:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830042.357537:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228434816:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787830042.357544:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228434944:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830042.357571:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830042.357584:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830042/real 1787830042] req@ffff985610f82580 x1874669228434944/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830067 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830042.357596:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82580 x1874669228434944/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830067 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830042.357611:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830042.357614:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830042.357617:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228434944:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787830042.357623:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228435072:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830042.357643:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830042.357664:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830042/real 1787830042] req@ffff985610f80f00 x1874669228435072/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830067 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830042.357677:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80f00 x1874669228435072/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830067 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830042.357684:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830042.357688:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830042.357721:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228435072:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830042.421868:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830043.316855:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830043.445070:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830044.340818:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830044.468885:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830045.364844:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830045.492880:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830046.389115:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830046.517028:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830047.412855:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787830047.476823:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830047.476838:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787830047.476853:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830047.476866:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6193 00000100:00080000:3.0:1787830047.476879:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830047.476888:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787830047.476934:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83480 x1874669228435200/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830047.476974:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830047.476978:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787830047.476980:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830047.476984:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6193 00000100:00080000:3.0:1787830047.476988:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830047.476993:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830047.477005:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830047.477014:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228435200:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787830047.477020:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f821c0 x1874669228435328/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830047.477031:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830047.477035:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787830047.477036:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830047.477040:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6193 00000100:00080000:3.0:1787830047.477044:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830047.477048:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830047.477052:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830047.477078:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80f00 x1874669228435456/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787830047.477092:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830047.477134:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830047/real 1787830047] req@ffff985610f83480 x1874669228435200/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830072 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830047.477152:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83480 x1874669228435200/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830072 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830047.477168:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830047.477172:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830047.477176:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228435200:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787830047.477204:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f821c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228435328:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830047.477227:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830047.477240:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830047/real 1787830047] req@ffff985610f821c0 x1874669228435328/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830072 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830047.477251:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f821c0 x1874669228435328/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830072 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830047.477264:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830047.477268:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830047.477271:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f821c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228435328:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787830047.477277:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228435456:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830047.477296:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830047.477306:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830047/real 1787830047] req@ffff985610f80f00 x1874669228435456/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830072 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830047.477317:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80f00 x1874669228435456/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830072 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830047.477323:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830047.477326:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830047.477328:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228435456:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830047.540825:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830048.436877:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830048.564892:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830049.460873:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830049.588905:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830050.484821:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830050.612928:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830051.508830:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830051.636845:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830052.532859:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787830052.596909:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830052.596922:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830052.596927:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830052.596940:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6198 00000100:00080000:0.0:1787830052.596955:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830052.596963:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787830052.597014:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5a40 x1874669228435584/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830052.597045:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830052.597049:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830052.597051:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830052.597055:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6198 00000100:00080000:0.0:1787830052.597059:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830052.597064:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787830052.597075:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830052.597089:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6580 x1874669228435712/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830052.597101:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1787830052.597102:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228435584:192.168.203.114@tcp:250:kworker.0 00000100:00080000:0.0:1787830052.597107:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830052.597112:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830052.597117:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6198 00000100:00080000:0.0:1787830052.597120:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830052.597124:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787830052.597129:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830052.597150:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df70c0 x1874669228435840/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787830052.597190:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830052.597233:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830052/real 1787830052] req@ffff985607df5a40 x1874669228435584/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830077 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830052.597252:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5a40 x1874669228435584/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830077 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830052.597263:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830052.597267:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830052.597271:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228435584:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787830052.597297:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228435712:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830052.597320:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830052.597332:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830052/real 1787830052] req@ffff985607df6580 x1874669228435712/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830077 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830052.597350:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6580 x1874669228435712/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830077 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830052.597359:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830052.597362:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830052.597366:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228435712:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787830052.597373:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228435840:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830052.597392:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830052.597403:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830052/real 1787830052] req@ffff985607df70c0 x1874669228435840/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830077 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830052.597413:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df70c0 x1874669228435840/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830077 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830052.597429:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830052.597432:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830052.597439:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228435840:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830052.660922:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830053.556802:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830053.684809:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830054.580803:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830054.708836:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830055.604907:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830055.732967:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830056.628889:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830056.757239:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830057.652872:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787830057.717941:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830057.717949:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787830057.717954:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830057.717963:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6203 00000100:00080000:3.0:1787830057.717975:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830057.717982:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787830057.718016:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82580 x1874669228435968/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830057.718083:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228435968:192.168.203.114@tcp:250:kworker.0 00000800:00000100:0.0:1787830057.718140:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00080000:3.0:1787830057.718170:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830057.718174:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787830057.718176:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830057.718181:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6203 00000100:00080000:3.0:1787830057.718184:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830057.718189:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830057.718198:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830057.718212:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82940 x1874669228436096/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830057.718433:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830057/real 1787830057] req@ffff985610f82580 x1874669228435968/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830082 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830057.718458:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82580 x1874669228435968/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830082 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830057.718469:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830057.718474:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830057.718479:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228435968:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787830057.718504:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228436096:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830057.718534:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830057.718548:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830057/real 1787830057] req@ffff985610f82940 x1874669228436096/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830082 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830057.718567:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82940 x1874669228436096/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830082 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830057.718574:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830057.718577:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830057.718581:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228436096:192.168.203.114@tcp:38:kworker.0 00000100:00080000:3.0:1787830057.720157:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830057.720161:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787830057.720163:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830057.720169:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6203 00000100:00080000:3.0:1787830057.720174:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830057.720179:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830057.720186:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830057.720204:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82d00 x1874669228436224/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787830057.720246:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228436224:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830057.720280:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830057.720425:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830057/real 1787830057] req@ffff985610f82d00 x1874669228436224/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830082 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830057.720439:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82d00 x1874669228436224/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830082 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830057.720448:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830057.720451:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830057.720455:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228436224:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830057.780887:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830058.677780:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830058.804884:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830059.700785:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830059.828897:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830060.724935:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830060.854058:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830061.748873:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830061.879763:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830062.772903:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787830062.837010:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830062.837024:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830062.837029:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830062.837041:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6208 00000100:00080000:1.0:1787830062.837053:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830062.837063:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787830062.837115:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a12c0 x1874669228436352/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830062.837185:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228436352:192.168.203.114@tcp:250:kworker.0 00000800:00000100:0.0:1787830062.837268:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00080000:1.0:1787830062.837300:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830062.837305:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830062.837307:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830062.837311:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6208 00000100:00080000:1.0:1787830062.837315:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830062.837320:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830062.837330:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830062.837344:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3480 x1874669228436480/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830062.837401:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830062/real 1787830062] req@ffff9856085a12c0 x1874669228436352/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830087 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830062.837420:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a12c0 x1874669228436352/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830087 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830062.837431:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830062.837436:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830062.837440:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228436352:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787830062.837469:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228436480:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830062.837494:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830062.837508:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830062/real 1787830062] req@ffff9856085a3480 x1874669228436480/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830087 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830062.837541:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3480 x1874669228436480/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830087 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830062.837550:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830062.837553:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830062.837556:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228436480:192.168.203.114@tcp:38:kworker.0 00000100:00080000:1.0:1787830062.837820:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830062.837825:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830062.837828:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830062.837832:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6208 00000100:00080000:1.0:1787830062.837837:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830062.837842:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830062.837848:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830062.837861:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a30c0 x1874669228436608/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830062.837906:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228436608:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830062.837932:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830062.837997:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830062/real 1787830062] req@ffff9856085a30c0 x1874669228436608/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830087 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830062.838009:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a30c0 x1874669228436608/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830087 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830062.838017:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830062.838020:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830062.838024:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228436608:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830062.901015:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830063.798885:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830063.925907:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830064.820854:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830064.948900:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830065.844866:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830065.973914:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830066.869382:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830066.997001:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830067.892856:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787830067.956813:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830067.956827:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830067.956832:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830067.956845:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6213 00000100:00080000:1.0:1787830067.956859:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830067.956867:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787830067.956915:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0f00 x1874669228436736/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830067.956953:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830067.956957:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830067.956959:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830067.956962:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6213 00000100:00080000:1.0:1787830067.956966:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830067.956970:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830067.956981:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830067.956995:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a30c0 x1874669228436864/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830067.957006:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830067.957009:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830067.957011:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830067.957015:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6213 00000100:00080000:1.0:1787830067.957019:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830067.957023:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830067.957027:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830067.957038:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3480 x1874669228436992/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787830067.957154:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228436736:192.168.203.114@tcp:250:kworker.0 00000800:00000100:2.0:1787830067.957232:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830067.957271:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830067/real 1787830067] req@ffff9856085a0f00 x1874669228436736/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830092 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830067.957287:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0f00 x1874669228436736/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830092 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830067.957297:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830067.957301:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830067.957305:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228436736:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787830067.957312:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228436864:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787830067.957334:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830067.957346:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830067/real 1787830067] req@ffff9856085a30c0 x1874669228436864/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830092 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830067.957356:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a30c0 x1874669228436864/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830092 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830067.957363:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830067.957366:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830067.957370:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228436864:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787830067.957375:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228436992:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787830067.957394:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830067.957405:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830067/real 1787830067] req@ffff9856085a3480 x1874669228436992/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830092 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830067.957415:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3480 x1874669228436992/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830092 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830067.957422:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830067.957424:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830067.957427:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228436992:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830068.020885:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830068.916845:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830069.044932:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830069.940896:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830070.068987:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830070.966440:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830071.093001:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830071.988858:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830072.117449:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830073.013132:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787830073.077458:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830073.077472:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830073.077478:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830073.077491:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6218 00000100:00080000:1.0:1787830073.077506:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830073.077516:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787830073.077565:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a12c0 x1874669228437120/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830073.077601:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830073.077605:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830073.077607:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830073.077611:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6218 00000100:00080000:1.0:1787830073.077614:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830073.077618:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830073.077639:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830073.077651:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2940 x1874669228437248/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830073.077667:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830073.077670:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830073.077672:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830073.077676:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6218 00000100:00080000:1.0:1787830073.077679:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830073.077683:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830073.077687:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830073.077718:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a21c0 x1874669228437376/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787830073.077781:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228437120:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787830073.077929:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228437248:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787830073.077966:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228437376:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830073.141615:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830074.036797:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830074.164846:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830075.060846:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830075.188927:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830076.085096:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000400:00000100:0.0:1787830076.149944:0:2370:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.114:988 00000400:00000100:0.0:1787830076.150031:0:2370:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.114@tcp at host 192.168.203.114 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:0.0:1787830076.150052:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:0.0:1787830076.150105:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:0.0:1787830076.150115:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000100:00100000:2.0:1787830076.151978:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830073/real 1787830076] req@ffff9856085a21c0 x1874669228437376/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830098 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830076.152018:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a21c0 x1874669228437376/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830098 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830076.152031:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830076.152037:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830076.152041:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228437376:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787830076.152050:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830073/real 1787830076] req@ffff9856085a12c0 x1874669228437120/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830098 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830076.152064:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a12c0 x1874669228437120/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830098 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830076.152073:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830076.152076:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830076.152079:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228437120:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787830076.152086:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830073/real 1787830076] req@ffff9856085a2940 x1874669228437248/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830098 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830076.152096:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2940 x1874669228437248/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830098 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830076.152105:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830076.152109:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830076.152113:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228437248:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830076.212919:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830077.109046:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830077.238676:0:92967:0:(ldlm_pool.c:1330:ldlm_pools_recalc_task()) Negative interval(0) 00010000:00010000:2.0:1787830077.238941:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830078.133211:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787830078.196942:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830078.196955:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787830078.196960:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830078.196972:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6223 00000100:00080000:3.0:1787830078.196987:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830078.196995:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787830078.197044:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f803c0 x1874669228437504/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830078.197080:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830078.197083:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787830078.197086:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830078.197090:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6223 00000100:00080000:3.0:1787830078.197095:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830078.197099:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830078.197111:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830078.197129:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83840 x1874669228437632/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830078.197139:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830078.197143:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787830078.197144:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830078.197148:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6223 00000100:00080000:3.0:1787830078.197152:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830078.197155:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830078.197164:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830078.197174:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81e00 x1874669228437760/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787830078.197394:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228437504:192.168.203.114@tcp:250:kworker.0 00000800:00000100:2.0:1787830078.197475:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830078.197513:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830078/real 1787830078] req@ffff985610f803c0 x1874669228437504/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830103 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830078.197532:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f803c0 x1874669228437504/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830103 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830078.197543:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830078.197548:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830078.197552:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228437504:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787830078.197559:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228437632:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787830078.198853:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830078.198903:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830078/real 1787830078] req@ffff985610f83840 x1874669228437632/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830103 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830078.198925:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83840 x1874669228437632/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830103 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830078.198936:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830078.198940:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830078.198944:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228437632:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787830078.198956:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228437760:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787830078.198999:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830078.199009:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830078/real 1787830078] req@ffff985610f81e00 x1874669228437760/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830103 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830078.199019:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81e00 x1874669228437760/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830103 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830078.199026:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830078.199028:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830078.199031:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228437760:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830078.261913:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830079.158148:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830079.284901:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830080.181002:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830080.309285:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830081.204995:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830081.332892:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830082.228823:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830082.358614:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830083.252954:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787830083.316872:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830083.316883:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787830083.316887:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830083.316896:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6228 00000100:00080000:3.0:1787830083.316906:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830083.316914:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787830083.316949:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81680 x1874669228437888/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787830083.317131:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228437888:192.168.203.114@tcp:250:kworker.0 00000800:00000100:2.0:1787830083.317186:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00080000:3.0:1787830083.317214:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830083.317218:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787830083.317220:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830083.317223:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6228 00000100:00080000:3.0:1787830083.317228:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830083.317231:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830083.317241:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830083.317253:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f812c0 x1874669228438016/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787830083.317642:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830083/real 1787830083] req@ffff985610f81680 x1874669228437888/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830108 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830083.319235:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830083.319242:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787830083.319254:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830083.319262:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6228 00000100:00080000:3.0:1787830083.319269:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830083.319275:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830083.319283:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830083.319314:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80b40 x1874669228438144/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787830083.319947:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81680 x1874669228437888/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830108 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830083.319958:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830083.319962:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830083.319966:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228437888:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787830083.319993:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228438016:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787830083.320052:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830083.320070:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830083/real 1787830083] req@ffff985610f812c0 x1874669228438016/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830108 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830083.320082:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f812c0 x1874669228438016/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830108 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830083.320090:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830083.320093:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830083.320096:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228438016:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787830083.320102:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228438144:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787830083.320122:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830083.320133:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830083/real 1787830083] req@ffff985610f80b40 x1874669228438144/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830108 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830083.320143:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80b40 x1874669228438144/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830108 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830083.320150:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830083.320153:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830083.320156:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228438144:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830083.380839:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830084.277043:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830084.404843:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830085.300937:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830085.429043:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830086.325896:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830086.453234:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830087.348810:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830087.477144:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830088.372799:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787830088.436806:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830088.436816:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787830088.436820:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830088.436829:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6234 00000100:00080000:3.0:1787830088.436841:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830088.436847:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787830088.436885:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80000 x1874669228438272/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830088.436914:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830088.436918:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787830088.436919:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830088.436923:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6234 00000100:00080000:3.0:1787830088.436927:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830088.436930:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830088.436939:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830088.436951:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81a40 x1874669228438400/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830088.436960:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830088.436963:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787830088.436965:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830088.436968:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6234 00000100:00080000:3.0:1787830088.436971:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830088.436975:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830088.436979:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830088.436988:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f830c0 x1874669228438528/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787830088.437185:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228438272:192.168.203.114@tcp:250:kworker.0 00000800:00000100:2.0:1787830088.437238:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830088.437267:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830088/real 1787830088] req@ffff985610f80000 x1874669228438272/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830113 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830088.437281:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80000 x1874669228438272/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830113 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830088.437291:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830088.437294:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830088.437298:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228438272:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787830088.437305:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228438400:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787830088.437326:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830088.437339:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830088/real 1787830088] req@ffff985610f81a40 x1874669228438400/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830113 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830088.437351:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81a40 x1874669228438400/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830113 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830088.437359:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830088.437362:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830088.437365:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228438400:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787830088.437370:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228438528:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787830088.437388:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830088.437399:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830088/real 1787830088] req@ffff985610f830c0 x1874669228438528/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830113 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830088.437417:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f830c0 x1874669228438528/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830113 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830088.437424:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830088.437427:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830088.437430:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228438528:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830088.502290:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830089.396897:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830089.527589:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830090.420930:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830090.548983:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830091.444804:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830091.572968:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830092.469735:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830092.597028:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830093.493070:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787830093.556934:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830093.556949:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830093.556954:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830093.556977:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6239 00000100:00080000:1.0:1787830093.556991:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830093.557002:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787830093.557049:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a1a40 x1874669228438656/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830093.557085:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830093.557089:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830093.557091:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830093.557095:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6239 00000100:00080000:1.0:1787830093.557099:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830093.557103:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830093.557115:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830093.557128:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2580 x1874669228438784/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830093.557137:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830093.557140:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830093.557142:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830093.557146:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6239 00000100:00080000:1.0:1787830093.557149:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830093.557154:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830093.557159:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830093.557169:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3840 x1874669228438912/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787830093.557326:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228438656:192.168.203.114@tcp:250:kworker.0 00000800:00000100:2.0:1787830093.557404:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830093.557446:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830093/real 1787830093] req@ffff9856085a1a40 x1874669228438656/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830118 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830093.557465:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a1a40 x1874669228438656/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830118 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830093.557477:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830093.557492:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830093.557498:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228438656:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787830093.557506:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228438784:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787830093.557603:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830093.557620:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830093/real 1787830093] req@ffff9856085a2580 x1874669228438784/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830118 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830093.557636:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2580 x1874669228438784/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830118 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830093.557645:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830093.557648:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830093.557653:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228438784:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787830093.557660:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228438912:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787830093.557683:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830093.557728:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830093/real 1787830093] req@ffff9856085a3840 x1874669228438912/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830118 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830093.557743:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3840 x1874669228438912/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830118 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830093.557752:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830093.557755:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830093.557759:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228438912:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830093.620979:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830094.516790:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830094.644967:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830095.540850:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830095.668885:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830096.564838:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830096.694660:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830097.588846:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830097.717131:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830098.612847:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787830098.676915:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830098.676924:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830098.676928:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830098.676937:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6244 00000100:00080000:2.0:1787830098.676947:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830098.676953:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787830098.676997:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140da940 x1874669228439040/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830098.677026:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830098.677030:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830098.677032:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830098.677036:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6244 00000100:00080000:2.0:1787830098.677039:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830098.677043:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830098.677051:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830098.677070:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d9a40 x1874669228439168/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830098.677080:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830098.677084:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830098.677086:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830098.677089:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6244 00000100:00080000:2.0:1787830098.677093:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830098.677096:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830098.677101:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830098.677109:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8b40 x1874669228439296/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830098.677188:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228439040:192.168.203.114@tcp:250:kworker.0 00000800:00000100:0.0:1787830098.677251:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830098.677283:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830098/real 1787830098] req@ffff9856140da940 x1874669228439040/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830123 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830098.677313:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140da940 x1874669228439040/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830123 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830098.677329:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830098.677333:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830098.677337:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228439040:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787830098.677344:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228439168:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830098.677368:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830098.677381:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830098/real 1787830098] req@ffff9856140d9a40 x1874669228439168/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830123 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830098.677394:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d9a40 x1874669228439168/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830123 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830098.677402:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830098.677406:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830098.677408:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228439168:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787830098.677414:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228439296:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830098.677434:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830098.677457:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830098/real 1787830098] req@ffff9856140d8b40 x1874669228439296/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830123 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830098.677469:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8b40 x1874669228439296/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830123 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830098.677477:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830098.677480:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830098.677483:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228439296:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830098.741346:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830099.636861:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830099.765027:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830100.660806:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830100.789064:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830101.685444:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830101.812956:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830102.708827:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830102.836993:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830103.732803:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787830103.797561:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830103.797575:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787830103.797579:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830103.797587:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6249 00000100:00080000:3.0:1787830103.797599:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830103.797604:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787830103.797643:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83c00 x1874669228439424/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830103.797669:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830103.797673:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787830103.797674:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830103.797678:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6249 00000100:00080000:3.0:1787830103.797682:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830103.797685:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830103.797718:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830103.797729:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80780 x1874669228439552/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830103.797735:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830103.797738:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787830103.797739:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830103.797742:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6249 00000100:00080000:3.0:1787830103.797746:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830103.797749:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830103.797755:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830103.797765:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f830c0 x1874669228439680/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830103.798924:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228439424:192.168.203.114@tcp:250:kworker.0 00000800:00000100:0.0:1787830103.799020:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830103.799067:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830103/real 1787830103] req@ffff985610f83c00 x1874669228439424/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830128 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830103.799087:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83c00 x1874669228439424/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830128 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830103.799099:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830103.799104:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830103.799109:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228439424:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787830103.799117:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228439552:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830103.799139:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830103.799221:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830103/real 1787830103] req@ffff985610f80780 x1874669228439552/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830128 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830103.799235:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80780 x1874669228439552/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830128 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830103.799242:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830103.799246:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830103.799249:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228439552:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787830103.799266:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228439680:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830103.799290:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830103.799301:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830103/real 1787830103] req@ffff985610f830c0 x1874669228439680/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830128 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830103.799313:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f830c0 x1874669228439680/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830128 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830103.799320:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830103.799324:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830103.799327:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228439680:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830103.860953:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830104.756871:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830104.885106:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830105.780846:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830105.908879:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830106.804844:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830106.933455:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830107.828842:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830107.957800:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830108.853218:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787830108.921806:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830108.921820:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830108.921825:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830108.921847:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6254 00000100:00080000:0.0:1787830108.921863:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830108.921872:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787830108.921924:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5680 x1874669228439808/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830108.921970:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830108.921973:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830108.921974:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830108.921978:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6254 00000100:00080000:0.0:1787830108.921986:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830108.921991:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787830108.922011:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830108.922024:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df43c0 x1874669228439936/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830108.922033:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830108.922036:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830108.922038:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830108.922041:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6254 00000100:00080000:0.0:1787830108.922044:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830108.922048:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787830108.922052:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830108.922063:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df61c0 x1874669228440064/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787830108.922096:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228439808:192.168.203.114@tcp:250:kworker.0 00000800:00000100:2.0:1787830108.922186:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830108.922228:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830108/real 1787830108] req@ffff985607df5680 x1874669228439808/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830133 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830108.922246:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5680 x1874669228439808/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830133 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830108.922257:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830108.922261:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830108.922265:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228439808:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787830108.922273:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228439936:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787830108.922295:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830108.922308:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830108/real 1787830108] req@ffff985607df43c0 x1874669228439936/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830133 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830108.922320:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df43c0 x1874669228439936/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830133 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830108.922334:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830108.922337:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830108.922341:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228439936:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787830108.922346:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228440064:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787830108.922366:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830108.922388:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830108/real 1787830108] req@ffff985607df61c0 x1874669228440064/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830133 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830108.922399:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df61c0 x1874669228440064/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830133 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830108.922406:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830108.922409:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830108.922412:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228440064:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830108.980936:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830109.876820:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830110.005821:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830110.902090:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830111.030216:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830111.924948:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830112.053303:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830112.949070:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830113.077027:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830113.972915:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787830114.036980:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830114.036995:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830114.037001:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830114.037024:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6259 00000100:00080000:0.0:1787830114.037039:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830114.037049:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787830114.037107:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5e00 x1874669228440192/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830114.037149:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830114.037153:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830114.037154:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830114.037159:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6259 00000100:00080000:0.0:1787830114.037171:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830114.037175:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787830114.037187:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830114.037200:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6d00 x1874669228440320/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830114.037208:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830114.037212:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830114.037214:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830114.037217:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6259 00000100:00080000:0.0:1787830114.037221:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830114.037225:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787830114.037229:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830114.037239:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4f00 x1874669228440448/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787830114.038386:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228440192:192.168.203.114@tcp:250:kworker.0 00000800:00000100:2.0:1787830114.038489:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830114.038535:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830114/real 1787830114] req@ffff985607df5e00 x1874669228440192/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830139 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830114.038554:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5e00 x1874669228440192/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830139 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830114.038567:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830114.038572:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830114.038577:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228440192:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787830114.038590:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228440320:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787830114.038619:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830114.038632:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830114/real 1787830114] req@ffff985607df6d00 x1874669228440320/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830139 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830114.038644:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6d00 x1874669228440320/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830139 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830114.038652:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830114.038655:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830114.038659:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228440320:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787830114.038665:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228440448:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787830114.038685:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830114.038732:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830114/real 1787830114] req@ffff985607df4f00 x1874669228440448/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830139 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830114.038744:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4f00 x1874669228440448/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830139 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830114.038752:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830114.038755:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830114.038758:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228440448:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830114.100929:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830114.996879:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830115.124919:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830116.020825:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830116.148889:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830117.044845:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830117.173027:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830118.068922:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830118.197036:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830119.092819:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787830119.156895:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830119.156904:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787830119.156908:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830119.156917:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6264 00000100:00080000:3.0:1787830119.156927:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830119.156946:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787830119.156992:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81a40 x1874669228440576/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830119.157023:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830119.157027:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787830119.157029:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830119.157033:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6264 00000100:00080000:3.0:1787830119.157037:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830119.157040:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830119.157049:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830119.157060:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80000 x1874669228440704/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830119.157080:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830119.157083:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787830119.157085:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830119.157089:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6264 00000100:00080000:3.0:1787830119.157092:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830119.157095:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830119.157100:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830119.157110:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80b40 x1874669228440832/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787830119.157398:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228440576:192.168.203.114@tcp:250:kworker.0 00000800:00000100:2.0:1787830119.157507:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830119.157547:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830119/real 1787830119] req@ffff985610f81a40 x1874669228440576/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830144 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830119.157563:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81a40 x1874669228440576/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830144 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830119.157572:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830119.157575:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830119.157579:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228440576:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787830119.157584:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228440704:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787830119.157602:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830119.157609:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830119/real 1787830119] req@ffff985610f80000 x1874669228440704/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830144 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830119.157616:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80000 x1874669228440704/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830144 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830119.157622:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830119.157624:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830119.157626:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228440704:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787830119.157629:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228440832:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787830119.157639:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830119.157644:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830119/real 1787830119] req@ffff985610f80b40 x1874669228440832/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830144 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830119.157651:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80b40 x1874669228440832/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830144 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830119.157656:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830119.157657:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830119.157659:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228440832:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830119.221676:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830120.117430:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830120.247886:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830121.140900:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830121.268936:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830122.164780:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830122.292899:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830123.189101:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830123.316952:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830124.212816:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787830124.276998:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830124.277012:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830124.277017:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830124.277031:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6269 00000100:00080000:2.0:1787830124.277046:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830124.277067:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787830124.277116:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d9680 x1874669228440960/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830124.277159:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830124.277163:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830124.277164:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830124.277168:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6269 00000100:00080000:2.0:1787830124.277172:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830124.277176:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830124.277187:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830124.277200:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d83c0 x1874669228441088/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830124.277210:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830124.277213:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830124.277215:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830124.277218:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6269 00000100:00080000:2.0:1787830124.277222:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830124.277225:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830124.277230:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830124.277240:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db480 x1874669228441216/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830124.278371:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228440960:192.168.203.114@tcp:250:kworker.0 00000800:00000100:0.0:1787830124.278462:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830124.280451:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830124/real 1787830124] req@ffff9856140d9680 x1874669228440960/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830149 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830124.280487:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d9680 x1874669228440960/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830149 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830124.280501:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830124.280507:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830124.280511:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228440960:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787830124.280525:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228441088:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830124.280590:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830124.280611:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830124/real 1787830124] req@ffff9856140d83c0 x1874669228441088/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830149 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830124.280623:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d83c0 x1874669228441088/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830149 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830124.280631:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830124.280634:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830124.280638:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228441088:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787830124.280645:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228441216:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830124.280667:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830124.280678:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830124/real 1787830124] req@ffff9856140db480 x1874669228441216/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830149 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830124.280689:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db480 x1874669228441216/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830149 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830124.280788:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830124.280791:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830124.280795:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228441216:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830124.340830:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830125.237101:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830125.365214:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830126.260951:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830126.389474:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830127.285053:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830127.413173:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830128.309678:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830128.437182:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830129.333331:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787830129.396888:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830129.396901:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830129.396907:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830129.396919:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6275 00000100:00080000:2.0:1787830129.396934:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830129.396942:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787830129.396988:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8780 x1874669228441344/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830129.397026:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830129.397030:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830129.397032:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830129.397037:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6275 00000100:00080000:2.0:1787830129.397040:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830129.397045:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830129.397056:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830129.397070:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db480 x1874669228441472/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830129.397081:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830129.397084:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830129.397086:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830129.397091:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6275 00000100:00080000:2.0:1787830129.397095:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830129.397099:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830129.397103:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830129.397114:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d83c0 x1874669228441600/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830129.397185:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228441344:192.168.203.114@tcp:250:kworker.0 00000800:00000100:0.0:1787830129.397277:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830129.397316:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830129/real 1787830129] req@ffff9856140d8780 x1874669228441344/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830154 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830129.397336:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8780 x1874669228441344/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830154 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830129.397348:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830129.397352:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830129.397357:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228441344:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787830129.397365:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228441472:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830129.397390:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830129.397405:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830129/real 1787830129] req@ffff9856140db480 x1874669228441472/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830154 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830129.397424:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db480 x1874669228441472/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830154 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830129.397431:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830129.397435:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830129.397438:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228441472:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787830129.397444:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228441600:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830129.397464:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830129.397477:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830129/real 1787830129] req@ffff9856140d83c0 x1874669228441600/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830154 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830129.397490:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d83c0 x1874669228441600/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830154 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830129.397498:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830129.397502:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830129.397506:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228441600:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830129.460895:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830130.357911:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830130.484908:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830131.380811:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830131.508958:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830132.404851:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830132.533181:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830133.428845:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830133.557814:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830134.452889:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787830134.517178:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830134.517193:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830134.517212:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830134.517227:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6280 00000100:00080000:1.0:1787830134.517242:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830134.517252:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787830134.517303:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a03c0 x1874669228441728/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830134.517336:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830134.517340:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830134.517342:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830134.517346:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6280 00000100:00080000:1.0:1787830134.517350:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830134.517355:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830134.517367:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830134.517380:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3c00 x1874669228441856/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830134.517389:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830134.517392:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830134.517394:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830134.517398:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6280 00000100:00080000:1.0:1787830134.517402:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830134.517405:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830134.517410:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830134.517420:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0780 x1874669228441984/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830134.517562:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228441728:192.168.203.114@tcp:250:kworker.0 00000800:00000100:0.0:1787830134.517644:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830134.517689:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830134/real 1787830134] req@ffff9856085a03c0 x1874669228441728/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830159 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830134.517745:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a03c0 x1874669228441728/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830159 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830134.517756:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830134.517761:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830134.517765:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228441728:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787830134.517772:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228441856:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830134.517798:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830134.517812:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830134/real 1787830134] req@ffff9856085a3c00 x1874669228441856/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830159 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830134.517825:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3c00 x1874669228441856/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830159 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830134.517834:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830134.517837:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830134.517841:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228441856:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787830134.517846:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228441984:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830134.517868:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830134.517879:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830134/real 1787830134] req@ffff9856085a0780 x1874669228441984/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830159 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830134.517892:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0780 x1874669228441984/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830159 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830134.517905:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830134.517909:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830134.517912:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228441984:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830134.580967:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830135.476819:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830135.605578:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830136.500928:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830136.628906:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830137.524942:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830137.652903:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830138.548838:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830138.677223:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830139.572820:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787830139.636895:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830139.636912:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830139.636917:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830139.636930:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6285 00000100:00080000:1.0:1787830139.636944:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830139.636953:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787830139.637000:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a1680 x1874669228442112/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830139.637034:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830139.637039:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830139.637041:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830139.637045:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6285 00000100:00080000:1.0:1787830139.637049:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830139.637053:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830139.637064:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830139.637076:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0b40 x1874669228442240/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830139.637086:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830139.637090:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830139.637092:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830139.637096:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6285 00000100:00080000:1.0:1787830139.637100:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830139.637103:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830139.637109:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830139.637125:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0000 x1874669228442368/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830139.637189:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228442112:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787830139.637306:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228442240:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787830139.637337:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228442368:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830139.700962:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830140.597261:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830140.725111:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830141.621548:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830141.748873:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830142.644895:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000400:00000100:1.0:1787830142.709050:0:2371:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.114:988 00000400:00000100:1.0:1787830142.709100:0:2371:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.114@tcp at host 192.168.203.114 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1787830142.709119:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:1.0:1787830142.709176:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:1.0:1787830142.709185:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000100:00100000:0.0:1787830142.709271:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830139/real 1787830142] req@ffff9856085a1680 x1874669228442112/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830164 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830142.709301:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a1680 x1874669228442112/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830164 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830142.709316:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830142.709321:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830142.709326:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228442112:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787830142.709335:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830139/real 1787830142] req@ffff9856085a0b40 x1874669228442240/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830164 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830142.709348:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0b40 x1874669228442240/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830164 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830142.709357:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830142.709361:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830142.709364:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228442240:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787830142.709371:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830139/real 1787830142] req@ffff9856085a0000 x1874669228442368/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830164 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830142.709383:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0000 x1874669228442368/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830164 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830142.709391:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830142.709394:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830142.709398:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228442368:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830142.772968:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830143.668911:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830143.796956:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830144.694086:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787830144.756823:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830144.756833:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787830144.756841:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830144.756851:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6290 00000100:00080000:3.0:1787830144.756860:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830144.756867:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787830144.756899:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f812c0 x1874669228442496/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830144.756936:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830144.756939:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787830144.756941:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830144.756945:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6290 00000100:00080000:3.0:1787830144.756949:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830144.756953:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830144.756963:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830144.756975:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81680 x1874669228442624/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830144.756984:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830144.756987:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787830144.756989:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830144.756992:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6290 00000100:00080000:3.0:1787830144.756995:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830144.756999:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830144.757003:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830144.757012:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81e00 x1874669228442752/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830144.757078:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228442496:192.168.203.114@tcp:250:kworker.0 00000800:00000100:0.0:1787830144.757145:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830144.757176:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830144/real 1787830144] req@ffff985610f812c0 x1874669228442496/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830169 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830144.757192:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f812c0 x1874669228442496/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830169 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830144.757201:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830144.757204:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830144.757208:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228442496:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787830144.757214:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228442624:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830144.757237:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830144.757248:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830144/real 1787830144] req@ffff985610f81680 x1874669228442624/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830169 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830144.757259:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81680 x1874669228442624/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830169 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830144.757266:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830144.757269:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830144.757272:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228442624:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787830144.757277:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228442752:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830144.757296:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830144.757306:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830144/real 1787830144] req@ffff985610f81e00 x1874669228442752/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830169 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830144.757316:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81e00 x1874669228442752/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830169 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830144.757322:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830144.757325:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830144.757328:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228442752:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830144.821176:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830145.716811:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830145.845066:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830146.740853:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830146.869067:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830147.764839:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830147.893168:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830148.790657:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830148.918193:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830149.813082:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787830149.877308:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830149.877320:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787830149.877326:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830149.877337:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6295 00000100:00080000:3.0:1787830149.877352:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830149.877361:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787830149.877407:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83840 x1874669228442880/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830149.877455:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830149.877459:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787830149.877461:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830149.877465:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6295 00000100:00080000:3.0:1787830149.877469:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830149.877473:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830149.877483:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830149.877495:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f803c0 x1874669228443008/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830149.877505:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830149.877508:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787830149.877510:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830149.877514:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6295 00000100:00080000:3.0:1787830149.877517:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830149.877521:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830149.877526:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830149.877535:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82d00 x1874669228443136/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830149.877666:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228442880:192.168.203.114@tcp:250:kworker.0 00000800:00000100:0.0:1787830149.877998:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830149.878042:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830149/real 1787830149] req@ffff985610f83840 x1874669228442880/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830174 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830149.878061:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83840 x1874669228442880/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830174 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830149.878072:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830149.878076:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830149.878081:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228442880:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787830149.878101:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228443008:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830149.878129:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830149.878143:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830149/real 1787830149] req@ffff985610f803c0 x1874669228443008/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830174 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830149.878154:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f803c0 x1874669228443008/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830174 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830149.878162:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830149.878165:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830149.878168:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228443008:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787830149.878174:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228443136:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830149.878195:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830149.878205:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830149/real 1787830149] req@ffff985610f82d00 x1874669228443136/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830174 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830149.878217:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82d00 x1874669228443136/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830174 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830149.878230:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830149.878233:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830149.878236:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228443136:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830149.941069:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830150.836839:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830150.965062:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830151.861042:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830151.988904:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830152.884843:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830153.013054:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830153.908823:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830154.037254:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830154.932846:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787830154.996826:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830154.996848:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830154.996853:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830154.996863:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6300 00000100:00080000:0.0:1787830154.996873:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830154.996881:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787830154.996918:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7840 x1874669228443264/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830154.996952:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830154.996956:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830154.996958:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830154.996961:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6300 00000100:00080000:0.0:1787830154.996965:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830154.996969:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787830154.996978:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830154.996989:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df52c0 x1874669228443392/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830154.997004:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830154.997007:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830154.997008:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830154.997011:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6300 00000100:00080000:0.0:1787830154.997015:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830154.997018:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787830154.997022:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830154.997031:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7480 x1874669228443520/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787830154.997938:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228443264:192.168.203.114@tcp:250:kworker.0 00000800:00000100:2.0:1787830154.997998:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830154.998028:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830154/real 1787830154] req@ffff985607df7840 x1874669228443264/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830179 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830154.998045:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7840 x1874669228443264/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830179 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830154.998072:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830154.998076:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830154.998080:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228443264:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787830154.998087:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228443392:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787830154.998108:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830154.998120:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830154/real 1787830154] req@ffff985607df52c0 x1874669228443392/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830179 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830154.998131:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df52c0 x1874669228443392/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830179 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830154.998139:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830154.998142:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830154.998145:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228443392:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787830154.998150:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228443520:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787830154.998175:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830154.998188:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830154/real 1787830154] req@ffff985607df7480 x1874669228443520/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830179 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830154.998199:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7480 x1874669228443520/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830179 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830154.998206:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830154.998209:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830154.998212:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228443520:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830155.060772:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830155.956946:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830156.084930:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830156.981143:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830157.109064:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830158.004844:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830158.133091:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830159.028866:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830159.157132:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830160.052854:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787830160.117223:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830160.117245:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830160.117251:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830160.117266:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6305 00000100:00080000:1.0:1787830160.117284:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830160.117295:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787830160.117359:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2d00 x1874669228443648/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830160.117408:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830160.117411:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830160.117413:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830160.117419:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6305 00000100:00080000:1.0:1787830160.117423:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830160.117437:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830160.117454:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830160.117472:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0000 x1874669228443776/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830160.117481:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830160.117485:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830160.117487:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830160.117491:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6305 00000100:00080000:1.0:1787830160.117495:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830160.117500:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830160.117505:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830160.117516:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0b40 x1874669228443904/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787830160.117647:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228443648:192.168.203.114@tcp:250:kworker.0 00000800:00000100:2.0:1787830160.117796:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830160.117843:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830160/real 1787830160] req@ffff9856085a2d00 x1874669228443648/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830185 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830160.117861:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2d00 x1874669228443648/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830185 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830160.117872:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830160.117876:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830160.117881:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228443648:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787830160.117890:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228443776:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787830160.117912:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830160.117924:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830160/real 1787830160] req@ffff9856085a0000 x1874669228443776/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830185 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830160.117936:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0000 x1874669228443776/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830185 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830160.117944:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830160.117947:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830160.117950:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228443776:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787830160.117955:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228443904:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787830160.117974:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830160.117985:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830160/real 1787830160] req@ffff9856085a0b40 x1874669228443904/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830185 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830160.117996:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0b40 x1874669228443904/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830185 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830160.118003:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830160.118007:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830160.118010:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228443904:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830160.180898:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830161.076862:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830161.204957:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830162.100865:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830162.230417:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830163.124818:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830163.254665:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830164.148925:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830164.277018:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830165.172805:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787830165.236850:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830165.236861:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830165.236866:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830165.236875:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6310 00000100:00080000:1.0:1787830165.236887:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830165.236895:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787830165.236943:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a1680 x1874669228444032/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830165.237025:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228444032:192.168.203.114@tcp:250:kworker.0 00000800:00000100:3.0:1787830165.237090:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00080000:1.0:1787830165.237123:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830165.237127:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830165.237129:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830165.237134:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6310 00000100:00080000:1.0:1787830165.237138:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830165.237143:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830165.237153:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830165.237168:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0780 x1874669228444160/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830165.237296:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830165/real 1787830165] req@ffff9856085a1680 x1874669228444032/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830190 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830165.237318:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a1680 x1874669228444032/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830190 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830165.237331:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830165.237336:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830165.237341:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228444032:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787830165.237382:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228444160:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830165.237407:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830165.237420:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830165/real 1787830165] req@ffff9856085a0780 x1874669228444160/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830190 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830165.237431:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0780 x1874669228444160/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830190 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830165.237438:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830165.237440:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830165.237443:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228444160:192.168.203.114@tcp:38:kworker.0 00000100:00080000:1.0:1787830165.238116:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830165.238122:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830165.238124:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830165.238130:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6310 00000100:00080000:1.0:1787830165.238135:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830165.238141:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830165.238150:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830165.238168:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3c00 x1874669228444288/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830165.238238:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228444288:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830165.238277:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830165.238292:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830165/real 1787830165] req@ffff9856085a3c00 x1874669228444288/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830190 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830165.238305:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3c00 x1874669228444288/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830190 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830165.238313:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830165.238317:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830165.238320:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228444288:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830165.300915:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830166.196959:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830166.327337:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830167.220863:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830167.352188:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830168.244842:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787830168.247065:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830168.247076:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830168.247080:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830168.247092:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6315 00000100:00080000:2.0:1787830168.247105:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830168.247113:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787830168.247160:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d9680 x1874669228444416/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830168.247198:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830168.247202:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830168.247204:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830168.247207:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6315 00000100:00080000:2.0:1787830168.247211:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830168.247215:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830168.247225:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830168.247238:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8b40 x1874669228444544/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830168.247247:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830168.247250:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830168.247252:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830168.247256:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6316 00000100:00080000:2.0:1787830168.247259:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830168.247263:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830168.247269:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830168.247279:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d9a40 x1874669228444672/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830168.247431:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228444416:192.168.203.114@tcp:250:kworker.0 00000800:00000100:3.0:1787830168.247509:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830168.247552:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830168/real 1787830168] req@ffff9856140d9680 x1874669228444416/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830193 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830168.247570:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d9680 x1874669228444416/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830193 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830168.247581:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830168.247586:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830168.247590:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228444416:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787830168.247596:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228444544:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830168.247630:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830168.247645:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830168/real 1787830168] req@ffff9856140d8b40 x1874669228444544/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830193 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830168.247657:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8b40 x1874669228444544/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830193 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830168.247665:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830168.247668:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830168.247671:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228444544:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787830168.247677:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228444672:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830168.247716:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830168.247728:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830168/real 1787830168] req@ffff9856140d9a40 x1874669228444672/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830193 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830168.247739:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d9a40 x1874669228444672/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830193 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830168.247747:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830168.247750:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830168.247753:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228444672:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830168.373455:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830169.268853:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830169.396985:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830170.292986:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830170.421041:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830171.316814:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830171.449446:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830172.340838:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830172.468926:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830173.364911:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787830173.429533:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830173.429544:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787830173.429549:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830173.429558:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6319 00000100:00080000:3.0:1787830173.429568:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830173.429575:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787830173.429612:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82940 x1874669228444800/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830173.429645:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830173.429649:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787830173.429651:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830173.429655:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6319 00000100:00080000:3.0:1787830173.429658:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830173.429662:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830173.429671:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830173.429682:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82580 x1874669228444928/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830173.429735:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830173.429740:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787830173.429742:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:2.0:1787830173.429756:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228444800:192.168.203.114@tcp:250:kworker.0 00000800:00000100:2.0:1787830173.429857:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830173.429909:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830173/real 1787830173] req@ffff985610f82940 x1874669228444800/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830198 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830173.429939:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82940 x1874669228444800/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830198 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830173.429951:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830173.429956:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830173.429960:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228444800:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787830173.429992:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228444928:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787830173.430018:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830173.430033:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830173/real 1787830173] req@ffff985610f82580 x1874669228444928/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830198 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830173.430045:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82580 x1874669228444928/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830198 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830173.430053:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830173.430056:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830173.430060:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228444928:192.168.203.114@tcp:38:kworker.0 00000100:00080000:3.0:1787830173.431232:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6319 00000100:00080000:3.0:1787830173.431241:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830173.431248:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830173.431259:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830173.431284:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80f00 x1874669228445056/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787830173.431402:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228445056:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787830173.431473:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830173.431500:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830173/real 1787830173] req@ffff985610f80f00 x1874669228445056/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830198 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830173.431516:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80f00 x1874669228445056/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830198 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830173.431527:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830173.431530:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830173.431534:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228445056:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830173.492818:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830174.388989:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830174.516885:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830175.412810:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830175.540890:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830176.436869:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830176.566337:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830177.460847:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830177.589000:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830178.485044:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787830178.549035:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830178.549049:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830178.549055:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830178.549068:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6324 00000100:00080000:0.0:1787830178.549083:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830178.549092:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787830178.549140:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4b40 x1874669228445184/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830178.549183:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830178.549187:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830178.549189:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830178.549202:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6324 00000100:00080000:0.0:1787830178.549207:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830178.549210:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787830178.549221:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830178.549233:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6940 x1874669228445312/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830178.549243:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830178.549246:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830178.549248:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830178.549252:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6324 00000100:00080000:0.0:1787830178.549255:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830178.549259:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787830178.549263:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830178.549273:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4780 x1874669228445440/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787830178.549417:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228445184:192.168.203.114@tcp:250:kworker.0 00000800:00000100:2.0:1787830178.549496:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830178.549537:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830178/real 1787830178] req@ffff985607df4b40 x1874669228445184/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830203 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830178.549555:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4b40 x1874669228445184/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830203 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830178.549566:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830178.549570:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830178.549574:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228445184:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787830178.549584:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228445312:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787830178.549610:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830178.549624:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830178/real 1787830178] req@ffff985607df6940 x1874669228445312/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830203 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830178.549635:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6940 x1874669228445312/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830203 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830178.549654:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830178.549657:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830178.549661:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228445312:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787830178.549667:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228445440:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787830178.549689:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830178.549729:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830178/real 1787830178] req@ffff985607df4780 x1874669228445440/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830203 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830178.549741:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4780 x1874669228445440/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830203 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830178.549749:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830178.549752:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830178.549756:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228445440:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830178.612924:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830179.509575:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830179.636944:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830180.532825:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830180.660903:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830181.556918:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830181.687683:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830182.580767:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830182.709076:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830183.604825:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787830183.668913:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830183.668927:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787830183.668932:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830183.668944:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6329 00000100:00080000:3.0:1787830183.668957:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830183.668965:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787830183.669012:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f821c0 x1874669228445568/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830183.669048:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830183.669052:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787830183.669054:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830183.669058:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6329 00000100:00080000:3.0:1787830183.669061:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830183.669065:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830183.669076:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830183.669091:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83480 x1874669228445696/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830183.669101:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830183.669104:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787830183.669106:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830183.669109:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6329 00000100:00080000:3.0:1787830183.669113:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830183.669117:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830183.669121:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830183.669133:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80f00 x1874669228445824/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787830183.669238:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f821c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228445568:192.168.203.114@tcp:250:kworker.0 00000800:00000100:2.0:1787830183.669318:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830183.669356:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830183/real 1787830183] req@ffff985610f821c0 x1874669228445568/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830208 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830183.669376:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f821c0 x1874669228445568/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830208 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830183.669386:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830183.669390:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830183.669394:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f821c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228445568:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787830183.669400:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228445696:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787830183.669424:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830183.669435:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830183/real 1787830183] req@ffff985610f83480 x1874669228445696/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830208 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830183.669446:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83480 x1874669228445696/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830208 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830183.669454:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830183.669458:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830183.669460:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228445696:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787830183.669466:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228445824:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787830183.669542:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830183.669553:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830183/real 1787830183] req@ffff985610f80f00 x1874669228445824/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830208 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830183.669571:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80f00 x1874669228445824/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830208 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830183.669578:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830183.669581:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830183.669584:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228445824:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830183.733012:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830184.628842:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830184.756914:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830185.652857:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830185.780834:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830186.677002:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830186.804957:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830187.700863:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830187.828879:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830188.724868:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787830188.789300:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830188.789308:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787830188.789312:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830188.789321:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6334 00000100:00080000:3.0:1787830188.789330:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830188.789336:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787830188.789369:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82580 x1874669228445952/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830188.789397:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830188.789401:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787830188.789402:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830188.789406:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6334 00000100:00080000:3.0:1787830188.789408:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830188.789412:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830188.789419:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830188.789428:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82940 x1874669228446080/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830188.789437:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830188.789439:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787830188.789441:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830188.789444:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6334 00000100:00080000:3.0:1787830188.789447:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830188.789449:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830188.789453:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830188.789459:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82d00 x1874669228446208/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787830188.790756:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228445952:192.168.203.114@tcp:250:kworker.0 00000800:00000100:2.0:1787830188.790865:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830188.790912:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830188/real 1787830188] req@ffff985610f82580 x1874669228445952/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830213 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830188.790931:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82580 x1874669228445952/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830213 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830188.790944:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830188.790948:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830188.790953:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228445952:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787830188.790961:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228446080:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787830188.790984:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830188.790996:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830188/real 1787830188] req@ffff985610f82940 x1874669228446080/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830213 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830188.791013:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82940 x1874669228446080/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830213 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830188.791020:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830188.791024:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830188.791026:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228446080:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787830188.791033:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228446208:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787830188.791053:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830188.791064:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830188/real 1787830188] req@ffff985610f82d00 x1874669228446208/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830213 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830188.791074:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82d00 x1874669228446208/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830213 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830188.791081:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830188.791084:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830188.791087:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228446208:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830188.852977:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830189.748934:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830189.877168:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830190.772830:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830190.900849:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830191.796886:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830191.926729:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830192.820891:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830192.949946:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830193.844920:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787830193.909396:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830193.909477:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830193.909481:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830193.909491:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6339 00000100:00080000:0.0:1787830193.909502:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830193.909510:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787830193.909557:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4000 x1874669228446336/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830193.909583:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830193.909587:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830193.909589:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830193.909593:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6339 00000100:00080000:0.0:1787830193.909608:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830193.909612:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787830193.909622:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830193.909634:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7c00 x1874669228446464/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830193.909645:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830193.909649:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830193.909650:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:2.0:1787830193.909663:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228446336:192.168.203.114@tcp:250:kworker.0 00000100:00080000:0.0:1787830193.909775:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6339 00000100:00080000:0.0:1787830193.909793:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000800:00000100:2.0:1787830193.909801:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00080000:0.0:1787830193.909805:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787830193.909813:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830193.909840:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4780 x1874669228446592/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787830193.909856:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830193/real 1787830193] req@ffff985607df4000 x1874669228446336/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830218 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830193.909887:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4000 x1874669228446336/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830218 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830193.909899:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830193.909904:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830193.909908:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228446336:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787830193.909939:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228446464:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787830193.909964:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830193.909977:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830193/real 1787830193] req@ffff985607df7c00 x1874669228446464/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830218 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830193.909990:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7c00 x1874669228446464/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830218 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830193.909997:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830193.910000:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830193.910004:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228446464:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787830193.910010:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228446592:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787830193.910031:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830193.910042:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830193/real 1787830193] req@ffff985607df4780 x1874669228446592/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830218 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830193.910054:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4780 x1874669228446592/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830218 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830193.910067:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830193.910071:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830193.910074:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228446592:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830193.972977:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830194.868973:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830194.999005:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830195.893170:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830196.020998:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830196.916867:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830197.045248:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830197.940856:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830198.068972:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830198.964909:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787830199.028881:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830199.028895:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830199.028900:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830199.028913:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6344 00000100:00080000:1.0:1787830199.028926:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830199.028935:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787830199.028986:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a03c0 x1874669228446720/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830199.029018:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830199.029021:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830199.029022:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830199.029025:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6344 00000100:00080000:1.0:1787830199.029028:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830199.029032:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830199.029041:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830199.029051:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3840 x1874669228446848/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830199.029058:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830199.029060:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830199.029061:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830199.029063:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6344 00000100:00080000:1.0:1787830199.029066:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830199.029069:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830199.029072:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830199.029079:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2580 x1874669228446976/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787830199.029104:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228446720:192.168.203.114@tcp:250:kworker.0 00000800:00000100:2.0:1787830199.029197:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830199.029237:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830199/real 1787830199] req@ffff9856085a03c0 x1874669228446720/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830224 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830199.029256:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a03c0 x1874669228446720/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830224 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830199.029267:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830199.029271:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830199.029275:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228446720:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787830199.029283:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228446848:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787830199.029308:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830199.029321:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830199/real 1787830199] req@ffff9856085a3840 x1874669228446848/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830224 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830199.029333:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3840 x1874669228446848/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830224 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830199.029346:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830199.029350:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830199.029353:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228446848:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787830199.029359:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228446976:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787830199.029379:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830199.029389:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830199/real 1787830199] req@ffff9856085a2580 x1874669228446976/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830224 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830199.029399:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2580 x1874669228446976/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830224 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830199.029407:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830199.029410:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830199.029413:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228446976:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830199.093322:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830199.988856:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830200.117497:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830201.012975:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830201.141002:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830202.036811:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830202.164877:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830203.062364:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830203.189006:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830204.084841:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787830204.149229:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830204.149244:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830204.149249:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830204.149261:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6349 00000100:00080000:2.0:1787830204.149276:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830204.149286:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787830204.149331:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140da940 x1874669228447104/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830204.149374:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830204.149378:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830204.149380:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830204.149384:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6349 00000100:00080000:2.0:1787830204.149387:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830204.149392:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830204.149402:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830204.149414:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db0c0 x1874669228447232/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830204.149423:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830204.149426:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830204.149428:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830204.149432:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6349 00000100:00080000:2.0:1787830204.149435:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830204.149439:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830204.149444:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830204.149453:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140dad00 x1874669228447360/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830204.149569:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228447104:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787830204.149682:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228447232:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787830204.149755:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228447360:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830204.212852:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830205.109135:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830205.237013:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830206.132855:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830206.260918:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830207.156846:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000400:00000100:1.0:1787830207.221106:0:2372:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.114:988 00000400:00000100:1.0:1787830207.221158:0:2372:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.114@tcp at host 192.168.203.114 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1787830207.221182:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000100:00100000:3.0:1787830207.221281:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830204/real 1787830207] req@ffff9856140da940 x1874669228447104/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830229 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830207.221313:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140da940 x1874669228447104/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830229 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830207.221326:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830207.221332:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830207.221336:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228447104:192.168.203.114@tcp:250:kworker.0 00000800:00000100:1.0:1787830207.221349:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:1.0:1787830207.221364:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000100:00100000:3.0:1787830207.221588:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830204/real 1787830207] req@ffff9856140db0c0 x1874669228447232/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830229 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830207.221606:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db0c0 x1874669228447232/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830229 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830207.221615:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830207.221619:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830207.221622:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228447232:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787830207.221633:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830204/real 1787830207] req@ffff9856140dad00 x1874669228447360/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830229 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830207.221647:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140dad00 x1874669228447360/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830229 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830207.221655:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830207.221659:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830207.221662:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228447360:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787830207.285380:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830208.180831:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830208.308874:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830209.204857:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787830209.269450:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830209.269459:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830209.269463:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830209.269472:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6354 00000100:00080000:2.0:1787830209.269482:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830209.269489:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787830209.269522:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8f00 x1874669228447488/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830209.269549:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830209.269553:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830209.269555:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830209.269558:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6354 00000100:00080000:2.0:1787830209.269562:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830209.269565:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830209.269574:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830209.269721:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140da1c0 x1874669228447616/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830209.269731:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830209.269734:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830209.269736:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830209.269739:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6354 00000100:00080000:2.0:1787830209.269745:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830209.269750:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830209.269755:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830209.269765:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db840 x1874669228447744/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830209.270476:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228447488:192.168.203.114@tcp:250:kworker.0 00000800:00000100:3.0:1787830209.270544:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830209.270581:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830209/real 1787830209] req@ffff9856140d8f00 x1874669228447488/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830234 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830209.270598:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8f00 x1874669228447488/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830234 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830209.270609:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830209.270613:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830209.270617:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228447488:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787830209.270624:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228447616:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830209.270645:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830209.270656:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830209/real 1787830209] req@ffff9856140da1c0 x1874669228447616/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830234 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830209.270666:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140da1c0 x1874669228447616/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830234 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830209.270686:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830209.270828:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830209.270833:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228447616:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787830209.270840:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228447744:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830209.270863:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830209.270874:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830209/real 1787830209] req@ffff9856140db840 x1874669228447744/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830234 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830209.270884:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db840 x1874669228447744/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830234 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830209.270891:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830209.270895:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830209.270897:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228447744:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830209.332850:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830210.228791:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830210.357252:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830211.253331:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830211.382406:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830212.276816:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830212.405385:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830213.301853:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830213.432137:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830214.324810:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787830214.388894:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830214.388903:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830214.388907:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830214.388916:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6360 00000100:00080000:0.0:1787830214.388926:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830214.388933:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787830214.388966:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6940 x1874669228447872/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830214.389004:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830214.389007:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830214.389009:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830214.389013:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6360 00000100:00080000:0.0:1787830214.389016:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830214.389019:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787830214.389028:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830214.389039:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4b40 x1874669228448000/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830214.389049:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830214.389052:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830214.389054:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830214.389057:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6360 00000100:00080000:0.0:1787830214.389061:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830214.389064:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787830214.389068:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830214.389087:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7480 x1874669228448128/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830214.389176:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228447872:192.168.203.114@tcp:250:kworker.0 00000800:00000100:3.0:1787830214.389229:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830214.389259:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830214/real 1787830214] req@ffff985607df6940 x1874669228447872/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830239 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830214.389286:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6940 x1874669228447872/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830239 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830214.389296:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830214.389301:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830214.389304:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228447872:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787830214.389311:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228448000:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830214.389330:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830214.389340:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830214/real 1787830214] req@ffff985607df4b40 x1874669228448000/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830239 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830214.389352:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4b40 x1874669228448000/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830239 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830214.389359:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830214.389362:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830214.389365:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228448000:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787830214.389371:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228448128:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830214.389387:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830214.389397:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830214/real 1787830214] req@ffff985607df7480 x1874669228448128/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830239 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830214.389408:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7480 x1874669228448128/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830239 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830214.389416:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830214.389419:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830214.389422:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228448128:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830214.452866:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830215.348898:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830215.478369:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830216.373092:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830216.501662:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830217.396973:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830217.524888:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830218.421045:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830218.550993:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830219.444876:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787830219.508883:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830219.508893:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830219.508897:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830219.508907:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6365 00000100:00080000:1.0:1787830219.508916:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830219.508924:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787830219.508958:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a1a40 x1874669228448256/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830219.509036:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228448256:192.168.203.114@tcp:250:kworker.0 00000800:00000100:3.0:1787830219.509092:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00080000:1.0:1787830219.509130:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830219.509134:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830219.509136:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830219.509140:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6365 00000100:00080000:1.0:1787830219.509144:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830219.509148:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830219.509157:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830219.509170:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2940 x1874669228448384/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830219.509272:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830219/real 1787830219] req@ffff9856085a1a40 x1874669228448256/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830244 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830219.509283:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830219.509287:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830219.509289:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830219.509292:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6365 00000100:00100000:3.0:1787830219.509293:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a1a40 x1874669228448256/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830244 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830219.509296:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830219.509300:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00080000:3.0:1787830219.509302:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000008:00100000:1.0:1787830219.509304:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00080000:3.0:1787830219.509306:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830219.509309:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228448256:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787830219.509315:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a12c0 x1874669228448512/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830219.509329:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228448384:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830219.509352:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830219.509371:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830219/real 1787830219] req@ffff9856085a2940 x1874669228448384/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830244 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830219.509380:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2940 x1874669228448384/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830244 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830219.509386:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830219.509388:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830219.509390:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228448384:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787830219.509394:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228448512:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830219.509407:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830219.509414:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830219/real 1787830219] req@ffff9856085a12c0 x1874669228448512/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830244 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830219.509420:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a12c0 x1874669228448512/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830244 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830219.509425:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830219.509426:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830219.509428:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228448512:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830219.572869:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830220.468862:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830220.596923:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830221.492927:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830221.620834:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830222.516934:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830222.645105:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830223.542293:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830223.669062:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830224.564928:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787830224.629289:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830224.629302:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830224.629307:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830224.629320:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6370 00000100:00080000:1.0:1787830224.629334:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830224.629343:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787830224.629400:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a21c0 x1874669228448640/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830224.629585:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830224.629590:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830224.629593:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830224.629602:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6370 00000100:00080000:1.0:1787830224.629611:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830224.629618:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830224.629629:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830224.629659:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3480 x1874669228448768/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830224.629669:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830224.629672:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830224.629674:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830224.629679:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6370 00000100:00080000:1.0:1787830224.629683:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787830224.629687:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228448640:192.168.203.114@tcp:250:kworker.0 00000100:00080000:1.0:1787830224.629689:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830224.629721:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830224.629733:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a30c0 x1874669228448896/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787830224.629802:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830224.629842:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830224/real 1787830224] req@ffff9856085a21c0 x1874669228448640/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830249 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830224.629860:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a21c0 x1874669228448640/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830249 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830224.629870:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830224.629875:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830224.629878:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228448640:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787830224.629885:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228448768:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830224.629904:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830224.629916:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830224/real 1787830224] req@ffff9856085a3480 x1874669228448768/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830249 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830224.629928:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3480 x1874669228448768/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830249 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830224.629943:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830224.629946:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830224.629949:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228448768:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787830224.629973:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228448896:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830224.629993:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830224.630004:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830224/real 1787830224] req@ffff9856085a30c0 x1874669228448896/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830249 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830224.630015:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a30c0 x1874669228448896/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830249 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830224.630022:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830224.630026:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830224.630028:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228448896:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830224.692831:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830225.590044:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830225.717851:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830226.612810:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787830226.740982:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830227.637096:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830227.766430:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830228.661349:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830228.789094:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830229.685774:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787830229.749035:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830229.749043:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830229.749047:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830229.749056:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6375 00000100:00080000:2.0:1787830229.749064:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830229.749071:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787830229.749103:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140dbc00 x1874669228449024/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830229.749134:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830229.749137:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830229.749139:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830229.749142:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6375 00000100:00080000:2.0:1787830229.749145:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830229.749148:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830229.749158:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830229.749170:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db840 x1874669228449152/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830229.749179:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830229.749182:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830229.749183:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830229.749187:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6375 00000100:00080000:2.0:1787830229.749190:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830229.749193:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830229.749197:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830229.749206:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140da1c0 x1874669228449280/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787830229.749251:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228449024:192.168.203.114@tcp:250:kworker.0 00000800:00000100:1.0:1787830229.749325:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830229.749357:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830229/real 1787830229] req@ffff9856140dbc00 x1874669228449024/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830254 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830229.749374:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140dbc00 x1874669228449024/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830254 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830229.749386:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830229.749390:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830229.749395:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228449024:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787830229.749403:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228449152:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830229.749429:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830229.749443:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830229/real 1787830229] req@ffff9856140db840 x1874669228449152/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830254 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830229.749456:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db840 x1874669228449152/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830254 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830229.749464:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830229.749468:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830229.749471:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228449152:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787830229.749479:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228449280:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830229.749504:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830229.749518:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830229/real 1787830229] req@ffff9856140da1c0 x1874669228449280/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830254 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830229.749550:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140da1c0 x1874669228449280/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830254 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830229.749558:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830229.749561:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830229.749564:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228449280:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830229.813853:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830230.708868:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830230.837081:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830231.732830:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830231.861873:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830232.756786:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830232.886603:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830233.781474:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830233.909101:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830234.804841:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787830234.870256:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830234.870266:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830234.870270:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830234.870279:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6380 00000100:00080000:1.0:1787830234.870289:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830234.870296:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787830234.870331:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0f00 x1874669228449408/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830234.870360:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830234.870364:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830234.870366:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830234.870370:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6380 00000100:00080000:1.0:1787830234.870373:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830234.870377:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830234.870386:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830234.870399:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a30c0 x1874669228449536/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830234.870472:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830234.870475:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830234.870477:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830234.870482:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6380 00000100:00080000:1.0:1787830234.870487:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830234.870491:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830234.870505:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830234.870519:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3480 x1874669228449664/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830234.870540:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228449408:192.168.203.114@tcp:250:kworker.0 00000800:00000100:3.0:1787830234.870628:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830234.870674:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830234/real 1787830234] req@ffff9856085a0f00 x1874669228449408/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830259 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830234.870738:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0f00 x1874669228449408/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830259 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830234.870750:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830234.870755:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830234.870760:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228449408:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787830234.870768:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228449536:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830234.870791:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830234.870803:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830234/real 1787830234] req@ffff9856085a30c0 x1874669228449536/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830259 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830234.870815:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a30c0 x1874669228449536/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830259 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830234.870822:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830234.870826:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830234.870829:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228449536:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787830234.870858:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228449664:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830234.870876:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830234.870885:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830234/real 1787830234] req@ffff9856085a3480 x1874669228449664/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830259 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830234.870895:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3480 x1874669228449664/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830259 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830234.870902:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830234.870905:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830234.870908:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228449664:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830234.932874:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830235.828840:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830235.956976:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830236.852868:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830236.982233:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830237.876850:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830238.005317:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830238.900810:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830239.029523:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830239.924808:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787830239.988806:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830239.988816:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830239.988819:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830239.988829:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6385 00000100:00080000:1.0:1787830239.988839:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830239.988845:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787830239.988895:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a21c0 x1874669228449792/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830239.988924:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830239.988927:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830239.988929:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830239.988933:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6385 00000100:00080000:1.0:1787830239.988937:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830239.988941:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830239.988950:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830239.988964:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a12c0 x1874669228449920/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830239.988972:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830239.988975:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830239.988976:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830239.988979:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6385 00000100:00080000:1.0:1787830239.988983:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830239.988986:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830239.988990:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830239.988999:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2940 x1874669228450048/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830239.989334:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228449792:192.168.203.114@tcp:250:kworker.0 00000800:00000100:3.0:1787830239.989386:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830239.989417:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830239/real 1787830239] req@ffff9856085a21c0 x1874669228449792/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830264 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830239.989431:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a21c0 x1874669228449792/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830264 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830239.989440:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830239.989444:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830239.989447:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228449792:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787830239.989452:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228449920:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830239.989485:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830239.989497:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830239/real 1787830239] req@ffff9856085a12c0 x1874669228449920/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830264 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830239.989508:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a12c0 x1874669228449920/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830264 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830239.989515:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830239.989517:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830239.989520:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228449920:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787830239.989525:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228450048:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830239.989541:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830239.989550:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830239/real 1787830239] req@ffff9856085a2940 x1874669228450048/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830264 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830239.989559:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2940 x1874669228450048/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830264 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830239.989571:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830239.989574:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830239.989576:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228450048:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830240.052869:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830240.948856:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830241.077114:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830241.972820:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830242.100911:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830242.997078:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830243.124917:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830244.021126:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830244.149416:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830245.045087:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787830245.108823:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830245.108833:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830245.108837:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830245.108847:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6390 00000100:00080000:2.0:1787830245.108857:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830245.108865:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787830245.108899:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8f00 x1874669228450176/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830245.108973:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228450176:192.168.203.114@tcp:250:kworker.0 00000100:00080000:2.0:1787830245.108984:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830245.108987:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830245.108989:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830245.108993:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6390 00000100:00080000:2.0:1787830245.108996:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830245.109000:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830245.109009:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830245.109022:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140dad00 x1874669228450304/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830245.109031:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000800:00000100:3.0:1787830245.109033:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00080000:2.0:1787830245.109034:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830245.109036:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830245.109038:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6390 00000100:00080000:2.0:1787830245.109041:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830245.109044:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830245.109048:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830245.109056:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db0c0 x1874669228450432/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830245.109064:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830245/real 1787830245] req@ffff9856140d8f00 x1874669228450176/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830270 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830245.109080:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8f00 x1874669228450176/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830270 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830245.109089:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830245.109093:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830245.109096:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228450176:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787830245.109118:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228450304:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830245.109141:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830245.109153:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830245/real 1787830245] req@ffff9856140dad00 x1874669228450304/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830270 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830245.109165:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140dad00 x1874669228450304/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830270 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830245.109172:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830245.109175:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830245.109178:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228450304:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787830245.109184:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228450432:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830245.109202:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830245.109214:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830245/real 1787830245] req@ffff9856140db0c0 x1874669228450432/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830270 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830245.109224:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db0c0 x1874669228450432/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830270 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830245.109231:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830245.109233:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830245.109237:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228450432:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830245.172847:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830246.069015:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830246.198467:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830247.093378:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830247.220970:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830248.117597:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830248.244883:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830249.140809:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830249.268936:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830250.164922:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787830250.229072:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830250.229082:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830250.229086:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830250.229095:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6395 00000100:00080000:1.0:1787830250.229105:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830250.229111:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787830250.229147:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a1a40 x1874669228450560/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830250.229176:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830250.229179:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830250.229181:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830250.229185:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6395 00000100:00080000:1.0:1787830250.229188:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830250.229192:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830250.229201:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830250.229212:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2580 x1874669228450688/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830250.229220:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830250.229223:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830250.229224:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830250.229227:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6395 00000100:00080000:1.0:1787830250.229231:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830250.229234:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830250.229238:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830250.229247:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3840 x1874669228450816/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830250.229445:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228450560:192.168.203.114@tcp:250:kworker.0 00000800:00000100:3.0:1787830250.229504:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830250.229537:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830250/real 1787830250] req@ffff9856085a1a40 x1874669228450560/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830275 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830250.229554:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a1a40 x1874669228450560/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830275 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830250.229564:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830250.229567:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830250.229571:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228450560:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787830250.229578:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228450688:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830250.229599:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830250.229610:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830250/real 1787830250] req@ffff9856085a2580 x1874669228450688/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830275 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830250.229622:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2580 x1874669228450688/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830275 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830250.229630:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830250.229633:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830250.229637:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228450688:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787830250.229655:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228450816:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830250.229676:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830250.229687:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830250/real 1787830250] req@ffff9856085a3840 x1874669228450816/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830275 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830250.229726:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3840 x1874669228450816/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830275 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830250.229733:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830250.229736:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830250.229739:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228450816:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830250.292797:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830251.188798:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830251.317566:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830252.213367:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830252.340957:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830253.236878:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830253.364997:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830254.261028:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830254.388968:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830255.284805:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787830255.349240:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830255.349251:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830255.349255:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830255.349265:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6400 00000100:00080000:1.0:1787830255.349275:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830255.349283:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787830255.349582:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a03c0 x1874669228450944/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830255.349619:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830255.349622:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830255.349624:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830255.349637:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6400 00000100:00080000:1.0:1787830255.349642:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830255.349647:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830255.349656:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830255.349669:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3c00 x1874669228451072/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830255.349678:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830255.349682:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830255.349684:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830255.349688:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6400 00000100:00080000:1.0:1787830255.349720:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830255.349724:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830255.349730:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830255.349742:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0780 x1874669228451200/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830255.349988:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228450944:192.168.203.114@tcp:250:kworker.0 00000800:00000100:3.0:1787830255.350057:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830255.350102:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830255/real 1787830255] req@ffff9856085a03c0 x1874669228450944/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830280 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830255.350120:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a03c0 x1874669228450944/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830280 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830255.350136:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830255.350140:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830255.350144:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228450944:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787830255.350150:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228451072:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830255.350172:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830255.350184:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830255/real 1787830255] req@ffff9856085a3c00 x1874669228451072/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830280 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830255.350196:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3c00 x1874669228451072/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830280 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830255.350204:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830255.350207:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830255.350210:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228451072:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787830255.350215:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228451200:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830255.350235:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830255.350246:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830255/real 1787830255] req@ffff9856085a0780 x1874669228451200/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830280 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830255.350257:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0780 x1874669228451200/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830280 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830255.350264:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830255.350267:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830255.350270:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228451200:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830255.412831:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830256.308831:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830256.436969:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830257.332853:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830257.461016:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830258.356863:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830258.485143:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830259.380789:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830259.510605:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830260.404837:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787830260.468838:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830260.468851:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830260.468856:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830260.468867:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6406 00000100:00080000:1.0:1787830260.468883:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830260.468893:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787830260.468940:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a1680 x1874669228451328/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830260.469025:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228451328:192.168.203.114@tcp:250:kworker.0 00000800:00000100:3.0:1787830260.469116:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00080000:1.0:1787830260.469153:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830260.469157:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830260.469159:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830260.469164:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6406 00000100:00080000:1.0:1787830260.469169:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830260.469173:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830260.469185:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830260.469198:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0b40 x1874669228451456/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830260.469263:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830260/real 1787830260] req@ffff9856085a1680 x1874669228451328/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830285 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830260.469284:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a1680 x1874669228451328/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830285 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830260.469295:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830260.469300:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830260.469305:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228451328:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787830260.469345:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228451456:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830260.469416:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830260.469431:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830260/real 1787830260] req@ffff9856085a0b40 x1874669228451456/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830285 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830260.469444:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0b40 x1874669228451456/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830285 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830260.469452:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830260.469456:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830260.469459:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228451456:192.168.203.114@tcp:38:kworker.0 00000100:00080000:1.0:1787830260.469602:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830260.469607:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830260.469609:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830260.469614:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6406 00000100:00080000:1.0:1787830260.469618:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830260.469623:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830260.469629:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830260.469643:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0000 x1874669228451584/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830260.469685:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228451584:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830260.469732:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830260.469826:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830260/real 1787830260] req@ffff9856085a0000 x1874669228451584/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830285 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830260.469839:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0000 x1874669228451584/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830285 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830260.469847:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830260.469851:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830260.469854:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228451584:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830260.532929:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830261.429907:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830261.556962:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830262.452839:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830262.582803:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830263.476838:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830263.604980:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830264.501445:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830264.628985:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830265.524868:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787830265.588808:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830265.588819:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830265.588823:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830265.588832:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6411 00000100:00080000:1.0:1787830265.588842:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830265.588850:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787830265.588889:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2d00 x1874669228451712/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830265.588927:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830265.588931:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830265.588934:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830265.588939:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6411 00000100:00080000:1.0:1787830265.588945:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830265.588950:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830265.588959:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830265.588972:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0000 x1874669228451840/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830265.588983:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830265.588987:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830265.588990:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830265.588993:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6411 00000100:00080000:1.0:1787830265.589003:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830265.589008:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830265.589013:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830265.589035:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0b40 x1874669228451968/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787830265.590211:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228451712:192.168.203.114@tcp:250:kworker.0 00000800:00000100:2.0:1787830265.590312:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830265.590353:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830265/real 1787830265] req@ffff9856085a2d00 x1874669228451712/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830290 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830265.590383:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2d00 x1874669228451712/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830290 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830265.590394:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830265.590399:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830265.590403:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228451712:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787830265.590410:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228451840:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787830265.590437:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830265.590450:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830265/real 1787830265] req@ffff9856085a0000 x1874669228451840/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830290 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830265.590460:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0000 x1874669228451840/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830290 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830265.590468:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830265.590472:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830265.590475:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228451840:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787830265.590481:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228451968:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787830265.590499:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830265.590509:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830265/real 1787830265] req@ffff9856085a0b40 x1874669228451968/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830290 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830265.590520:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0b40 x1874669228451968/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830290 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830265.590528:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830265.590532:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830265.590536:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228451968:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830265.653127:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830266.548833:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830266.676943:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830267.572797:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830267.701014:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830268.596907:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830268.724956:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830269.620809:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830269.749308:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830270.644889:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787830270.708863:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830270.708873:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830270.708877:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830270.708886:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6416 00000100:00080000:0.0:1787830270.708896:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830270.708902:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787830270.708936:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df52c0 x1874669228452096/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830270.708970:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830270.708974:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830270.708976:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830270.708979:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6416 00000100:00080000:0.0:1787830270.708983:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830270.708987:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787830270.708995:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830270.709007:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7840 x1874669228452224/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830270.709023:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830270.709025:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830270.709027:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830270.709031:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6416 00000100:00080000:0.0:1787830270.709034:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830270.709038:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787830270.709042:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830270.709052:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4f00 x1874669228452352/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787830270.709098:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228452096:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787830270.709194:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228452224:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787830270.709221:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228452352:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830270.773049:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830271.671354:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830271.797497:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830272.692807:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830272.821476:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830273.717071:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000400:00000100:0.0:1787830273.781013:0:2373:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.114:988 00000400:00000100:0.0:1787830273.781057:0:2373:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.114@tcp at host 192.168.203.114 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:0.0:1787830273.781073:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:0.0:1787830273.781110:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:0.0:1787830273.781124:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000100:00100000:2.0:1787830273.781284:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830270/real 1787830273] req@ffff985607df52c0 x1874669228452096/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830295 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830273.781307:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df52c0 x1874669228452096/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830295 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830273.781319:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830273.781323:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830273.781327:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228452096:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787830273.781334:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830270/real 1787830273] req@ffff985607df7840 x1874669228452224/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830295 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830273.781346:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7840 x1874669228452224/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830295 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830273.781353:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830273.781356:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830273.781359:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228452224:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787830273.781365:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830270/real 1787830273] req@ffff985607df4f00 x1874669228452352/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830295 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830273.781375:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4f00 x1874669228452352/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830295 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830273.781382:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830273.781385:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830273.781388:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228452352:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787830273.845804:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830274.740803:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830274.869803:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830275.764798:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787830275.828980:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830275.828989:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830275.828992:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830275.829000:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6421 00000100:00080000:1.0:1787830275.829010:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830275.829023:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787830275.829054:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a1680 x1874669228452480/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830275.829079:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830275.829083:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830275.829085:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830275.829090:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6421 00000100:00080000:1.0:1787830275.829095:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830275.829099:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830275.829109:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830275.829121:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0780 x1874669228452608/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830275.829136:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830275.829139:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830275.829141:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830275.829145:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6421 00000100:00080000:1.0:1787830275.829149:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830275.829152:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830275.829156:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830275.829165:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3c00 x1874669228452736/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787830275.829256:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228452480:192.168.203.114@tcp:250:kworker.0 00000800:00000100:2.0:1787830275.829327:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830275.829360:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830275/real 1787830275] req@ffff9856085a1680 x1874669228452480/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830300 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830275.829376:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a1680 x1874669228452480/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830300 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830275.829386:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830275.829390:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830275.829394:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228452480:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787830275.829401:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228452608:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787830275.829424:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830275.829437:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830275/real 1787830275] req@ffff9856085a0780 x1874669228452608/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830300 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830275.829448:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0780 x1874669228452608/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830300 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830275.829456:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830275.829459:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830275.829461:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228452608:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787830275.829466:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228452736:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787830275.829491:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830275.829502:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830275/real 1787830275] req@ffff9856085a3c00 x1874669228452736/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830300 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830275.829515:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3c00 x1874669228452736/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830300 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830275.829531:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830275.829535:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830275.829538:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228452736:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830275.893063:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830276.788830:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830276.916910:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830277.812864:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830277.941523:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830278.836866:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830278.965050:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830279.861219:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830279.988964:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830280.884827:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787830280.949095:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830280.949106:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830280.949110:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830280.949120:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6426 00000100:00080000:0.0:1787830280.949129:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830280.949137:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787830280.949184:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6d00 x1874669228452864/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830280.949215:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830280.949219:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830280.949221:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830280.949225:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6426 00000100:00080000:0.0:1787830280.949228:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830280.949232:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787830280.949241:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830280.949255:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5e00 x1874669228452992/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787830280.949259:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228452864:192.168.203.114@tcp:250:kworker.0 00000100:00080000:0.0:1787830280.949271:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830280.949275:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830280.949276:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830280.949280:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6426 00000100:00080000:0.0:1787830280.949284:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830280.949288:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787830280.949292:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830280.949304:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df61c0 x1874669228453120/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787830280.949327:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830280.949359:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830280/real 1787830280] req@ffff985607df6d00 x1874669228452864/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830305 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830280.949376:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6d00 x1874669228452864/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830305 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830280.949385:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830280.949389:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830280.949392:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228452864:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787830280.949413:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228452992:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787830280.949436:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830280.949448:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830280/real 1787830280] req@ffff985607df5e00 x1874669228452992/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830305 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830280.949466:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5e00 x1874669228452992/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830305 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830280.949473:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830280.949475:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830280.949477:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228452992:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787830280.949481:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228453120:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787830280.949500:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830280.949510:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830280/real 1787830280] req@ffff985607df61c0 x1874669228453120/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830305 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830280.949521:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df61c0 x1874669228453120/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830305 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830280.949529:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830280.949532:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830280.949535:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228453120:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830281.012853:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830281.908857:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830282.036874:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830282.934042:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830283.062658:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830283.956841:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830284.085402:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830284.980879:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830285.109324:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830286.004903:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787830286.068877:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830286.068887:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830286.068891:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830286.068899:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6431 00000100:00080000:2.0:1787830286.068910:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830286.068918:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787830286.068951:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140da940 x1874669228453248/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830286.068973:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830286.068976:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830286.068977:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830286.068979:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6431 00000100:00080000:2.0:1787830286.068982:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830286.068984:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830286.068992:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830286.069003:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d9a40 x1874669228453376/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830286.069023:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1787830286.069025:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228453248:192.168.203.114@tcp:250:kworker.0 00000100:00080000:2.0:1787830286.069028:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830286.069032:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830286.069038:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6431 00000100:00080000:2.0:1787830286.069041:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830286.069046:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830286.069051:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830286.069062:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8b40 x1874669228453504/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787830286.069097:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830286.069131:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830286/real 1787830286] req@ffff9856140da940 x1874669228453248/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830311 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830286.069168:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140da940 x1874669228453248/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830311 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830286.069178:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830286.069182:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830286.069186:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228453248:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787830286.069192:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228453376:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830286.069215:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830286.069228:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830286/real 1787830286] req@ffff9856140d9a40 x1874669228453376/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830311 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830286.069240:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d9a40 x1874669228453376/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830311 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830286.069248:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830286.069251:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830286.069254:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228453376:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787830286.069279:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228453504:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830286.069311:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830286.069322:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830286/real 1787830286] req@ffff9856140d8b40 x1874669228453504/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830311 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830286.069332:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8b40 x1874669228453504/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830311 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830286.069338:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830286.069341:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830286.069344:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228453504:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830286.132874:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830287.029005:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830287.156903:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830288.053136:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830288.180917:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830289.079196:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830289.205250:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830290.100901:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830290.228969:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830291.124857:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787830291.188884:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830291.188894:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830291.188897:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830291.188906:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6436 00000100:00080000:1.0:1787830291.188916:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830291.188923:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787830291.188961:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a03c0 x1874669228453632/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830291.188994:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830291.188998:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830291.188999:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830291.189002:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6436 00000100:00080000:1.0:1787830291.189005:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830291.189008:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830291.189018:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830291.189033:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3840 x1874669228453760/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830291.189043:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830291.189046:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830291.189048:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830291.189052:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6436 00000100:00080000:1.0:1787830291.189057:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830291.189066:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830291.189072:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830291.189082:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2580 x1874669228453888/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830291.189171:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228453632:192.168.203.114@tcp:250:kworker.0 00000800:00000100:3.0:1787830291.189261:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830291.189301:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830291/real 1787830291] req@ffff9856085a03c0 x1874669228453632/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830316 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830291.189318:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a03c0 x1874669228453632/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830316 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830291.189326:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830291.189330:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830291.189333:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228453632:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787830291.189338:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228453760:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830291.189354:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830291.189361:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830291/real 1787830291] req@ffff9856085a3840 x1874669228453760/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830316 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830291.189372:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3840 x1874669228453760/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830316 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830291.189381:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830291.189384:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830291.189387:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228453760:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787830291.189416:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228453888:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830291.189436:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830291.189460:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830291/real 1787830291] req@ffff9856085a2580 x1874669228453888/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830316 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830291.189474:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2580 x1874669228453888/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830316 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830291.189487:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830291.189491:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830291.189494:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228453888:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830291.253171:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830292.149085:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830292.276994:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830293.173360:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830293.300864:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830294.196982:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830294.325366:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830295.221085:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830295.348876:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830296.244814:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787830296.308819:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830296.308827:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830296.308832:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830296.308840:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6441 00000100:00080000:1.0:1787830296.308850:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830296.308856:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787830296.308890:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a1a40 x1874669228454016/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830296.308917:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830296.308920:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830296.308922:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830296.308925:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6441 00000100:00080000:1.0:1787830296.308928:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830296.308932:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830296.308940:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830296.308951:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2940 x1874669228454144/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787830296.308956:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228454016:192.168.203.114@tcp:250:kworker.0 00000100:00080000:1.0:1787830296.308966:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830296.308970:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830296.308971:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830296.308975:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6441 00000100:00080000:1.0:1787830296.308979:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830296.308982:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830296.308986:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830296.308997:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a12c0 x1874669228454272/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787830296.309022:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830296.309055:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830296/real 1787830296] req@ffff9856085a1a40 x1874669228454016/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830321 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830296.309139:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a1a40 x1874669228454016/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830321 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830296.309149:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830296.309153:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830296.309157:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228454016:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787830296.309181:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228454144:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787830296.309204:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830296.309216:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830296/real 1787830296] req@ffff9856085a2940 x1874669228454144/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830321 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830296.309224:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2940 x1874669228454144/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830321 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830296.309229:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830296.309231:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830296.309234:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228454144:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787830296.309237:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228454272:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787830296.309250:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830296.309256:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830296/real 1787830296] req@ffff9856085a12c0 x1874669228454272/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830321 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830296.309263:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a12c0 x1874669228454272/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830321 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830296.309268:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830296.309269:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830296.309271:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228454272:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830296.372868:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830297.268829:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830297.397110:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830298.292863:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830298.422188:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830299.316865:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830299.444917:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830300.340818:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830300.469554:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830301.365521:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787830301.429349:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830301.429354:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830301.429358:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830301.429365:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6447 00000100:00080000:1.0:1787830301.429373:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830301.429377:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787830301.429402:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a21c0 x1874669228454400/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830301.429420:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830301.429424:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830301.429425:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830301.429429:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6447 00000100:00080000:1.0:1787830301.429432:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830301.429435:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830301.429444:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830301.429453:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228454400:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787830301.429455:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3480 x1874669228454528/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830301.429463:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830301.429467:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830301.429468:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830301.429482:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6447 00000100:00080000:1.0:1787830301.429486:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830301.429489:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830301.429493:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830301.429503:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a30c0 x1874669228454656/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787830301.429514:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830301.429541:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830301/real 1787830301] req@ffff9856085a21c0 x1874669228454400/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830326 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830301.429556:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a21c0 x1874669228454400/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830326 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830301.429565:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830301.429568:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830301.429572:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228454400:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787830301.429593:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228454528:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787830301.429733:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830301.429746:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830301/real 1787830301] req@ffff9856085a3480 x1874669228454528/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830326 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830301.429757:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3480 x1874669228454528/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830326 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830301.429765:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830301.429769:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830301.429772:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228454528:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787830301.429777:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228454656:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787830301.429795:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830301.429805:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830301/real 1787830301] req@ffff9856085a30c0 x1874669228454656/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830326 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830301.429816:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a30c0 x1874669228454656/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830326 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830301.429824:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830301.429827:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830301.429830:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228454656:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830301.493109:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830302.388819:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830302.517990:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830303.412857:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830303.540886:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830304.437213:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830304.565276:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830305.460847:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830305.588869:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830306.484810:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787830306.548805:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830306.548815:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830306.548819:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830306.548828:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6452 00000100:00080000:1.0:1787830306.548837:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830306.548844:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787830306.548876:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0f00 x1874669228454784/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830306.548898:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830306.548901:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830306.548902:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830306.548905:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6452 00000100:00080000:1.0:1787830306.548908:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830306.548910:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830306.548917:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830306.548926:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a30c0 x1874669228454912/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830306.548932:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830306.548934:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830306.548935:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830306.548938:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6452 00000100:00080000:1.0:1787830306.548939:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830306.548943:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787830306.548945:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228454784:192.168.203.114@tcp:250:kworker.0 00000008:00100000:1.0:1787830306.548947:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830306.548956:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3480 x1874669228455040/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787830306.549014:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830306.549048:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830306/real 1787830306] req@ffff9856085a0f00 x1874669228454784/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830331 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830306.549063:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0f00 x1874669228454784/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830331 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830306.549073:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830306.549077:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830306.549080:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228454784:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787830306.549087:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228454912:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787830306.549109:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830306.549121:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830306/real 1787830306] req@ffff9856085a30c0 x1874669228454912/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830331 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830306.549132:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a30c0 x1874669228454912/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830331 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830306.549140:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830306.549143:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830306.549147:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228454912:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787830306.549170:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228455040:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787830306.549188:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830306.549195:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830306/real 1787830306] req@ffff9856085a3480 x1874669228455040/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830331 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830306.549206:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3480 x1874669228455040/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830331 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830306.549211:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830306.549213:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830306.549216:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228455040:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830306.612969:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830307.508843:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830307.636943:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830308.532826:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830308.660893:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830309.557231:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830309.685023:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830310.580861:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830310.708908:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830311.604810:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787830311.669022:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830311.669032:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830311.669036:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830311.669047:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6457 00000100:00080000:2.0:1787830311.669056:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830311.669063:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787830311.669096:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d9680 x1874669228455168/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830311.669131:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830311.669135:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830311.669137:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830311.669141:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6457 00000100:00080000:2.0:1787830311.669145:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830311.669149:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830311.669159:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830311.669171:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d83c0 x1874669228455296/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830311.669180:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830311.669184:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830311.669185:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830311.669189:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6457 00000100:00080000:2.0:1787830311.669192:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830311.669196:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830311.669200:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830311.669209:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db480 x1874669228455424/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830311.669299:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228455168:192.168.203.114@tcp:250:kworker.0 00000800:00000100:3.0:1787830311.669357:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830311.669389:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830311/real 1787830311] req@ffff9856140d9680 x1874669228455168/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830336 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830311.669406:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d9680 x1874669228455168/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830336 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830311.669415:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830311.669419:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830311.669423:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228455168:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787830311.669429:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228455296:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830311.669451:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830311.669463:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830311/real 1787830311] req@ffff9856140d83c0 x1874669228455296/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830336 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830311.669482:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d83c0 x1874669228455296/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830336 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830311.669491:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830311.669494:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830311.669497:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228455296:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787830311.669503:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228455424:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830311.669523:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830311.669535:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830311/real 1787830311] req@ffff9856140db480 x1874669228455424/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830336 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830311.669546:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db480 x1874669228455424/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830336 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830311.669553:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830311.669555:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830311.669558:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228455424:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830311.732828:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830312.628807:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830312.757715:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830313.654077:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830313.781003:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830314.677022:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830314.805067:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830315.703031:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830315.828859:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830316.725265:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787830316.792331:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830316.792341:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830316.792345:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830316.792354:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6462 00000100:00080000:1.0:1787830316.792364:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830316.792371:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787830316.792405:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a21c0 x1874669228455552/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830316.792438:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830316.792442:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830316.792444:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830316.792448:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6462 00000100:00080000:1.0:1787830316.792452:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830316.792456:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830316.792466:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830316.792479:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a12c0 x1874669228455680/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830316.792488:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830316.792492:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830316.792494:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830316.792498:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6462 00000100:00080000:1.0:1787830316.792501:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830316.792505:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830316.792510:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830316.792519:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2940 x1874669228455808/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830316.792612:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228455552:192.168.203.114@tcp:250:kworker.0 00000800:00000100:3.0:1787830316.792667:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830316.792730:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830316/real 1787830316] req@ffff9856085a21c0 x1874669228455552/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830341 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830316.792747:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a21c0 x1874669228455552/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830341 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830316.792756:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830316.792760:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830316.792764:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228455552:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787830316.792770:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228455680:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830316.792792:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830316.792804:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830316/real 1787830316] req@ffff9856085a12c0 x1874669228455680/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830341 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830316.792815:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a12c0 x1874669228455680/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830341 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830316.792823:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830316.792826:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830316.792829:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228455680:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787830316.792834:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228455808:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830316.792852:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830316.792863:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830316/real 1787830316] req@ffff9856085a2940 x1874669228455808/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830341 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830316.792875:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2940 x1874669228455808/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830341 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830316.792882:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830316.792885:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830316.792888:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228455808:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830316.853082:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830317.748811:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830317.876939:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830318.772800:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830318.901373:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830319.796859:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830319.925007:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830320.820800:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830320.949340:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830321.844861:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787830321.909001:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830321.909010:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830321.909014:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830321.909024:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6467 00000100:00080000:2.0:1787830321.909035:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830321.909042:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787830321.909078:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8780 x1874669228455936/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830321.909108:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830321.909112:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830321.909114:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830321.909119:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6467 00000100:00100000:3.0:1787830321.909753:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228455936:192.168.203.114@tcp:250:kworker.0 00000800:00000100:3.0:1787830321.909813:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830321.909856:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830321/real 1787830321] req@ffff9856140d8780 x1874669228455936/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830346 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830321.909874:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8780 x1874669228455936/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830346 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830321.909899:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830321.909904:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830321.909908:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228455936:192.168.203.114@tcp:250:kworker.0 00000100:00080000:2.0:1787830321.909969:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830321.909977:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830321.909986:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830321.910003:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8780 x1874669228456064/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830321.910024:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830321.910029:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830321.910031:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830321.910034:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6467 00000100:00080000:2.0:1787830321.910039:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830321.910042:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830321.910048:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830321.910060:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db480 x1874669228456192/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830321.910569:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228456064:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830321.910600:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830321.910628:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830321/real 1787830321] req@ffff9856140d8780 x1874669228456064/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830346 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830321.910641:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8780 x1874669228456064/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830346 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830321.910649:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830321.910652:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830321.910656:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228456064:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787830321.910662:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228456192:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830321.910681:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830321.910723:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830321/real 1787830321] req@ffff9856140db480 x1874669228456192/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830346 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830321.910736:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db480 x1874669228456192/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830346 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830321.910744:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830321.910747:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830321.910750:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228456192:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830321.972848:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830322.868804:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830322.997293:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830323.892821:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830324.020904:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830324.916801:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830325.045006:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830325.940883:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830326.068967:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830326.965756:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787830327.029128:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830327.029138:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830327.029143:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830327.029152:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6472 00000100:00080000:2.0:1787830327.029163:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830327.029170:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787830327.029207:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d83c0 x1874669228456320/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830327.029239:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830327.029243:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830327.029244:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830327.029248:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6472 00000100:00080000:2.0:1787830327.029251:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830327.029254:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830327.029263:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830327.029273:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d9680 x1874669228456448/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830327.029282:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830327.029284:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830327.029286:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830327.029289:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6472 00000100:00080000:2.0:1787830327.029292:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830327.029295:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830327.029299:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830327.029308:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8b40 x1874669228456576/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830327.029459:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228456320:192.168.203.114@tcp:250:kworker.0 00000800:00000100:3.0:1787830327.029564:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830327.029596:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830327/real 1787830327] req@ffff9856140d83c0 x1874669228456320/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830352 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830327.029613:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d83c0 x1874669228456320/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830352 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830327.029622:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830327.029626:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830327.029629:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228456320:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787830327.029636:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228456448:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830327.029658:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830327.029668:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830327/real 1787830327] req@ffff9856140d9680 x1874669228456448/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830352 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830327.029679:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d9680 x1874669228456448/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830352 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830327.029686:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830327.029689:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830327.029763:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228456448:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787830327.029769:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228456576:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830327.029788:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830327.029800:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830327/real 1787830327] req@ffff9856140d8b40 x1874669228456576/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830352 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830327.029811:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8b40 x1874669228456576/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830352 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830327.029817:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830327.029820:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830327.029823:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228456576:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830327.093040:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830327.988866:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830328.116965:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830329.012796:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830329.141054:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830330.037065:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830330.165014:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830331.061092:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830331.188949:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830332.084876:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787830332.149154:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830332.149167:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787830332.149172:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830332.149185:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6477 00000100:00080000:3.0:1787830332.149198:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830332.149206:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787830332.149252:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f803c0 x1874669228456704/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787830332.149321:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228456704:192.168.203.114@tcp:250:kworker.0 00000800:00000100:2.0:1787830332.149399:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00080000:3.0:1787830332.149434:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830332.149438:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787830332.149440:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830332.149445:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6477 00000100:00080000:3.0:1787830332.149450:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830332.149454:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830332.149465:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830332.149479:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83840 x1874669228456832/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787830332.149550:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830332/real 1787830332] req@ffff985610f803c0 x1874669228456704/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830357 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830332.149574:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f803c0 x1874669228456704/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830357 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830332.149586:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830332.149591:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00080000:3.0:1787830332.149596:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1787830332.149597:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228456704:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787830332.149613:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787830332.149617:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830332.149628:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6477 00000100:00080000:3.0:1787830332.149638:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830332.149647:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787830332.149655:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228456832:192.168.203.114@tcp:38:kworker.0 00000008:00100000:3.0:1787830332.149659:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000800:00000100:2.0:1787830332.149719:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830332.149735:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830332/real 1787830332] req@ffff985610f83840 x1874669228456832/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830357 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830332.149748:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83840 x1874669228456832/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830357 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830332.149757:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830332.149764:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830332.149766:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81e00 x1874669228456960/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787830332.149773:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228456832:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787830332.149839:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228456960:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787830332.149868:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830332.149880:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830332/real 1787830332] req@ffff985610f81e00 x1874669228456960/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830357 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830332.149893:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81e00 x1874669228456960/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830357 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830332.149900:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830332.149904:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830332.149907:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228456960:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830332.212900:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830333.108835:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830333.236913:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830334.132960:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830334.261009:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830335.156811:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830335.284912:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830336.180912:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830336.309048:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830337.206218:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787830337.268855:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830337.268868:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787830337.268874:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830337.268887:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6482 00000100:00080000:3.0:1787830337.268902:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830337.268913:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787830337.268958:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81680 x1874669228457088/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830337.268999:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830337.269004:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787830337.269006:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830337.269010:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6482 00000100:00080000:3.0:1787830337.269014:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830337.269019:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830337.269032:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830337.269049:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f812c0 x1874669228457216/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830337.269060:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830337.269063:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787830337.269066:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830337.269070:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6482 00000100:00080000:3.0:1787830337.269074:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830337.269078:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830337.269083:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830337.269091:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80b40 x1874669228457344/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787830337.270498:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228457088:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787830337.270589:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228457216:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787830337.270620:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228457344:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830337.332845:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830338.228847:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830338.357050:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830339.252980:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830339.380890:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830340.276800:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000400:00000100:0.0:1787830340.341378:0:2370:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.114:988 00000400:00000100:0.0:1787830340.341420:0:2370:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.114@tcp at host 192.168.203.114 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:0.0:1787830340.341436:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:0.0:1787830340.341475:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:0.0:1787830340.341495:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000100:00100000:2.0:1787830340.341632:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830337/real 1787830340] req@ffff985610f81680 x1874669228457088/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830362 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830340.341648:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81680 x1874669228457088/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830362 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830340.341657:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830340.341660:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830340.341663:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228457088:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787830340.341668:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830337/real 1787830340] req@ffff985610f812c0 x1874669228457216/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830362 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830340.341675:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f812c0 x1874669228457216/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830362 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830340.341680:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830340.341682:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830340.341684:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228457216:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787830340.341688:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830337/real 1787830340] req@ffff985610f80b40 x1874669228457344/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830362 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830340.341709:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80b40 x1874669228457344/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830362 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830340.341714:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830340.341716:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830340.341718:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228457344:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787830340.406218:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830341.300919:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830341.432441:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830342.324868:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787830342.394979:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830342.394990:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830342.394994:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830342.395003:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6488 00000100:00080000:1.0:1787830342.395014:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830342.395020:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787830342.395059:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a1a40 x1874669228457472/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830342.395096:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830342.395100:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830342.395101:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830342.395105:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6488 00000100:00080000:1.0:1787830342.395108:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830342.395111:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830342.395121:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830342.395132:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2580 x1874669228457600/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830342.395142:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830342.395145:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830342.395146:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830342.395149:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6488 00000100:00080000:1.0:1787830342.395153:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830342.395156:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830342.395161:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830342.395171:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3840 x1874669228457728/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787830342.395334:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228457472:192.168.203.114@tcp:250:kworker.0 00000800:00000100:2.0:1787830342.395394:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830342.395429:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830342/real 1787830342] req@ffff9856085a1a40 x1874669228457472/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830367 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830342.395445:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a1a40 x1874669228457472/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830367 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830342.395455:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830342.395459:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830342.395463:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228457472:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787830342.395470:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228457600:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787830342.395491:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830342.395503:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830342/real 1787830342] req@ffff9856085a2580 x1874669228457600/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830367 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830342.395528:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2580 x1874669228457600/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830367 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830342.395536:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830342.395539:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830342.395542:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228457600:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787830342.395547:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228457728:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787830342.395566:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830342.395577:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830342/real 1787830342] req@ffff9856085a3840 x1874669228457728/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830367 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830342.395587:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3840 x1874669228457728/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830367 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830342.395600:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830342.395603:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830342.395605:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228457728:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830342.452830:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830343.348797:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830343.479732:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830344.373020:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830344.502406:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830345.398681:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830345.524893:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830346.420832:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830346.549413:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830347.444992:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787830347.510253:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830347.510264:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787830347.510267:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830347.510277:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6493 00000100:00080000:3.0:1787830347.510287:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830347.510295:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787830347.510333:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80000 x1874669228457856/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830347.510369:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830347.510374:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787830347.510375:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830347.510379:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6493 00000100:00080000:3.0:1787830347.510383:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830347.510386:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830347.510395:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830347.510407:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81a40 x1874669228457984/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830347.510416:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830347.510419:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787830347.510421:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830347.510424:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6493 00000100:00080000:3.0:1787830347.510427:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830347.510430:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830347.510435:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830347.510444:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f830c0 x1874669228458112/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830347.510517:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228457856:192.168.203.114@tcp:250:kworker.0 00000800:00000100:0.0:1787830347.510579:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830347.510613:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830347/real 1787830347] req@ffff985610f80000 x1874669228457856/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830372 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830347.510627:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80000 x1874669228457856/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830372 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830347.510637:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830347.510641:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830347.510644:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228457856:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787830347.510652:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228457984:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830347.510674:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830347.510686:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830347/real 1787830347] req@ffff985610f81a40 x1874669228457984/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830372 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830347.510740:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81a40 x1874669228457984/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830372 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830347.510748:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830347.510751:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830347.510754:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228457984:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787830347.510759:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228458112:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830347.510781:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830347.510793:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830347/real 1787830347] req@ffff985610f830c0 x1874669228458112/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830372 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830347.510804:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f830c0 x1874669228458112/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830372 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830347.510812:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830347.510815:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830347.510818:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228458112:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830347.572850:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830348.468809:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830348.597212:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830349.492814:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830349.620900:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830350.516860:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830350.645067:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830351.542002:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830351.670114:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830352.564829:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787830352.628887:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830352.628913:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830352.628918:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830352.628931:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6498 00000100:00080000:0.0:1787830352.628946:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830352.628964:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787830352.629012:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df43c0 x1874669228458240/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830352.629049:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830352.629053:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830352.629055:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830352.629059:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6498 00000100:00080000:0.0:1787830352.629063:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830352.629066:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787830352.629077:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830352.629089:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5680 x1874669228458368/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830352.629099:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830352.629103:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830352.629105:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830352.629110:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6498 00000100:00080000:0.0:1787830352.629114:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830352.629117:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787830352.629123:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830352.629134:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df70c0 x1874669228458496/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787830352.629145:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228458240:192.168.203.114@tcp:250:kworker.0 00000800:00000100:2.0:1787830352.629228:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830352.629266:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830352/real 1787830352] req@ffff985607df43c0 x1874669228458240/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830377 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830352.629283:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df43c0 x1874669228458240/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830377 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830352.629292:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830352.629296:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830352.629300:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228458240:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787830352.629308:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228458368:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787830352.629330:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830352.629342:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830352/real 1787830352] req@ffff985607df5680 x1874669228458368/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830377 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830352.629354:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5680 x1874669228458368/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830377 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830352.629363:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830352.629366:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830352.629369:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228458368:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787830352.629394:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228458496:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787830352.629413:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830352.629424:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830352/real 1787830352] req@ffff985607df70c0 x1874669228458496/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830377 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830352.629435:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df70c0 x1874669228458496/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830377 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830352.629443:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830352.629447:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830352.629449:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228458496:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830352.693659:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830353.589086:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830353.717364:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830354.613076:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830354.740895:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830355.636866:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830355.764900:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830356.660939:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830356.789050:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830357.684848:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787830357.749999:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830357.750012:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787830357.750017:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830357.750028:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6503 00000100:00080000:3.0:1787830357.750042:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830357.750051:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787830357.750099:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80780 x1874669228458624/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830357.750130:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830357.750134:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787830357.750135:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830357.750140:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6503 00000100:00080000:3.0:1787830357.750143:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830357.750148:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830357.750158:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830357.750172:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83c00 x1874669228458752/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830357.750183:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830357.750186:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787830357.750189:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830357.750193:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6503 00000100:00080000:3.0:1787830357.750196:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830357.750200:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830357.750206:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830357.750224:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f830c0 x1874669228458880/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787830357.750226:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228458624:192.168.203.114@tcp:250:kworker.0 00000800:00000100:2.0:1787830357.750313:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830357.750354:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830357/real 1787830357] req@ffff985610f80780 x1874669228458624/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830382 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830357.750375:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80780 x1874669228458624/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830382 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830357.750386:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830357.750390:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830357.750393:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228458624:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787830357.750458:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228458752:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787830357.750483:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830357.750495:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830357/real 1787830357] req@ffff985610f83c00 x1874669228458752/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830382 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830357.750507:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83c00 x1874669228458752/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830382 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830357.750515:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830357.750519:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830357.750522:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228458752:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787830357.750547:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228458880:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787830357.750567:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830357.750577:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830357/real 1787830357] req@ffff985610f830c0 x1874669228458880/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830382 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830357.750589:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f830c0 x1874669228458880/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830382 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830357.750604:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830357.750607:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830357.750611:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228458880:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830357.813958:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830358.708874:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830358.836944:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830359.732837:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830359.860934:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830360.756816:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830360.884987:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830361.780795:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830361.908890:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830362.805674:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787830362.869426:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830362.869436:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787830362.869440:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830362.869449:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6508 00000100:00080000:3.0:1787830362.869460:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830362.869467:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787830362.869504:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81a40 x1874669228459008/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830362.869530:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830362.869534:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787830362.869536:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830362.869539:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6508 00000100:00080000:3.0:1787830362.869542:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830362.869546:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830362.869555:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830362.869568:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80000 x1874669228459136/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787830362.869572:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228459008:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787830362.869582:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830362.869586:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787830362.869588:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830362.869592:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6508 00000100:00080000:3.0:1787830362.869596:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830362.869600:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830362.869616:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830362.869629:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80b40 x1874669228459264/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787830362.869647:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830362.869680:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830362/real 1787830362] req@ffff985610f81a40 x1874669228459008/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830387 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830362.869716:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81a40 x1874669228459008/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830387 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830362.869727:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830362.869731:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830362.869734:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228459008:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787830362.869753:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228459136:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787830362.869769:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830362.869777:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830362/real 1787830362] req@ffff985610f80000 x1874669228459136/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830387 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830362.869784:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80000 x1874669228459136/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830387 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830362.869793:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830362.869795:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830362.869797:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228459136:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787830362.869800:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228459264:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787830362.869810:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830362.869815:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830362/real 1787830362] req@ffff985610f80b40 x1874669228459264/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830387 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830362.869821:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80b40 x1874669228459264/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830387 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830362.869826:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830362.869827:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830362.869829:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228459264:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830362.932818:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830363.829227:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830363.957299:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830364.852884:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830364.980890:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830365.876792:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830366.004937:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830366.900894:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830367.029590:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830367.924831:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787830367.988907:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830367.988921:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787830367.988926:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830367.988937:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6513 00000100:00080000:3.0:1787830367.988950:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830367.988959:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787830367.989004:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f812c0 x1874669228459392/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830367.989037:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830367.989041:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787830367.989043:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830367.989047:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6513 00000100:00080000:3.0:1787830367.989051:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830367.989055:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830367.989067:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830367.989080:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81680 x1874669228459520/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830367.989089:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830367.989093:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787830367.989094:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830367.989098:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6513 00000100:00080000:3.0:1787830367.989101:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830367.989105:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830367.989109:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830367.989119:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81e00 x1874669228459648/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787830367.989364:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228459392:192.168.203.114@tcp:250:kworker.0 00000800:00000100:2.0:1787830367.989438:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830367.989477:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830367/real 1787830367] req@ffff985610f812c0 x1874669228459392/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830392 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830367.989493:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f812c0 x1874669228459392/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830392 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830367.989510:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830367.989514:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830367.989518:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228459392:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787830367.989526:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228459520:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787830367.989546:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830367.989567:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830367/real 1787830367] req@ffff985610f81680 x1874669228459520/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830392 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830367.989602:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81680 x1874669228459520/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830392 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830367.989611:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830367.989614:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830367.989618:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228459520:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787830367.989624:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228459648:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787830367.989650:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830367.989662:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830367/real 1787830367] req@ffff985610f81e00 x1874669228459648/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830392 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830367.989673:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81e00 x1874669228459648/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830392 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830367.989681:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830367.989684:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830367.989688:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228459648:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830368.052866:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830368.950405:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830369.077156:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830369.972838:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830370.100981:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830370.996819:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830371.125053:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830372.020849:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830372.149122:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830373.044912:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787830373.108839:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830373.108853:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787830373.108858:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830373.108882:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6518 00000100:00080000:3.0:1787830373.108897:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830373.108914:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787830373.108958:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83840 x1874669228459776/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830373.108993:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830373.108997:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787830373.109000:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830373.109006:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6518 00000100:00080000:3.0:1787830373.109011:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830373.109017:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830373.109028:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830373.109034:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228459776:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787830373.109049:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f803c0 x1874669228459904/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830373.109061:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830373.109065:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787830373.109068:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830373.109071:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6518 00000800:00000100:0.0:1787830373.109101:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830373.109135:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830373/real 1787830373] req@ffff985610f83840 x1874669228459776/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830398 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830373.109153:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83840 x1874669228459776/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830398 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830373.109162:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830373.109166:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830373.109172:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00080000:0.0:1787830373.109177:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000008:00100000:3.0:1787830373.109183:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830373.109184:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228459776:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787830373.109205:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82d00 x1874669228460032/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830373.109222:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228459904:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830373.109268:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830373.109284:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830373/real 1787830373] req@ffff985610f803c0 x1874669228459904/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830398 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830373.109297:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f803c0 x1874669228459904/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830398 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830373.109305:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830373.109308:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830373.109312:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228459904:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787830373.109324:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228460032:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830373.109348:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830373.109358:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830373/real 1787830373] req@ffff985610f82d00 x1874669228460032/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830398 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830373.109369:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82d00 x1874669228460032/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830398 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830373.109376:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830373.109379:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830373.109382:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228460032:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830373.173209:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830374.068861:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830374.199415:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830375.093030:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830375.221019:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830376.117413:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830376.244944:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830377.140860:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830377.268909:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830378.164854:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787830378.229270:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830378.229285:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830378.229290:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830378.229302:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6523 00000100:00080000:2.0:1787830378.229317:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830378.229325:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787830378.229374:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d9a40 x1874669228460160/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830378.229407:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830378.229411:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830378.229413:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830378.229417:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6523 00000100:00080000:2.0:1787830378.229421:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830378.229425:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830378.229439:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830378.229451:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140da940 x1874669228460288/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830378.229466:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830378.229469:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830378.229471:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830378.229474:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6523 00000100:00080000:2.0:1787830378.229478:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830378.229481:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830378.229486:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830378.229506:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db0c0 x1874669228460416/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830378.229749:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228460160:192.168.203.114@tcp:250:kworker.0 00000800:00000100:0.0:1787830378.229828:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830378.229868:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830378/real 1787830378] req@ffff9856140d9a40 x1874669228460160/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830403 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830378.229887:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d9a40 x1874669228460160/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830403 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830378.229898:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830378.229902:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830378.229907:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228460160:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787830378.229914:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228460288:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830378.229941:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830378.229954:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830378/real 1787830378] req@ffff9856140da940 x1874669228460288/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830403 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830378.229965:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140da940 x1874669228460288/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830403 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830378.229973:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830378.229976:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830378.229980:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228460288:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787830378.229986:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228460416:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830378.230005:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830378.230015:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830378/real 1787830378] req@ffff9856140db0c0 x1874669228460416/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830403 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830378.230025:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db0c0 x1874669228460416/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830403 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830378.230034:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830378.230037:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830378.230040:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228460416:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830378.292904:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830379.189055:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830379.316962:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830380.214825:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830380.341236:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830381.236951:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830381.366627:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830382.260824:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830382.390985:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830383.284813:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787830383.349179:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830383.349194:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787830383.349199:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830383.349214:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6528 00000100:00080000:3.0:1787830383.349229:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830383.349238:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787830383.349287:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82940 x1874669228460544/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830383.349387:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228460544:192.168.203.114@tcp:250:kworker.0 00000800:00000100:0.0:1787830383.349482:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00080000:3.0:1787830383.349519:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830383.349523:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787830383.349525:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830383.349530:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6528 00000100:00080000:3.0:1787830383.349534:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830383.349539:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830383.349550:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830383.349565:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82580 x1874669228460672/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830383.350407:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830383/real 1787830383] req@ffff985610f82940 x1874669228460544/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830408 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830383.350432:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82940 x1874669228460544/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830408 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830383.350444:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830383.350449:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830383.350454:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228460544:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787830383.353284:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228460672:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830383.353374:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830383.353407:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830383/real 1787830383] req@ffff985610f82580 x1874669228460672/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830408 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830383.353426:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82580 x1874669228460672/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830408 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830383.353437:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830383.353442:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830383.353446:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228460672:192.168.203.114@tcp:38:kworker.0 00000100:00080000:3.0:1787830383.354222:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830383.354230:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787830383.354233:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830383.354241:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6528 00000100:00080000:3.0:1787830383.354251:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830383.354259:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830383.354272:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830383.354306:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80f00 x1874669228460800/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830383.354427:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228460800:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830383.354480:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830383.354499:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830383/real 1787830383] req@ffff985610f80f00 x1874669228460800/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830408 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830383.354520:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80f00 x1874669228460800/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830408 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830383.354529:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830383.354532:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830383.354535:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228460800:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830383.413368:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830384.309371:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830384.437890:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830385.332842:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830385.461089:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830386.357107:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830386.485028:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830387.380854:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830387.510472:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830388.404854:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787830388.468893:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830388.468904:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830388.468909:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830388.468923:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6534 00000100:00080000:0.0:1787830388.468937:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830388.468945:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787830388.468993:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6580 x1874669228460928/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830388.469030:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830388.469034:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830388.469036:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830388.469039:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6534 00000100:00080000:0.0:1787830388.469043:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830388.469047:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787830388.469060:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830388.469072:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5a40 x1874669228461056/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830388.469082:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830388.469086:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830388.469088:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830388.469091:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6534 00000100:00080000:0.0:1787830388.469095:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830388.469099:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787830388.469103:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830388.469116:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df70c0 x1874669228461184/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787830388.469227:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228460928:192.168.203.114@tcp:250:kworker.0 00000800:00000100:2.0:1787830388.469309:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830388.469360:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830388/real 1787830388] req@ffff985607df6580 x1874669228460928/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830413 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830388.469379:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6580 x1874669228460928/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830413 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830388.469391:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830388.469397:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830388.469401:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228460928:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787830388.469408:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228461056:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787830388.469444:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830388.469457:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830388/real 1787830388] req@ffff985607df5a40 x1874669228461056/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830413 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830388.469468:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5a40 x1874669228461056/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830413 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830388.469489:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830388.469492:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830388.469496:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228461056:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787830388.469502:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228461184:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787830388.469526:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830388.469537:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830388/real 1787830388] req@ffff985607df70c0 x1874669228461184/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830413 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830388.469548:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df70c0 x1874669228461184/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830413 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830388.469555:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830388.469559:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830388.469562:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228461184:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830388.532837:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830389.429434:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830389.556953:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830390.452911:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830390.580943:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830391.476846:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830391.604977:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830392.500875:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830392.630437:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830393.524790:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787830393.588948:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830393.588957:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830393.588961:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830393.588972:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6539 00000100:00080000:1.0:1787830393.588982:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830393.588990:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787830393.589025:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a03c0 x1874669228461312/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787830393.589098:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228461312:192.168.203.114@tcp:250:kworker.0 00000800:00000100:2.0:1787830393.589154:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00080000:1.0:1787830393.589186:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830393.589193:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830393.589195:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830393.589200:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6539 00000100:00080000:1.0:1787830393.589203:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830393.589207:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830393.589216:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830393.589218:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830393/real 1787830393] req@ffff9856085a03c0 x1874669228461312/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830418 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830393.589228:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3c00 x1874669228461440/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787830393.589234:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a03c0 x1874669228461312/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830418 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830393.589237:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830393.589241:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830393.589243:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830393.589244:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830393.589245:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6539 00000100:00080000:1.0:1787830393.589248:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830393.589248:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830393.589251:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228461312:192.168.203.114@tcp:250:kworker.0 00000100:00080000:1.0:1787830393.589252:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830393.589256:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830393.589263:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0780 x1874669228461568/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787830393.589272:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228461440:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787830393.589304:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830393.589816:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830393/real 1787830393] req@ffff9856085a3c00 x1874669228461440/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830418 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830393.589829:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3c00 x1874669228461440/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830418 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830393.589837:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830393.589840:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830393.589843:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228461440:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787830393.589857:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228461568:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787830393.589883:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830393.589894:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830393/real 1787830393] req@ffff9856085a0780 x1874669228461568/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830418 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830393.589905:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0780 x1874669228461568/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830418 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830393.589913:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830393.589916:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830393.589919:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228461568:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830393.653058:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830394.548821:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830394.677433:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830395.572877:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830395.701817:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830396.597495:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830396.724943:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830397.621137:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830397.749124:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830398.645203:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787830398.709791:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830398.709804:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787830398.709809:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830398.709822:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6544 00000100:00080000:3.0:1787830398.709836:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830398.709844:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787830398.709899:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83480 x1874669228461696/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830398.709927:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830398.709931:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787830398.709933:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830398.709937:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6544 00000100:00080000:3.0:1787830398.709940:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830398.709943:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830398.709954:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830398.709966:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f821c0 x1874669228461824/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830398.709975:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830398.709980:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787830398.709981:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830398.709985:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6544 00000100:00080000:3.0:1787830398.709989:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830398.709992:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830398.709995:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830398.710005:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80f00 x1874669228461952/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787830398.710037:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228461696:192.168.203.114@tcp:250:kworker.0 00000800:00000100:2.0:1787830398.710116:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830398.710153:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830398/real 1787830398] req@ffff985610f83480 x1874669228461696/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830423 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830398.710170:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83480 x1874669228461696/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830423 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830398.710181:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830398.710185:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830398.710189:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228461696:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787830398.710197:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f821c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228461824:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787830398.710222:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830398.710235:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830398/real 1787830398] req@ffff985610f821c0 x1874669228461824/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830423 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830398.710248:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f821c0 x1874669228461824/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830423 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830398.710257:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830398.710261:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830398.710264:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f821c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228461824:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787830398.710291:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228461952:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787830398.710314:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830398.710325:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830398/real 1787830398] req@ffff985610f80f00 x1874669228461952/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830423 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830398.710337:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80f00 x1874669228461952/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830423 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830398.710344:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830398.710348:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830398.710351:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228461952:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830398.773460:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830399.668852:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830399.796929:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830400.692806:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830400.822357:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830401.716779:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830401.846551:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830402.740809:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830402.868972:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830403.765060:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787830403.828813:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830403.828823:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787830403.828828:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830403.828837:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6549 00000100:00080000:3.0:1787830403.828847:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830403.828855:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787830403.828887:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82580 x1874669228462080/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830403.828915:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830403.828918:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787830403.828920:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830403.828924:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6549 00000100:00080000:3.0:1787830403.828927:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830403.828931:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830403.828939:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830403.828950:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82940 x1874669228462208/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830403.828960:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830403.828962:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787830403.828964:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830403.828967:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6549 00000100:00080000:3.0:1787830403.828971:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830403.828974:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830403.828979:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830403.828988:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82d00 x1874669228462336/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787830403.829124:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228462080:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787830403.829211:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228462208:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787830403.829235:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228462336:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830403.893106:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830404.789006:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830404.916903:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830405.812850:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830405.941639:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830406.837723:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000400:00000100:1.0:1787830406.901311:0:2371:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.114:988 00000400:00000100:1.0:1787830406.901369:0:2371:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.114@tcp at host 192.168.203.114 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1787830406.901389:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:1.0:1787830406.901433:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:1.0:1787830406.901443:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000100:00100000:2.0:1787830406.901514:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830403/real 1787830406] req@ffff985610f82580 x1874669228462080/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830428 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830406.901545:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82580 x1874669228462080/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830428 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830406.901555:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830406.901560:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830406.901563:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228462080:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787830406.901571:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830403/real 1787830406] req@ffff985610f82940 x1874669228462208/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830428 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830406.901581:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82940 x1874669228462208/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830428 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830406.901588:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830406.901591:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830406.901594:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228462208:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787830406.901600:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830403/real 1787830406] req@ffff985610f82d00 x1874669228462336/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830428 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830406.901623:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82d00 x1874669228462336/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830428 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830406.901631:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830406.901634:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830406.901637:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228462336:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787830406.965249:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830407.860905:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830407.989006:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830408.884834:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787830408.952238:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830408.952252:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830408.952257:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830408.952269:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6554 00000100:00080000:1.0:1787830408.952291:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830408.952300:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787830408.952347:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a1680 x1874669228462464/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830408.952380:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830408.952383:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830408.952385:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830408.952389:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6554 00000100:00080000:1.0:1787830408.952392:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830408.952396:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830408.952407:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830408.952419:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0b40 x1874669228462592/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830408.952430:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1787830408.952433:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228462464:192.168.203.114@tcp:250:kworker.0 00000100:00080000:1.0:1787830408.952435:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830408.952439:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830408.952446:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6554 00000100:00080000:1.0:1787830408.952449:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830408.952453:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830408.952457:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830408.952469:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0000 x1874669228462720/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787830408.952524:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830408.952563:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830408/real 1787830408] req@ffff9856085a1680 x1874669228462464/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830433 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830408.952581:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a1680 x1874669228462464/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830433 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830408.952592:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830408.952596:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830408.952601:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228462464:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787830408.952630:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228462592:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787830408.952656:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830408.952669:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830408/real 1787830408] req@ffff9856085a0b40 x1874669228462592/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830433 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830408.952681:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0b40 x1874669228462592/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830433 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830408.952721:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830408.952726:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830408.952730:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228462592:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787830408.952736:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228462720:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787830408.952757:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830408.952769:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830408/real 1787830408] req@ffff9856085a0000 x1874669228462720/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830433 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830408.952795:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0000 x1874669228462720/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830433 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830408.952803:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830408.952807:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830408.952810:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228462720:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830409.012914:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830409.908979:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830410.037389:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830410.933121:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830411.060986:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830411.957981:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830412.084963:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830412.980863:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830413.109295:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830414.005173:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787830414.068865:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830414.068874:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787830414.068878:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830414.068892:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6559 00000100:00080000:3.0:1787830414.068906:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830414.068914:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787830414.068954:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f803c0 x1874669228462848/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830414.068993:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830414.068996:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787830414.068998:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830414.069002:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6559 00000100:00080000:3.0:1787830414.069006:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830414.069010:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830414.069019:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830414.069030:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83840 x1874669228462976/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830414.069040:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830414.069043:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787830414.069045:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830414.069048:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6559 00000100:00080000:3.0:1787830414.069052:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830414.069055:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830414.069060:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830414.069070:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81e00 x1874669228463104/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787830414.069212:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228462848:192.168.203.114@tcp:250:kworker.0 00000800:00000100:2.0:1787830414.069282:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830414.069313:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830414/real 1787830414] req@ffff985610f803c0 x1874669228462848/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830439 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830414.069329:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f803c0 x1874669228462848/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830439 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830414.069338:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830414.069342:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830414.069347:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228462848:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787830414.069354:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228462976:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787830414.069377:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830414.069389:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830414/real 1787830414] req@ffff985610f83840 x1874669228462976/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830439 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830414.069413:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83840 x1874669228462976/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830439 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830414.069421:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830414.069425:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830414.069428:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228462976:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787830414.069434:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228463104:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787830414.069452:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830414.069463:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830414/real 1787830414] req@ffff985610f81e00 x1874669228463104/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830439 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830414.069474:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81e00 x1874669228463104/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830439 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830414.069481:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830414.069484:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830414.069487:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228463104:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830414.132836:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830415.028881:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830415.157487:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830416.053309:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830416.182420:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830417.076794:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830417.204908:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830418.100953:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830418.228906:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830419.124989:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787830419.189792:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830419.189802:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787830419.189806:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830419.189815:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6564 00000100:00080000:3.0:1787830419.189826:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830419.189833:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787830419.189865:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81680 x1874669228463232/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830419.189899:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830419.189903:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787830419.189905:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830419.189908:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6564 00000100:00080000:3.0:1787830419.189912:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830419.189916:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830419.189924:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830419.189936:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f812c0 x1874669228463360/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830419.189945:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830419.189948:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787830419.189950:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830419.189954:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6564 00000100:00080000:3.0:1787830419.189957:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830419.189960:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830419.189964:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830419.189972:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80b40 x1874669228463488/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787830419.190254:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228463232:192.168.203.114@tcp:250:kworker.0 00000800:00000100:2.0:1787830419.190326:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830419.190359:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830419/real 1787830419] req@ffff985610f81680 x1874669228463232/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830444 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830419.190375:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81680 x1874669228463232/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830444 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830419.190386:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830419.190390:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830419.190394:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228463232:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787830419.190401:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228463360:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787830419.190422:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830419.190433:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830419/real 1787830419] req@ffff985610f812c0 x1874669228463360/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830444 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830419.190446:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f812c0 x1874669228463360/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830444 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830419.190454:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830419.190461:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830419.190465:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228463360:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787830419.190471:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228463488:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787830419.190490:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830419.190501:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830419/real 1787830419] req@ffff985610f80b40 x1874669228463488/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830444 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830419.190511:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80b40 x1874669228463488/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830444 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830419.190518:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830419.190521:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830419.190524:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228463488:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830419.252825:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830420.149011:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830420.276897:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830421.172829:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830421.300972:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830422.197149:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830422.324987:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830423.221375:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830423.349012:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830424.244855:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787830424.308878:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830424.308891:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830424.308896:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830424.308908:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6569 00000100:00080000:2.0:1787830424.308922:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830424.308930:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787830424.308978:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140dad00 x1874669228463616/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830424.309016:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830424.309020:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830424.309022:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830424.309026:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6569 00000100:00080000:2.0:1787830424.309030:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830424.309035:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830424.309046:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830424.309059:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8f00 x1874669228463744/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830424.309068:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830424.309072:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830424.309075:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830424.309079:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6569 00000100:00080000:2.0:1787830424.309083:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830424.309087:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830424.309093:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830424.309094:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228463616:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787830424.309109:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140da1c0 x1874669228463872/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787830424.309188:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830424.309230:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830424/real 1787830424] req@ffff9856140dad00 x1874669228463616/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830449 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830424.309249:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140dad00 x1874669228463616/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830449 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830424.309261:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830424.309265:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830424.309269:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228463616:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787830424.309276:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228463744:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830424.309297:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830424.309308:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830424/real 1787830424] req@ffff9856140d8f00 x1874669228463744/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830449 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830424.309319:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8f00 x1874669228463744/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830449 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830424.309327:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830424.309330:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830424.309333:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228463744:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787830424.309366:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228463872:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830424.309386:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830424.309397:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830424/real 1787830424] req@ffff9856140da1c0 x1874669228463872/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830449 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830424.309408:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140da1c0 x1874669228463872/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830449 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830424.309415:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830424.309419:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830424.309422:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228463872:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830424.373411:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830425.268866:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830425.397044:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830426.292872:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830426.420922:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830427.317063:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830427.444965:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830428.340819:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830428.470331:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830429.368271:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787830429.429087:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830429.429100:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787830429.429105:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830429.429118:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6575 00000100:00080000:3.0:1787830429.429132:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830429.429149:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787830429.429195:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80000 x1874669228464000/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830429.429224:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830429.429229:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787830429.429231:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830429.429238:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6575 00000100:00080000:3.0:1787830429.429243:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830429.429247:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830429.429258:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830429.429272:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81a40 x1874669228464128/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830429.429277:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228464000:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787830429.429290:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830429.429294:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787830429.429296:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830429.429300:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6575 00000100:00080000:3.0:1787830429.429303:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830429.429306:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830429.429311:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830429.429321:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f830c0 x1874669228464256/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787830429.429374:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830429.429414:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830429/real 1787830429] req@ffff985610f80000 x1874669228464000/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830454 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830429.429432:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80000 x1874669228464000/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830454 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830429.429442:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830429.429447:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830429.429451:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228464000:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787830429.429478:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228464128:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830429.429500:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830429.429512:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830429/real 1787830429] req@ffff985610f81a40 x1874669228464128/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830454 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830429.429524:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81a40 x1874669228464128/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830454 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830429.429532:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830429.429535:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830429.429539:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228464128:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787830429.429545:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228464256:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830429.429562:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830429.429573:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830429/real 1787830429] req@ffff985610f830c0 x1874669228464256/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830454 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830429.429585:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f830c0 x1874669228464256/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830454 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830429.429593:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830429.429596:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830429.429600:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228464256:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830429.492843:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830430.390155:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830430.518086:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830431.412827:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830431.540909:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830432.436838:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830432.564899:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830433.460834:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830433.589170:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830434.485023:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787830434.549309:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830434.549324:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787830434.549329:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830434.549343:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6580 00000100:00080000:3.0:1787830434.549357:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830434.549366:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787830434.549416:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83c00 x1874669228464384/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830434.549454:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830434.549459:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787830434.549461:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830434.549465:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6580 00000100:00080000:3.0:1787830434.549469:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830434.549474:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830434.549495:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830434.549508:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80780 x1874669228464512/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830434.549519:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830434.549523:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787830434.549525:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1787830434.549812:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228464384:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787830434.549849:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6580 00000100:00080000:3.0:1787830434.549855:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830434.549861:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830434.549868:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830434.549884:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f830c0 x1874669228464640/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787830434.549899:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830434.549943:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830434/real 1787830434] req@ffff985610f83c00 x1874669228464384/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830459 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830434.549961:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83c00 x1874669228464384/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830459 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830434.549973:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830434.549977:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830434.549981:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228464384:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787830434.549988:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228464512:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830434.550012:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830434.550024:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830434/real 1787830434] req@ffff985610f80780 x1874669228464512/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830459 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830434.550034:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80780 x1874669228464512/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830459 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830434.550041:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830434.550044:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830434.550047:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228464512:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787830434.550071:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228464640:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830434.550087:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830434.550095:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830434/real 1787830434] req@ffff985610f830c0 x1874669228464640/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830459 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830434.550116:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f830c0 x1874669228464640/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830459 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830434.550124:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830434.550127:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830434.550130:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228464640:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830434.613327:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830435.508767:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830435.637959:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830436.533006:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830436.660967:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830437.556827:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830437.685499:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830438.580983:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830438.708924:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830439.604841:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787830439.668827:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830439.668838:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787830439.668843:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830439.668851:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6585 00000100:00080000:3.0:1787830439.668861:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830439.668878:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787830439.668922:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81a40 x1874669228464768/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830439.668950:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830439.668953:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787830439.668955:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830439.668959:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6585 00000100:00080000:3.0:1787830439.668963:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830439.668967:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830439.668976:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830439.668986:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80000 x1874669228464896/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830439.668993:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830439.668995:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787830439.669003:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830439.669007:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6585 00000100:00080000:3.0:1787830439.669011:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00100000:0.0:1787830439.669015:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228464768:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787830439.669018:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830439.669027:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830439.669038:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80b40 x1874669228465024/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787830439.669089:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830439.669122:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830439/real 1787830439] req@ffff985610f81a40 x1874669228464768/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830464 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830439.669135:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81a40 x1874669228464768/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830464 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830439.669142:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830439.669145:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830439.669147:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228464768:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787830439.669152:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228464896:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830439.669170:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830439.669181:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830439/real 1787830439] req@ffff985610f80000 x1874669228464896/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830464 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830439.669192:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80000 x1874669228464896/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830464 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830439.669201:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830439.669204:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830439.669207:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228464896:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787830439.669227:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228465024:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830439.669241:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830439.669249:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830439/real 1787830439] req@ffff985610f80b40 x1874669228465024/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830464 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830439.669265:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80b40 x1874669228465024/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830464 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830439.669273:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830439.669276:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830439.669279:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228465024:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830439.732879:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830440.629016:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830440.756857:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830441.654892:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830441.781025:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830442.676855:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830442.805562:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830443.700822:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830443.828886:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830444.725169:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787830444.789166:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830444.789177:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830444.789181:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830444.789191:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6590 00000100:00080000:2.0:1787830444.789202:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830444.789210:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787830444.789242:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db840 x1874669228465152/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830444.789286:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830444.789295:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830444.789297:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830444.789301:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6590 00000100:00080000:2.0:1787830444.789306:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830444.789310:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830444.789319:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830444.789338:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140dbc00 x1874669228465280/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830444.789348:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830444.789359:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830444.789361:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830444.789365:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6590 00000100:00080000:2.0:1787830444.789368:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830444.789373:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830444.789377:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830444.789395:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140da1c0 x1874669228465408/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830444.789412:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228465152:192.168.203.114@tcp:250:kworker.0 00000800:00000100:3.0:1787830444.789472:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830444.790568:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830444/real 1787830444] req@ffff9856140db840 x1874669228465152/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830469 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830444.790589:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db840 x1874669228465152/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830469 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830444.790599:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830444.790603:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830444.790608:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228465152:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787830444.790618:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228465280:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830444.790649:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830444.790662:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830444/real 1787830444] req@ffff9856140dbc00 x1874669228465280/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830469 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830444.790673:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140dbc00 x1874669228465280/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830469 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830444.790681:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830444.790685:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830444.790688:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228465280:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787830444.790731:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228465408:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830444.790756:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830444.790772:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830444/real 1787830444] req@ffff9856140da1c0 x1874669228465408/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830469 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830444.790790:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140da1c0 x1874669228465408/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830469 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830444.790797:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830444.790801:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830444.790805:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228465408:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830444.854373:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830445.748889:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830445.876960:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830446.772812:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830446.900893:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830447.796824:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830447.924861:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830448.820983:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830448.950344:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830449.844922:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787830449.908821:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830449.908831:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787830449.908834:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830449.908844:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6595 00000100:00080000:3.0:1787830449.908853:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830449.908860:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787830449.908894:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f812c0 x1874669228465536/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830449.908918:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830449.908921:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787830449.908923:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830449.908925:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6595 00000100:00080000:3.0:1787830449.908927:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830449.908930:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830449.908937:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830449.908945:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81680 x1874669228465664/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830449.908955:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830449.908957:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787830449.908958:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830449.908961:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6595 00000100:00080000:3.0:1787830449.908962:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830449.908964:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830449.908967:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830449.908970:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228465536:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787830449.908975:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81e00 x1874669228465792/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787830449.909047:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830449.909080:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830449/real 1787830449] req@ffff985610f812c0 x1874669228465536/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830474 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830449.909097:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f812c0 x1874669228465536/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830474 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830449.909108:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830449.909112:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830449.909116:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228465536:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787830449.909123:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228465664:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830449.909145:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830449.909158:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830449/real 1787830449] req@ffff985610f81680 x1874669228465664/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830474 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830449.909168:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81680 x1874669228465664/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830474 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830449.909176:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830449.909178:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830449.909181:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228465664:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787830449.909203:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228465792:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830449.909223:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830449.909234:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830449/real 1787830449] req@ffff985610f81e00 x1874669228465792/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830474 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830449.909244:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81e00 x1874669228465792/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830474 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830449.909264:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830449.909267:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830449.909271:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228465792:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830449.973047:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830450.868844:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830450.997211:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830451.892802:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830452.021037:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830452.916806:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830453.044896:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830453.940869:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830454.070501:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830454.964865:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787830455.028914:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830455.028925:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830455.028930:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830455.028941:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6600 00000100:00080000:0.0:1787830455.028952:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830455.028959:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787830455.029008:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5680 x1874669228465920/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830455.029043:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830455.029046:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830455.029048:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830455.029052:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6600 00000100:00080000:0.0:1787830455.029056:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830455.029061:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787830455.029071:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830455.029083:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df43c0 x1874669228466048/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830455.029093:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830455.029096:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830455.029097:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830455.029101:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6600 00000100:00080000:0.0:1787830455.029105:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830455.029108:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787830455.029113:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830455.029124:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df61c0 x1874669228466176/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830455.029217:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228465920:192.168.203.114@tcp:250:kworker.0 00000800:00000100:3.0:1787830455.029317:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830455.029364:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830455/real 1787830455] req@ffff985607df5680 x1874669228465920/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830480 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830455.029383:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5680 x1874669228465920/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830480 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830455.029395:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830455.029399:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830455.029403:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228465920:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787830455.029411:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228466048:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830455.029432:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830455.029443:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830455/real 1787830455] req@ffff985607df43c0 x1874669228466048/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830480 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830455.029454:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df43c0 x1874669228466048/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830480 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830455.029462:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830455.029465:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830455.029468:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228466048:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787830455.029473:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228466176:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830455.029558:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830455.029571:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830455/real 1787830455] req@ffff985607df61c0 x1874669228466176/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830480 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830455.029582:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df61c0 x1874669228466176/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830480 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830455.029589:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830455.029593:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830455.029596:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228466176:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830455.092830:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830455.989978:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830456.116919:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830457.012798:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830457.140902:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830458.036867:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830458.164916:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830459.060876:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830459.189208:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830460.084837:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787830460.148915:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830460.148940:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787830460.148946:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830460.148959:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6605 00000100:00080000:3.0:1787830460.148974:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830460.148983:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787830460.149030:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83840 x1874669228466304/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830460.149107:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228466304:192.168.203.114@tcp:250:kworker.0 00000800:00000100:0.0:1787830460.149187:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00080000:3.0:1787830460.149222:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830460.149226:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787830460.149227:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830460.149232:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6605 00000100:00080000:3.0:1787830460.149235:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830460.149239:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830460.149250:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830460.149263:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f803c0 x1874669228466432/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830460.150175:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830460/real 1787830460] req@ffff985610f83840 x1874669228466304/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830485 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830460.150199:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83840 x1874669228466304/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830485 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830460.150212:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830460.150216:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830460.150220:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228466304:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787830460.150259:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228466432:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830460.150304:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830460.150318:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830460/real 1787830460] req@ffff985610f803c0 x1874669228466432/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830485 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830460.150330:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f803c0 x1874669228466432/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830485 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830460.150338:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830460.150341:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830460.150344:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228466432:192.168.203.114@tcp:38:kworker.0 00000100:00080000:3.0:1787830460.151664:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830460.151674:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787830460.151677:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830460.151686:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6605 00000100:00080000:3.0:1787830460.151711:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830460.151720:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830460.151734:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830460.151770:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82d00 x1874669228466560/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830460.152004:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228466560:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830460.152067:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830460.152095:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830460/real 1787830460] req@ffff985610f82d00 x1874669228466560/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830485 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830460.152826:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82d00 x1874669228466560/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830485 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830460.152840:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830460.152845:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830460.152849:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228466560:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830460.213155:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830461.108972:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830461.240041:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830462.132821:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830462.263991:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830463.156858:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830463.284877:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830464.183345:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830464.309078:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830465.204816:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787830465.268836:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830465.268846:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787830465.268850:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830465.268860:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6610 00000100:00080000:3.0:1787830465.268872:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830465.268880:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787830465.268925:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82940 x1874669228466688/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830465.268957:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830465.268960:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787830465.268961:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830465.268965:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6610 00000100:00080000:3.0:1787830465.268969:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830465.268973:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830465.268982:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830465.268993:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82580 x1874669228466816/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830465.269001:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830465.269005:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787830465.269007:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830465.269010:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6610 00000100:00080000:3.0:1787830465.269013:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830465.269016:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830465.269020:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830465.269029:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80f00 x1874669228466944/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830465.269352:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228466688:192.168.203.114@tcp:250:kworker.0 00000800:00000100:0.0:1787830465.269417:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830465.269447:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830465/real 1787830465] req@ffff985610f82940 x1874669228466688/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830490 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830465.269467:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82940 x1874669228466688/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830490 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830465.269477:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830465.269480:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830465.269484:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228466688:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787830465.269494:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228466816:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830465.269515:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830465.269526:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830465/real 1787830465] req@ffff985610f82580 x1874669228466816/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830490 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830465.269537:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82580 x1874669228466816/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830490 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830465.269545:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830465.269548:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830465.269551:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228466816:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787830465.269557:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228466944:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830465.269574:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830465.269585:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830465/real 1787830465] req@ffff985610f80f00 x1874669228466944/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830490 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830465.269594:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80f00 x1874669228466944/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830490 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830465.269606:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830465.269609:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830465.269612:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228466944:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830465.332845:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830466.228843:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830466.356994:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830467.252970:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830467.381738:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830468.276806:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830468.404885:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830469.300800:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830469.428877:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830470.324785:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787830470.388818:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830470.388828:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830470.388831:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830470.388841:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6616 00000100:00080000:1.0:1787830470.388852:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830470.388871:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787830470.388915:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2d00 x1874669228467072/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830470.388938:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830470.388942:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830470.388944:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830470.388947:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6616 00000100:00080000:1.0:1787830470.388957:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830470.388962:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830470.388971:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830470.388985:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0000 x1874669228467200/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830470.388987:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228467072:192.168.203.114@tcp:250:kworker.0 00000100:00080000:1.0:1787830470.388997:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830470.389000:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830470.389001:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830470.389003:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6616 00000100:00080000:1.0:1787830470.389005:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830470.389007:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830470.389011:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830470.389017:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0b40 x1874669228467328/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830470.389097:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228467200:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787830470.389135:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228467328:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830470.452820:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830471.348968:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830471.476935:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830472.372832:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830472.502976:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830473.396818:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000400:00000100:0.0:1787830473.461575:0:2372:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.114:988 00000400:00000100:0.0:1787830473.461615:0:2372:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.114@tcp at host 192.168.203.114 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:0.0:1787830473.461631:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:0.0:1787830473.461670:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:0.0:1787830473.461681:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000100:00100000:1.0:1787830473.462585:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830470/real 1787830473] req@ffff9856085a0000 x1874669228467200/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830495 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830473.462611:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0000 x1874669228467200/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830495 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830473.462621:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830473.462626:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830473.462630:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228467200:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787830473.462636:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830470/real 1787830473] req@ffff9856085a0b40 x1874669228467328/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830495 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830473.462645:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0b40 x1874669228467328/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830495 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830473.462651:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830473.462654:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830473.462657:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228467328:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787830473.462663:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830470/real 1787830473] req@ffff9856085a2d00 x1874669228467072/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830495 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830473.462673:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2d00 x1874669228467072/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830495 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830473.462678:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830473.462681:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830473.462683:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228467072:192.168.203.114@tcp:250:kworker.0 00010000:00010000:2.0:1787830473.525172:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830474.420814:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830474.549022:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830475.444834:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787830475.508896:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830475.508911:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830475.508916:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830475.508929:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6621 00000100:00080000:0.0:1787830475.508945:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830475.508954:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787830475.509003:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5e00 x1874669228467456/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830475.509043:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830475.509047:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830475.509049:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830475.509054:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6621 00000100:00080000:0.0:1787830475.509057:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830475.509062:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787830475.509073:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830475.509086:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6d00 x1874669228467584/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830475.509096:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830475.509099:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830475.509101:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830475.509105:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6621 00000100:00080000:0.0:1787830475.509109:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830475.509113:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787830475.509118:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830475.509129:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4f00 x1874669228467712/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787830475.510020:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228467456:192.168.203.114@tcp:250:kworker.0 00000800:00000100:1.0:1787830475.510094:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830475.510129:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830475/real 1787830475] req@ffff985607df5e00 x1874669228467456/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830500 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830475.510146:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5e00 x1874669228467456/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830500 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830475.510156:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830475.510160:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830475.510164:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228467456:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787830475.510171:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228467584:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830475.510197:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830475.510208:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830475/real 1787830475] req@ffff985607df6d00 x1874669228467584/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830500 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830475.510218:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6d00 x1874669228467584/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830500 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830475.510225:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830475.510228:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830475.510230:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228467584:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787830475.510236:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228467712:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830475.510254:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830475.510264:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830475/real 1787830475] req@ffff985607df4f00 x1874669228467712/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830500 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830475.510274:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4f00 x1874669228467712/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830500 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830475.510286:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830475.510289:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830475.510292:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228467712:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830475.573067:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830476.468800:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830476.597548:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830477.492855:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830477.620878:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830478.516834:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830478.644886:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830479.540863:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830479.670405:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830480.564819:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787830480.628835:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830480.628846:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830480.628851:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830480.628861:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6626 00000100:00080000:0.0:1787830480.628873:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830480.628883:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787830480.628925:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7840 x1874669228467840/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830480.628971:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830480.628977:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830480.628979:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830480.628985:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6626 00000100:00080000:0.0:1787830480.628991:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830480.628996:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787830480.629006:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830480.629027:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df52c0 x1874669228467968/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830480.629040:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830480.629044:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830480.629047:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830480.629053:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6626 00000100:00080000:0.0:1787830480.629059:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830480.629065:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787830480.629072:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830480.629089:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7480 x1874669228468096/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787830480.629266:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228467840:192.168.203.114@tcp:250:kworker.0 00000800:00000100:1.0:1787830480.629335:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830480.629375:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830480/real 1787830480] req@ffff985607df7840 x1874669228467840/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830505 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830480.629397:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7840 x1874669228467840/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830505 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830480.629409:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830480.629414:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830480.629418:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228467840:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787830480.629427:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228467968:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830480.629463:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830480.629497:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830480/real 1787830480] req@ffff985607df52c0 x1874669228467968/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830505 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830480.629515:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df52c0 x1874669228467968/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830505 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830480.629525:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830480.629528:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830480.629532:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228467968:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787830480.629540:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228468096:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830480.629571:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830480.629589:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830480/real 1787830480] req@ffff985607df7480 x1874669228468096/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830505 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830480.629603:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7480 x1874669228468096/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830505 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830480.629612:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830480.629615:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830480.629619:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228468096:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830480.692867:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830481.588818:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830481.716894:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830482.612955:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830482.740883:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830483.636819:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830483.765653:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830484.660847:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830484.789673:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830485.685196:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787830485.749149:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830485.749159:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787830485.749162:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830485.749171:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6631 00000100:00080000:3.0:1787830485.749181:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830485.749187:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787830485.749219:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f821c0 x1874669228468224/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830485.749237:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830485.749239:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787830485.749241:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830485.749245:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6631 00000100:00080000:3.0:1787830485.749248:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830485.749251:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830485.749260:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830485.749270:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83480 x1874669228468352/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830485.749279:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830485.749282:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787830485.749283:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830485.749286:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6631 00000100:00080000:3.0:1787830485.749289:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830485.749292:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830485.749296:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830485.749304:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80f00 x1874669228468480/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787830485.749392:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f821c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228468224:192.168.203.114@tcp:250:kworker.0 00000800:00000100:1.0:1787830485.749449:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830485.749476:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830485/real 1787830485] req@ffff985610f821c0 x1874669228468224/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830510 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830485.749492:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f821c0 x1874669228468224/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830510 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830485.749502:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830485.749505:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830485.749509:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f821c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228468224:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787830485.749515:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228468352:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830485.749535:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830485.749547:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830485/real 1787830485] req@ffff985610f83480 x1874669228468352/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830510 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830485.749560:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83480 x1874669228468352/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830510 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830485.749568:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830485.749571:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830485.749574:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228468352:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787830485.749580:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228468480:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830485.749598:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830485.749609:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830485/real 1787830485] req@ffff985610f80f00 x1874669228468480/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830510 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830485.749620:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80f00 x1874669228468480/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830510 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830485.749628:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830485.749631:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830485.749634:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228468480:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830485.812807:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830486.708841:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830486.837229:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830487.732890:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830487.861006:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830488.756860:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830488.885356:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830489.780846:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830489.908905:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830490.804833:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787830490.869661:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830490.869670:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787830490.869674:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830490.869683:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6636 00000100:00080000:3.0:1787830490.869724:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830490.869732:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787830490.869770:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82580 x1874669228468608/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830490.869799:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830490.869803:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787830490.869805:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830490.869809:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6636 00000100:00080000:3.0:1787830490.869812:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830490.869816:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830490.869825:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830490.869836:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82940 x1874669228468736/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830490.869844:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830490.869847:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787830490.869849:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830490.869853:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6636 00000100:00080000:3.0:1787830490.869856:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830490.869859:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830490.869864:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830490.869873:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82d00 x1874669228468864/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787830490.869965:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228468608:192.168.203.114@tcp:250:kworker.0 00000800:00000100:1.0:1787830490.870026:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830490.870056:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830490/real 1787830490] req@ffff985610f82580 x1874669228468608/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830515 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830490.870071:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82580 x1874669228468608/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830515 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830490.870086:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830490.870090:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830490.870093:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228468608:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787830490.870100:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228468736:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830490.870120:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830490.870131:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830490/real 1787830490] req@ffff985610f82940 x1874669228468736/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830515 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830490.870143:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82940 x1874669228468736/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830515 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830490.870151:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830490.870153:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830490.870156:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228468736:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787830490.870162:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228468864:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830490.870179:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830490.870188:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830490/real 1787830490] req@ffff985610f82d00 x1874669228468864/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830515 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830490.870199:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82d00 x1874669228468864/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830515 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830490.870207:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830490.870210:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830490.870213:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228468864:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830490.932890:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830491.829027:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830491.957201:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830492.852808:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830492.980871:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830493.876813:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830494.005632:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830494.901374:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830495.029212:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830495.924844:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787830495.988897:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830495.988907:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787830495.988911:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830495.988920:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6641 00000100:00080000:3.0:1787830495.988930:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830495.988938:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787830495.988974:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f803c0 x1874669228468992/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830495.988999:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830495.989002:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787830495.989004:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830495.989008:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6641 00000100:00080000:3.0:1787830495.989012:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830495.989016:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830495.989025:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830495.989048:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228468992:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787830495.989102:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83840 x1874669228469120/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787830495.989109:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00080000:3.0:1787830495.989120:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830495.989124:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787830495.989127:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830495.989132:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6641 00000100:00080000:3.0:1787830495.989137:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830495.989142:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830495.989148:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830495.989149:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830495/real 1787830495] req@ffff985610f803c0 x1874669228468992/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830520 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830495.989163:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81e00 x1874669228469248/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787830495.989172:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f803c0 x1874669228468992/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830520 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830495.989184:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830495.989189:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830495.989194:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228468992:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787830495.989217:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228469120:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830495.989239:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830495.989251:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830495/real 1787830495] req@ffff985610f83840 x1874669228469120/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830520 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830495.989263:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83840 x1874669228469120/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830520 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830495.989272:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830495.989275:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830495.989279:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228469120:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787830495.989285:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228469248:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830495.989303:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830495.989314:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830495/real 1787830495] req@ffff985610f81e00 x1874669228469248/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830520 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830495.989326:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81e00 x1874669228469248/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830520 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830495.989334:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830495.989337:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830495.989340:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228469248:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830496.052864:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830496.949895:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830497.076991:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830497.973026:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830498.101061:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830498.996995:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830499.124978:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830500.020866:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830500.150758:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830501.046079:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787830501.108824:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830501.108992:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787830501.108998:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830501.109012:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6646 00000100:00080000:3.0:1787830501.109027:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830501.109036:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787830501.109085:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81680 x1874669228469376/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830501.109124:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830501.109128:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787830501.109130:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830501.109134:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6646 00000100:00080000:3.0:1787830501.109138:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830501.109143:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830501.109155:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830501.109168:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f812c0 x1874669228469504/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830501.109177:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830501.109181:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787830501.109183:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830501.109186:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6646 00000100:00080000:3.0:1787830501.109190:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830501.109193:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830501.109198:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830501.109207:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80b40 x1874669228469632/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787830501.109314:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228469376:192.168.203.114@tcp:250:kworker.0 00000800:00000100:1.0:1787830501.109394:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830501.109432:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830501/real 1787830501] req@ffff985610f81680 x1874669228469376/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830526 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830501.109450:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81680 x1874669228469376/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830526 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830501.109461:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830501.109465:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830501.109469:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228469376:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787830501.109477:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228469504:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830501.109499:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830501.109511:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830501/real 1787830501] req@ffff985610f812c0 x1874669228469504/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830526 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830501.109522:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f812c0 x1874669228469504/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830526 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830501.109537:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830501.109540:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830501.109543:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228469504:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787830501.109549:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228469632:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830501.109566:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830501.109575:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830501/real 1787830501] req@ffff985610f80b40 x1874669228469632/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830526 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830501.109587:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80b40 x1874669228469632/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830526 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830501.109594:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830501.109597:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830501.109600:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228469632:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830501.172911:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830502.068844:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830502.197259:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830503.092844:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830503.221468:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830504.117016:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830504.244963:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830505.140831:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830505.268943:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830506.164837:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787830506.228943:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830506.228956:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787830506.228961:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830506.228973:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6651 00000100:00080000:3.0:1787830506.228986:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830506.228994:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787830506.229046:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80000 x1874669228469760/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830506.229074:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830506.229078:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787830506.229080:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830506.229084:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6651 00000100:00080000:3.0:1787830506.229087:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830506.229091:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830506.229101:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830506.229116:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81a40 x1874669228469888/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830506.229128:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830506.229131:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787830506.229133:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1787830506.229140:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228469760:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787830506.229141:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6651 00000100:00080000:3.0:1787830506.229147:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830506.229162:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830506.229168:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830506.229180:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f830c0 x1874669228470016/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787830506.229223:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830506.229262:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830506/real 1787830506] req@ffff985610f80000 x1874669228469760/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830531 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830506.229279:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80000 x1874669228469760/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830531 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830506.229292:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830506.229296:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830506.229300:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228469760:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787830506.229326:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228469888:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830506.229346:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830506.229356:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830506/real 1787830506] req@ffff985610f81a40 x1874669228469888/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830531 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830506.229368:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81a40 x1874669228469888/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830531 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830506.229376:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830506.229379:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830506.229382:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228469888:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787830506.229387:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228470016:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830506.229401:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830506.229410:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830506/real 1787830506] req@ffff985610f830c0 x1874669228470016/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830531 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830506.229420:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f830c0 x1874669228470016/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830531 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830506.229428:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830506.229431:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830506.229433:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228470016:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830506.292993:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830507.188850:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830507.318763:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830508.213040:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830508.341041:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830509.236903:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830509.366077:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830510.260834:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830510.389260:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830511.285226:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787830511.348846:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830511.348857:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787830511.348861:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830511.348874:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6656 00000100:00080000:3.0:1787830511.348886:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830511.348892:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787830511.348935:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80780 x1874669228470144/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830511.348968:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830511.348972:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787830511.348974:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830511.348978:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6656 00000100:00080000:3.0:1787830511.348981:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830511.348986:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830511.348998:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830511.349010:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83c00 x1874669228470272/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830511.349017:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830511.349020:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787830511.349021:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830511.349023:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6656 00000100:00080000:3.0:1787830511.349026:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830511.349028:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830511.349032:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830511.349040:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f830c0 x1874669228470400/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787830511.349200:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228470144:192.168.203.114@tcp:250:kworker.0 00000800:00000100:1.0:1787830511.349277:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830511.349320:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830511/real 1787830511] req@ffff985610f80780 x1874669228470144/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830536 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830511.349339:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80780 x1874669228470144/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830536 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830511.349350:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830511.349354:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830511.349359:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228470144:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787830511.349367:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228470272:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830511.349390:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830511.349403:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830511/real 1787830511] req@ffff985610f83c00 x1874669228470272/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830536 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830511.349415:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83c00 x1874669228470272/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830536 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830511.349423:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830511.349426:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830511.349429:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228470272:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787830511.349435:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228470400:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830511.349454:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830511.349464:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830511/real 1787830511] req@ffff985610f830c0 x1874669228470400/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830536 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830511.349475:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f830c0 x1874669228470400/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830536 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830511.349489:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830511.349493:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830511.349496:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228470400:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830511.413157:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830512.308865:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830512.437000:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830513.333305:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830513.464806:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830514.356912:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830514.484928:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830515.380838:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830515.509335:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830516.404871:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787830516.468800:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830516.468809:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787830516.468813:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830516.468821:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6662 00000100:00080000:3.0:1787830516.468830:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830516.468837:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787830516.468869:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81a40 x1874669228470528/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830516.468898:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830516.468901:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787830516.468903:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830516.468906:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6662 00000100:00080000:3.0:1787830516.468909:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830516.468913:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830516.468922:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830516.468932:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80000 x1874669228470656/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830516.468941:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830516.468943:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787830516.468945:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830516.468948:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6662 00000100:00080000:3.0:1787830516.468950:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830516.468953:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830516.468956:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830516.468963:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80b40 x1874669228470784/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787830516.469150:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228470528:192.168.203.114@tcp:250:kworker.0 00000800:00000100:1.0:1787830516.469209:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830516.469240:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830516/real 1787830516] req@ffff985610f81a40 x1874669228470528/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830541 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830516.469257:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81a40 x1874669228470528/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830541 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830516.469268:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830516.469272:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830516.469277:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228470528:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787830516.469285:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228470656:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830516.469309:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830516.469321:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830516/real 1787830516] req@ffff985610f80000 x1874669228470656/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830541 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830516.469333:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80000 x1874669228470656/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830541 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830516.469342:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830516.469346:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830516.469350:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228470656:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787830516.469357:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228470784:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830516.469375:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830516.469385:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830516/real 1787830516] req@ffff985610f80b40 x1874669228470784/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830541 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830516.469397:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80b40 x1874669228470784/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830541 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830516.469404:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830516.469407:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830516.469410:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228470784:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830516.532840:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830517.429272:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830517.557130:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830518.452805:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830518.580986:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830519.481894:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830519.605332:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830520.500800:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830520.629771:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830521.528861:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787830521.589189:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830521.589199:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787830521.589203:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830521.589213:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6667 00000100:00080000:3.0:1787830521.589223:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830521.589230:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787830521.589385:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f812c0 x1874669228470912/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787830521.589462:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228470912:192.168.203.114@tcp:250:kworker.0 00000800:00000100:1.0:1787830521.589541:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00080000:3.0:1787830521.589572:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830521.589577:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787830521.589578:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830521.589582:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6667 00000100:00080000:3.0:1787830521.589586:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830521.589591:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830521.589608:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830521.589621:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81680 x1874669228471040/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787830521.589667:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830521/real 1787830521] req@ffff985610f812c0 x1874669228470912/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830546 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830521.589682:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f812c0 x1874669228470912/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830546 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830521.589713:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830521.589717:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830521.589720:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228470912:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787830521.589742:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228471040:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830521.589763:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830521.589774:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830521/real 1787830521] req@ffff985610f81680 x1874669228471040/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830546 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830521.589785:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81680 x1874669228471040/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830546 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830521.589792:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830521.589795:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830521.589798:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228471040:192.168.203.114@tcp:38:kworker.0 00000100:00080000:3.0:1787830521.589888:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830521.589892:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787830521.589894:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830521.589898:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6667 00000100:00080000:3.0:1787830521.589902:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830521.589907:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830521.589912:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830521.589924:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81e00 x1874669228471168/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787830521.590179:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228471168:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830521.590203:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830521.590215:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830521/real 1787830521] req@ffff985610f81e00 x1874669228471168/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830546 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830521.590226:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81e00 x1874669228471168/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830546 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830521.590234:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830521.590237:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830521.590241:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228471168:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830521.652842:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830522.548796:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830522.676954:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830523.572837:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830523.700942:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830524.596978:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830524.726924:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830525.620857:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830525.748991:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830526.644839:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787830526.709024:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830526.709034:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830526.709038:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830526.709046:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6672 00000100:00080000:2.0:1787830526.709056:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830526.709061:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787830526.709094:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8f00 x1874669228471296/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787830526.709169:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228471296:192.168.203.114@tcp:250:kworker.0 00000800:00000100:1.0:1787830526.709223:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00080000:2.0:1787830526.709252:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830526.709256:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00100000:1.0:1787830526.709331:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830526/real 1787830526] req@ffff9856140d8f00 x1874669228471296/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830551 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830526.709347:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8f00 x1874669228471296/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830551 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830526.709362:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830526.709366:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830526.709370:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228471296:192.168.203.114@tcp:250:kworker.0 00000100:00080000:2.0:1787830526.709425:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830526.709429:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6672 00000100:00080000:2.0:1787830526.709433:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830526.709437:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830526.709445:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830526.709457:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140dad00 x1874669228471424/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830526.709489:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830526.709492:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830526.709495:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830526.709499:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6672 00000100:00080000:2.0:1787830526.709502:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830526.709506:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830526.709511:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830526.709521:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db0c0 x1874669228471552/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787830526.709611:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228471424:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830526.709634:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830526.709646:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830526/real 1787830526] req@ffff9856140dad00 x1874669228471424/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830551 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830526.709657:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140dad00 x1874669228471424/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830551 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830526.709665:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830526.709668:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830526.709671:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228471424:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787830526.709676:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228471552:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830526.709725:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830526.709736:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830526/real 1787830526] req@ffff9856140db0c0 x1874669228471552/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830551 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830526.709747:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db0c0 x1874669228471552/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830551 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830526.709754:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830526.709757:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830526.709769:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228471552:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830526.772873:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830527.668836:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830527.796908:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830528.692835:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830528.821243:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830529.717516:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830529.844991:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830530.740839:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830530.868992:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830531.764921:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787830531.828831:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830531.828845:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830531.828850:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830531.828863:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6677 00000100:00080000:2.0:1787830531.828877:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830531.828886:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787830531.829041:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140da940 x1874669228471680/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830531.829077:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830531.829081:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830531.829082:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830531.829087:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6677 00000100:00080000:2.0:1787830531.829091:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830531.829096:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830531.829109:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830531.829124:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d9a40 x1874669228471808/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830531.829135:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830531.829140:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:1.0:1787830531.829143:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228471680:192.168.203.114@tcp:250:kworker.0 00000100:00080000:2.0:1787830531.829143:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830531.829163:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6677 00000100:00080000:2.0:1787830531.829168:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830531.829173:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830531.829179:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830531.829193:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8b40 x1874669228471936/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787830531.829230:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830531.829272:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830531/real 1787830531] req@ffff9856140da940 x1874669228471680/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830556 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830531.829292:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140da940 x1874669228471680/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830556 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830531.829303:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830531.829307:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830531.829311:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228471680:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787830531.829340:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228471808:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830531.829364:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830531.829377:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830531/real 1787830531] req@ffff9856140d9a40 x1874669228471808/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830556 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830531.829389:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d9a40 x1874669228471808/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830556 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830531.829397:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830531.829401:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830531.829404:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228471808:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787830531.829411:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228471936:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830531.829447:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830531.829459:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830531/real 1787830531] req@ffff9856140d8b40 x1874669228471936/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830556 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830531.829470:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8b40 x1874669228471936/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830556 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830531.829478:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830531.829482:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830531.829485:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228471936:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830531.893110:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830532.788858:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830532.917202:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830533.812810:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830533.940876:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830534.836827:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830534.965141:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830535.860768:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830535.988971:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830536.884819:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787830536.949407:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830536.949416:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830536.949420:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830536.949428:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6682 00000100:00080000:2.0:1787830536.949438:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830536.949446:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787830536.949482:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d9680 x1874669228472064/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787830536.949558:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228472064:192.168.203.114@tcp:250:kworker.0 00000100:00080000:2.0:1787830536.949984:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830536.949989:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830536.949991:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830536.949995:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6682 00000100:00080000:2.0:1787830536.950000:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830536.950005:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830536.950019:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830536.950100:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d83c0 x1874669228472192/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830536.950112:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830536.950115:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830536.950117:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830536.950121:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6682 00000100:00080000:2.0:1787830536.950125:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830536.950129:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787830536.950144:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228472192:192.168.203.114@tcp:38:kworker.0 00000008:00100000:2.0:1787830536.950203:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830536.950215:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db480 x1874669228472320/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787830536.950253:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228472320:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830537.012813:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830537.910489:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830538.038005:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830538.932803:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830539.061992:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830539.956829:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000400:00000100:0.0:1787830540.022168:0:2373:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.114:988 00000400:00000100:0.0:1787830540.022226:0:2373:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.114@tcp at host 192.168.203.114 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:0.0:1787830540.022249:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:0.0:1787830540.022299:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:0.0:1787830540.022310:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000100:00100000:1.0:1787830540.022358:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830536/real 1787830540] req@ffff9856140db480 x1874669228472320/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830561 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830540.022399:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db480 x1874669228472320/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830561 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830540.022414:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830540.022419:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830540.022424:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228472320:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787830540.022433:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830536/real 1787830540] req@ffff9856140d83c0 x1874669228472192/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830561 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830540.022444:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d83c0 x1874669228472192/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830561 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830540.022458:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830540.022463:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830540.022466:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228472192:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787830540.022473:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830536/real 1787830540] req@ffff9856140d9680 x1874669228472064/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830561 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830540.022485:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d9680 x1874669228472064/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830561 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830540.022493:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830540.022497:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830540.022500:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228472064:192.168.203.114@tcp:250:kworker.0 00010000:00010000:2.0:1787830540.084872:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830540.980898:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830541.109757:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830542.004886:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787830542.068793:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830542.068802:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830542.068806:0:66417:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830542.068815:0:66417:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6687 00000100:00080000:2.0:1787830542.068824:0:66417:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830542.068831:0:66417:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787830542.068865:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8780 x1874669228472448/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830542.070293:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830542.070301:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830542.070304:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830542.070312:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6687 00000100:00080000:2.0:1787830542.070319:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830542.070325:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830542.070337:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830542.070372:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d9680 x1874669228472576/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787830542.070381:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228472448:192.168.203.114@tcp:250:kworker.0 00000100:00080000:2.0:1787830542.070389:0:66417:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830542.070395:0:66417:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830542.070397:0:66417:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830542.070403:0:66417:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6687 00000100:00080000:2.0:1787830542.070407:0:66417:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830542.070412:0:66417:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830542.070418:0:66417:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830542.070431:0:66417:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d83c0 x1874669228472704/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787830542.070488:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830542.070535:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830542/real 1787830542] req@ffff9856140d8780 x1874669228472448/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830567 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830542.070555:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8780 x1874669228472448/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830567 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830542.070567:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830542.070571:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830542.070576:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228472448:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787830542.070603:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228472576:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830542.070626:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830542.070637:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830542/real 1787830542] req@ffff9856140d9680 x1874669228472576/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830567 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830542.070648:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d9680 x1874669228472576/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830567 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830542.070657:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830542.070660:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830542.070663:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228472576:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787830542.070669:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228472704:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830542.070685:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830542.070731:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830542/real 1787830542] req@ffff9856140d83c0 x1874669228472704/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830567 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830542.070743:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d83c0 x1874669228472704/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830567 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830542.070750:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830542.070754:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830542.070757:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228472704:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830542.133006:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830543.028866:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830543.157059:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830544.052981:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830544.180962:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830545.076823:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830545.204887:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830546.100856:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830546.228929:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830547.124823:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787830547.189126:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830547.189136:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830547.189140:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830547.189151:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6692 00000100:00080000:2.0:1787830547.189161:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830547.189167:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787830547.189200:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db480 x1874669228472832/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830547.189233:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830547.189237:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830547.189239:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830547.189242:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6692 00000100:00080000:2.0:1787830547.189246:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830547.189250:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830547.189258:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830547.189270:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8b40 x1874669228472960/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830547.189279:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830547.189281:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830547.189283:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830547.189287:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6692 00000100:00080000:2.0:1787830547.189290:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830547.189294:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830547.189298:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830547.189307:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d9a40 x1874669228473088/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787830547.189404:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228472832:192.168.203.114@tcp:250:kworker.0 00000800:00000100:1.0:1787830547.189460:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830547.189493:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830547/real 1787830547] req@ffff9856140db480 x1874669228472832/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830572 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830547.189509:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db480 x1874669228472832/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830572 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830547.189519:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830547.189523:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830547.189526:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228472832:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787830547.189533:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228472960:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830547.189552:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830547.189563:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830547/real 1787830547] req@ffff9856140d8b40 x1874669228472960/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830572 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830547.189574:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8b40 x1874669228472960/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830572 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830547.189582:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830547.189585:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830547.189589:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228472960:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787830547.189594:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228473088:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830547.189610:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830547.189619:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830547/real 1787830547] req@ffff9856140d9a40 x1874669228473088/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830572 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830547.189637:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d9a40 x1874669228473088/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830572 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830547.189660:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830547.189664:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830547.189667:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228473088:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830547.254985:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830548.148976:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830548.277070:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830549.172884:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830549.301283:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830550.197540:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787830550.325359:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830551.221086:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830551.349011:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830552.244861:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787830552.308932:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830552.308943:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830552.308949:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830552.308961:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6697 00000100:00080000:2.0:1787830552.308975:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830552.308984:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787830552.309026:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140da940 x1874669228473216/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787830552.309125:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228473216:192.168.203.114@tcp:250:kworker.0 00000800:00000100:1.0:1787830552.309209:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00080000:2.0:1787830552.309242:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830552.309246:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830552.309249:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830552.309255:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6697 00000100:00080000:2.0:1787830552.309259:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830552.309264:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830552.309275:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830552.309305:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db0c0 x1874669228473344/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787830552.309676:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830552/real 1787830552] req@ffff9856140da940 x1874669228473216/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830577 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830552.310442:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140da940 x1874669228473216/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830577 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830552.310456:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830552.310461:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830552.310465:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228473216:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787830552.310511:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228473344:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830552.310565:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830552.310584:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830552/real 1787830552] req@ffff9856140db0c0 x1874669228473344/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830577 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830552.310600:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db0c0 x1874669228473344/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830577 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830552.310611:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830552.310614:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830552.310618:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228473344:192.168.203.114@tcp:38:kworker.0 00000100:00080000:2.0:1787830552.311088:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830552.311094:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830552.311097:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830552.311103:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6697 00000100:00080000:2.0:1787830552.311110:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830552.311117:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830552.311129:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830552.311153:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140dad00 x1874669228473472/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787830552.311229:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228473472:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830552.311265:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830552.311282:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830552/real 1787830552] req@ffff9856140dad00 x1874669228473472/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830577 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830552.311426:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140dad00 x1874669228473472/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830577 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830552.311434:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830552.311437:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830552.311441:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228473472:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830552.373451:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830553.268846:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830553.398113:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830554.292916:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830554.421639:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830555.316809:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787830555.445117:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830556.340811:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830556.468914:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830557.364905:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787830557.428916:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830557.428929:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830557.428935:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830557.428948:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6703 00000100:00080000:2.0:1787830557.428961:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830557.429032:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787830557.429079:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8f00 x1874669228473600/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830557.429116:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830557.429119:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830557.429121:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830557.429127:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6703 00000100:00080000:2.0:1787830557.429131:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830557.429135:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830557.429147:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830557.429159:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140da1c0 x1874669228473728/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830557.429169:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830557.429172:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830557.429174:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830557.429178:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6703 00000100:00080000:2.0:1787830557.429181:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830557.429185:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830557.429189:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830557.429201:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140dbc00 x1874669228473856/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787830557.429651:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228473600:192.168.203.114@tcp:250:kworker.0 00000800:00000100:1.0:1787830557.429764:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830557.429807:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830557/real 1787830557] req@ffff9856140d8f00 x1874669228473600/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830582 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830557.429828:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8f00 x1874669228473600/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830582 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830557.429840:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830557.429845:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830557.429849:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228473600:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787830557.429857:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228473728:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830557.429880:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830557.429892:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830557/real 1787830557] req@ffff9856140da1c0 x1874669228473728/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830582 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830557.429904:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140da1c0 x1874669228473728/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830582 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830557.429912:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830557.429915:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830557.429918:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228473728:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787830557.429924:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228473856:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830557.429944:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830557.429956:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830557/real 1787830557] req@ffff9856140dbc00 x1874669228473856/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830582 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830557.431076:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140dbc00 x1874669228473856/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830582 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830557.431103:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830557.431110:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830557.431115:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228473856:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830557.494430:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830558.388849:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830558.517373:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830559.412946:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830559.541000:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830560.436817:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787830560.564928:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830561.460990:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830561.588889:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830562.484834:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787830562.549721:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830562.549733:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830562.549738:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830562.549751:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6708 00000100:00080000:0.0:1787830562.549764:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830562.549772:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787830562.549820:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4b40 x1874669228473984/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830562.549856:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830562.549860:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830562.549862:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830562.549865:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6708 00000100:00080000:0.0:1787830562.549869:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830562.549873:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787830562.549884:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830562.549897:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6940 x1874669228474112/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830562.549906:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830562.549910:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830562.549912:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830562.549915:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6708 00000100:00080000:0.0:1787830562.549918:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830562.549922:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787830562.549926:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830562.549935:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4780 x1874669228474240/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787830562.550132:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228473984:192.168.203.114@tcp:250:kworker.0 00000800:00000100:1.0:1787830562.550213:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830562.550264:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830562/real 1787830562] req@ffff985607df4b40 x1874669228473984/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830587 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830562.550283:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4b40 x1874669228473984/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830587 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830562.550293:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830562.550298:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830562.550302:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228473984:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787830562.550309:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228474112:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830562.550332:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830562.550345:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830562/real 1787830562] req@ffff985607df6940 x1874669228474112/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830587 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830562.550361:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6940 x1874669228474112/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830587 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830562.550368:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830562.550371:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830562.550375:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228474112:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787830562.550379:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228474240:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830562.550397:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830562.550407:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830562/real 1787830562] req@ffff985607df4780 x1874669228474240/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830587 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830562.550418:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4780 x1874669228474240/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830587 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830562.550425:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830562.550428:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830562.550431:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228474240:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830562.612894:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830563.508801:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830563.638588:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830564.532854:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830564.660959:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830565.556828:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787830565.685958:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830566.582417:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830566.709488:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830567.604840:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787830567.668798:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830567.668819:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830567.668825:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830567.668836:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6713 00000100:00080000:0.0:1787830567.668849:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830567.668856:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787830567.668905:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7c00 x1874669228474368/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830567.668934:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830567.668937:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830567.668939:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830567.668942:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6713 00000100:00080000:0.0:1787830567.668944:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830567.668946:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787830567.668956:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830567.668968:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4000 x1874669228474496/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830567.668977:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830567.668981:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830567.668982:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830567.668986:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6713 00000100:00080000:0.0:1787830567.668990:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830567.668994:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787830567.668998:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830567.669008:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4780 x1874669228474624/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787830567.669172:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228474368:192.168.203.114@tcp:250:kworker.0 00000800:00000100:1.0:1787830567.669253:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830567.669283:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830567/real 1787830567] req@ffff985607df7c00 x1874669228474368/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830592 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830567.669300:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7c00 x1874669228474368/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830592 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830567.669311:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830567.669315:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830567.669320:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228474368:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787830567.669327:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228474496:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830567.669349:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830567.669362:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830567/real 1787830567] req@ffff985607df4000 x1874669228474496/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830592 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830567.669372:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4000 x1874669228474496/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830592 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830567.669444:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830567.669447:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830567.669450:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228474496:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787830567.669454:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228474624:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830567.669486:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830567.669500:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830567/real 1787830567] req@ffff985607df4780 x1874669228474624/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830592 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830567.669512:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4780 x1874669228474624/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830592 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830567.669525:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830567.669529:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830567.669532:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228474624:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830567.733037:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830568.628901:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830568.757720:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830569.652772:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830569.781108:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830570.677529:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787830570.805111:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830571.701387:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830571.829115:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830572.724820:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787830572.789171:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830572.789181:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830572.789185:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830572.789194:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6718 00000100:00080000:2.0:1787830572.789205:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830572.789212:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787830572.789245:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db840 x1874669228474752/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830572.789271:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830572.789275:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830572.789277:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830572.789281:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6718 00000100:00080000:2.0:1787830572.789284:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830572.789288:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830572.789296:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830572.789309:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140dbc00 x1874669228474880/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830572.789320:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830572.789324:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830572.789326:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830572.789330:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6718 00000100:00080000:2.0:1787830572.789334:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830572.789337:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830572.789342:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830572.789354:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140da1c0 x1874669228475008/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787830572.789369:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228474752:192.168.203.114@tcp:250:kworker.0 00000800:00000100:1.0:1787830572.789459:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830572.789504:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830572/real 1787830572] req@ffff9856140db840 x1874669228474752/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830597 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830572.789522:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db840 x1874669228474752/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830597 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830572.789533:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830572.789537:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830572.789541:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228474752:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787830572.789548:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228474880:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830572.789570:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830572.789581:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830572/real 1787830572] req@ffff9856140dbc00 x1874669228474880/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830597 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830572.789592:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140dbc00 x1874669228474880/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830597 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830572.789599:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830572.789603:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830572.789606:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228474880:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787830572.789635:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228475008:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830572.789656:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830572.789667:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830572/real 1787830572] req@ffff9856140da1c0 x1874669228475008/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830597 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830572.789678:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140da1c0 x1874669228475008/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830597 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830572.789685:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830572.789689:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830572.789724:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228475008:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830572.852837:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830573.748806:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830573.877031:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830574.772814:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830574.901207:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830575.797441:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787830575.924988:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830576.821082:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830576.948952:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830577.845773:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787830577.909201:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830577.909210:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787830577.909214:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830577.909224:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6723 00000100:00080000:3.0:1787830577.909234:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830577.909241:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787830577.909274:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83840 x1874669228475136/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830577.909302:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830577.909305:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787830577.909307:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830577.909311:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6723 00000100:00080000:3.0:1787830577.909314:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830577.909318:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830577.909327:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830577.909339:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f803c0 x1874669228475264/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830577.909349:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830577.909352:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787830577.909354:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830577.909360:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6723 00000100:00080000:3.0:1787830577.909363:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830577.909367:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830577.909372:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830577.909381:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82d00 x1874669228475392/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787830577.909417:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228475136:192.168.203.114@tcp:250:kworker.0 00000800:00000100:1.0:1787830577.909482:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830577.909525:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830577/real 1787830577] req@ffff985610f83840 x1874669228475136/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830602 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830577.909543:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83840 x1874669228475136/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830602 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830577.909553:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830577.909556:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830577.909561:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228475136:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787830577.909568:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228475264:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830577.909823:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830577.909838:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830577/real 1787830577] req@ffff985610f803c0 x1874669228475264/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830602 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830577.909851:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f803c0 x1874669228475264/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830602 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830577.909859:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830577.909874:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830577.909877:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228475264:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787830577.909901:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228475392:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830577.909921:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830577.909932:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830577/real 1787830577] req@ffff985610f82d00 x1874669228475392/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830602 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830577.909943:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82d00 x1874669228475392/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830602 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830577.909951:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830577.909954:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830577.909957:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228475392:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830577.972928:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830578.868856:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830578.997984:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830579.893023:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830580.020962:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830580.916868:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787830581.044910:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830581.940885:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830582.068994:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830582.964921:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787830583.028839:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830583.028849:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830583.028852:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830583.028862:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6728 00000100:00080000:2.0:1787830583.028872:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830583.028879:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787830583.028913:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8f00 x1874669228475520/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830583.028939:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830583.028943:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830583.028945:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830583.028949:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6728 00000100:00080000:2.0:1787830583.028952:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830583.028957:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830583.028965:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830583.028973:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228475520:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787830583.028979:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140dad00 x1874669228475648/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830583.028992:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830583.028996:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830583.028998:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830583.029002:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6728 00000100:00080000:2.0:1787830583.029006:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830583.029009:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830583.029014:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830583.029024:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db0c0 x1874669228475776/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787830583.029038:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830583.029088:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830583/real 1787830583] req@ffff9856140d8f00 x1874669228475520/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830608 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830583.029105:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8f00 x1874669228475520/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830608 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830583.029113:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830583.029117:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830583.029121:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228475520:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787830583.029143:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228475648:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830583.029166:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830583.029188:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830583/real 1787830583] req@ffff9856140dad00 x1874669228475648/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830608 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830583.029198:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140dad00 x1874669228475648/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830608 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830583.029205:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830583.029208:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830583.029211:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228475648:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787830583.029215:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228475776:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830583.029325:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830583.029337:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830583/real 1787830583] req@ffff9856140db0c0 x1874669228475776/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830608 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830583.029353:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db0c0 x1874669228475776/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830608 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830583.029360:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830583.029363:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830583.029366:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228475776:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830583.093103:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830583.989439:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830584.117213:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830585.012846:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830585.141403:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830586.036809:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:3.0:1787830586.165018:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830587.061065:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830587.188946:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830588.084821:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787830588.149211:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830588.149219:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830588.149222:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830588.149231:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6733 00000100:00080000:2.0:1787830588.149241:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830588.149248:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787830588.149276:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140da940 x1874669228475904/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830588.149298:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830588.149303:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830588.149304:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830588.149307:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6733 00000100:00080000:2.0:1787830588.149310:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830588.149314:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830588.149322:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830588.149334:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d9a40 x1874669228476032/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830588.149342:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830588.149346:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830588.149347:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830588.149351:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6733 00000100:00080000:2.0:1787830588.149354:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830588.149358:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830588.149362:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830588.149372:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8b40 x1874669228476160/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787830588.149475:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228475904:192.168.203.114@tcp:250:kworker.0 00000800:00000100:1.0:1787830588.149542:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830588.149575:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830588/real 1787830588] req@ffff9856140da940 x1874669228475904/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830613 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830588.149593:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140da940 x1874669228475904/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830613 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830588.149604:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830588.149609:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830588.149613:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228475904:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787830588.149621:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228476032:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830588.149646:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830588.149660:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830588/real 1787830588] req@ffff9856140d9a40 x1874669228476032/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830613 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830588.149679:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d9a40 x1874669228476032/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830613 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830588.149688:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830588.149723:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830588.149726:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228476032:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787830588.149731:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228476160:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830588.149748:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830588.149758:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830588/real 1787830588] req@ffff9856140d8b40 x1874669228476160/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830613 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830588.149769:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8b40 x1874669228476160/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830613 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830588.149777:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830588.149780:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830588.149783:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228476160:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830588.213764:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830589.108802:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830589.236862:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830590.132749:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830590.260921:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830591.156874:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830591.285002:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830592.181414:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830592.309401:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830593.204806:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787830593.268839:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830593.268847:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787830593.268851:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830593.268859:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6738 00000100:00080000:3.0:1787830593.268870:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830593.268877:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787830593.268910:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82940 x1874669228476288/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830593.268940:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830593.268944:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787830593.268945:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830593.268949:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6738 00000100:00080000:3.0:1787830593.268952:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830593.268955:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830593.268964:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830593.268975:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82580 x1874669228476416/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830593.268983:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830593.268986:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787830593.268987:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830593.268990:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6738 00000100:00080000:3.0:1787830593.268994:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830593.269019:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830593.269024:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830593.269033:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80f00 x1874669228476544/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787830593.269058:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228476288:192.168.203.114@tcp:250:kworker.0 00000800:00000100:1.0:1787830593.269111:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830593.269145:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830593/real 1787830593] req@ffff985610f82940 x1874669228476288/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830618 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830593.269160:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82940 x1874669228476288/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830618 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830593.269169:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830593.269173:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830593.269177:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228476288:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787830593.269184:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228476416:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830593.269206:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830593.269224:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830593/real 1787830593] req@ffff985610f82580 x1874669228476416/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830618 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830593.269232:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82580 x1874669228476416/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830618 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830593.269237:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830593.269239:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830593.269244:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228476416:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787830593.269248:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228476544:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830593.269260:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830593.269266:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830593/real 1787830593] req@ffff985610f80f00 x1874669228476544/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830618 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830593.269276:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80f00 x1874669228476544/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830618 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830593.269281:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830593.269283:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830593.269284:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228476544:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830593.332959:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830594.229118:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830594.357191:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830595.252863:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830595.383806:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830596.276848:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830596.405346:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830597.300867:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830597.429410:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830598.325153:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787830598.388883:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830598.388897:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787830598.388902:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830598.388916:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6744 00000100:00080000:3.0:1787830598.388930:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830598.388939:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787830598.388997:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83480 x1874669228476672/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830598.389050:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830598.389054:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787830598.389056:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830598.389069:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6744 00000100:00080000:3.0:1787830598.389074:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830598.389079:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830598.389090:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830598.389104:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f821c0 x1874669228476800/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830598.389119:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830598.389123:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787830598.389124:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830598.389128:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6744 00000100:00080000:3.0:1787830598.389132:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830598.389136:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830598.389141:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830598.389154:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80f00 x1874669228476928/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787830598.389273:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228476672:192.168.203.114@tcp:250:kworker.0 00000800:00000100:1.0:1787830598.389341:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830598.389373:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830598/real 1787830598] req@ffff985610f83480 x1874669228476672/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830623 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830598.389387:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83480 x1874669228476672/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830623 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830598.389398:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830598.389401:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830598.389405:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228476672:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787830598.389412:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f821c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228476800:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830598.389436:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830598.389448:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830598/real 1787830598] req@ffff985610f821c0 x1874669228476800/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830623 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830598.389461:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f821c0 x1874669228476800/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830623 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830598.389469:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830598.389472:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830598.389475:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f821c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228476800:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787830598.389481:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228476928:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830598.389501:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830598.389513:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830598/real 1787830598] req@ffff985610f80f00 x1874669228476928/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830623 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830598.389524:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80f00 x1874669228476928/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830623 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830598.389532:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830598.389535:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830598.389539:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228476928:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830598.453777:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830599.349118:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830599.478079:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830600.372850:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830600.502295:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830601.396809:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830601.524906:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830602.421230:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830602.549756:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830603.444817:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787830603.509588:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830603.509598:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830603.509602:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830603.509612:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6749 00000100:00080000:2.0:1787830603.509627:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830603.509635:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787830603.509671:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db480 x1874669228477056/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830603.509751:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830603.509756:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830603.509757:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830603.509762:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6749 00000100:00080000:2.0:1787830603.509765:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830603.509769:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830603.509778:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830603.509790:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d83c0 x1874669228477184/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830603.509799:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830603.509802:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830603.509804:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830603.509807:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6749 00000100:00080000:2.0:1787830603.509810:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830603.509813:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830603.509817:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830603.509825:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d9680 x1874669228477312/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787830603.510001:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228477056:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787830603.510137:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228477184:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787830603.510170:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228477312:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830603.573249:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830604.469014:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830604.596953:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830605.493113:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830605.623150:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830606.516812:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000400:00000100:0.0:1787830606.581004:0:2370:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.114:988 00000400:00000100:0.0:1787830606.581049:0:2370:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.114@tcp at host 192.168.203.114 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:0.0:1787830606.583434:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000100:00100000:1.0:1787830606.585687:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830603/real 1787830606] req@ffff9856140db480 x1874669228477056/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830628 ref 2 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830606.585733:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db480 x1874669228477056/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830628 ref 2 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830606.585844:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830606.585850:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830606.585855:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228477056:192.168.203.114@tcp:250:kworker.0 00000800:00000100:0.0:1787830606.586291:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000100:00100000:1.0:1787830606.586435:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830603/real 1787830606] req@ffff9856140d83c0 x1874669228477184/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830628 ref 2 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830606.586452:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d83c0 x1874669228477184/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830628 ref 2 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830606.586462:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830606.586465:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830606.586469:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228477184:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830606.586581:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000100:00100000:1.0:1787830606.586648:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830603/real 1787830606] req@ffff9856140d9680 x1874669228477312/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830628 ref 2 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830606.586662:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d9680 x1874669228477312/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830628 ref 2 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830606.586669:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830606.586672:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830606.586675:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228477312:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787830606.645453:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830607.540816:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830607.670458:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830608.564844:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787830608.628986:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830608.628996:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830608.629000:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830608.629010:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6754 00000100:00080000:0.0:1787830608.629020:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830608.629027:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787830608.629088:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6940 x1874669228477440/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830608.629120:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830608.629130:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830608.629131:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830608.629135:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6754 00000100:00080000:0.0:1787830608.629139:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830608.629143:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787830608.629151:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830608.629162:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4b40 x1874669228477568/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830608.629172:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830608.629174:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830608.629176:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830608.629179:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6754 00000100:00080000:0.0:1787830608.629182:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830608.629185:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787830608.629189:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830608.629198:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7480 x1874669228477696/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787830608.629285:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228477440:192.168.203.114@tcp:250:kworker.0 00000800:00000100:1.0:1787830608.629344:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830608.629374:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830608/real 1787830608] req@ffff985607df6940 x1874669228477440/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830633 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830608.629398:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6940 x1874669228477440/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830633 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830608.629407:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830608.629410:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830608.629413:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228477440:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787830608.629420:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228477568:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830608.629457:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830608.629481:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830608/real 1787830608] req@ffff985607df4b40 x1874669228477568/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830633 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830608.629494:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4b40 x1874669228477568/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830633 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830608.629502:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830608.629504:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830608.629507:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228477568:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787830608.629512:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228477696:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830608.629531:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830608.629542:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830608/real 1787830608] req@ffff985607df7480 x1874669228477696/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830633 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830608.629559:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7480 x1874669228477696/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830633 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830608.629566:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830608.629568:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830608.629571:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228477696:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830608.693024:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830609.589390:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830609.719073:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830610.612788:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830610.740929:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830611.636993:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830611.764917:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830612.660862:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830612.789440:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830613.684852:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787830613.748942:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830613.748952:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830613.748956:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830613.748966:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6759 00000100:00080000:0.0:1787830613.748975:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830613.748982:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787830613.749019:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df52c0 x1874669228477824/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830613.749039:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830613.749043:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830613.749044:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830613.749048:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6759 00000100:00080000:0.0:1787830613.749052:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830613.749055:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787830613.749064:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830613.749080:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7840 x1874669228477952/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830613.749089:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830613.749092:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830613.749094:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830613.749097:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6759 00000100:00080000:0.0:1787830613.749101:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830613.749105:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787830613.749109:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830613.749119:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4f00 x1874669228478080/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787830613.749380:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228477824:192.168.203.114@tcp:250:kworker.0 00000800:00000100:1.0:1787830613.749439:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830613.749481:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830613/real 1787830613] req@ffff985607df52c0 x1874669228477824/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830638 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830613.749497:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df52c0 x1874669228477824/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830638 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830613.749507:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830613.749511:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830613.749514:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228477824:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787830613.749521:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228477952:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830613.749543:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830613.749555:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830613/real 1787830613] req@ffff985607df7840 x1874669228477952/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830638 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830613.749565:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7840 x1874669228477952/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830638 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830613.749573:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830613.749576:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830613.749578:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228477952:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787830613.749583:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228478080:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830613.749598:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830613.749606:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830613/real 1787830613] req@ffff985607df4f00 x1874669228478080/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830638 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830613.749615:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4f00 x1874669228478080/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830638 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830613.749621:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830613.749624:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830613.749626:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228478080:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830613.812840:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830614.708908:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830614.836990:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830615.732795:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830615.861110:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830616.757364:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830616.885205:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830617.782631:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830617.909068:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830618.804877:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787830618.868899:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830618.868913:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830618.868918:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830618.868931:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6764 00000100:00080000:1.0:1787830618.868946:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830618.868955:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787830618.869006:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2d00 x1874669228478208/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830618.869053:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830618.869057:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830618.869059:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830618.869063:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6764 00000100:00080000:1.0:1787830618.869067:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830618.869072:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830618.869084:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830618.869097:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0b40 x1874669228478336/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830618.869110:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830618.869116:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:0.0:1787830618.869118:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228478208:192.168.203.114@tcp:250:kworker.0 00000100:00080000:1.0:1787830618.869119:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830618.869134:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6764 00000100:00080000:1.0:1787830618.869140:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830618.869145:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830618.869151:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830618.869165:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0000 x1874669228478464/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787830618.869367:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830618.869401:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830618/real 1787830618] req@ffff9856085a2d00 x1874669228478208/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830643 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830618.869417:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2d00 x1874669228478208/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830643 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830618.869428:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830618.869432:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830618.869435:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228478208:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787830618.869460:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228478336:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830618.869481:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830618.869492:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830618/real 1787830618] req@ffff9856085a0b40 x1874669228478336/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830643 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830618.869518:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0b40 x1874669228478336/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830643 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830618.869526:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830618.869529:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830618.869532:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228478336:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787830618.869539:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228478464:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830618.869558:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830618.869568:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830618/real 1787830618] req@ffff9856085a0000 x1874669228478464/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830643 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830618.869579:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0000 x1874669228478464/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830643 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830618.869587:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830618.869590:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830618.869593:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228478464:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830618.932855:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830619.828881:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830619.957372:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830620.853187:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830620.980926:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830621.876854:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830622.004845:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830622.900822:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830623.029196:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830623.924832:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787830623.989111:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830623.989121:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830623.989126:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830623.989137:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6769 00000100:00080000:0.0:1787830623.989148:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830623.989155:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787830623.989197:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6d00 x1874669228478592/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787830623.989540:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228478592:192.168.203.114@tcp:250:kworker.0 00000800:00000100:1.0:1787830623.989628:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00080000:0.0:1787830623.989671:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830623.989675:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830623.989677:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830623.989682:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6769 00000100:00080000:0.0:1787830623.989687:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830623.989738:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787830623.989751:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830623.989769:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5e00 x1874669228478720/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830623.989781:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830623.989785:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830623.989800:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830623.989805:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6769 00000100:00080000:0.0:1787830623.989810:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830623.989815:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787830623.989821:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830623.989838:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830623/real 1787830623] req@ffff985607df6d00 x1874669228478592/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830648 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830623.989841:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df61c0 x1874669228478848/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787830623.989920:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6d00 x1874669228478592/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830648 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830623.989934:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830623.989940:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830623.989945:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228478592:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787830623.989981:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228478720:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830623.990038:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830623.990055:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830623/real 1787830623] req@ffff985607df5e00 x1874669228478720/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830648 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830623.990067:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5e00 x1874669228478720/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830648 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830623.990075:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830623.990079:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830623.990082:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228478720:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787830623.990089:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228478848:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830623.990109:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830623.990120:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830623/real 1787830623] req@ffff985607df61c0 x1874669228478848/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830648 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830623.990132:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df61c0 x1874669228478848/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830648 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830623.990144:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830623.990147:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830623.990150:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228478848:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830624.052918:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830624.948920:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830625.076894:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830625.972847:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830626.101764:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830626.997849:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830627.124972:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830628.020844:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830628.149175:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830629.045202:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787830629.108830:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830629.108840:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830629.108844:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830629.108853:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6774 00000100:00080000:0.0:1787830629.108863:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830629.108870:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787830629.108918:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df43c0 x1874669228478976/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830629.108948:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830629.108952:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830629.108954:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830629.108957:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6774 00000100:00080000:0.0:1787830629.108961:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830629.108964:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787830629.108973:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830629.108986:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5680 x1874669228479104/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830629.108995:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830629.108998:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830629.109000:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830629.109004:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6774 00000100:00080000:0.0:1787830629.109007:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830629.109010:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787830629.109015:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830629.109024:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df70c0 x1874669228479232/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787830629.109224:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228478976:192.168.203.114@tcp:250:kworker.0 00000800:00000100:1.0:1787830629.109508:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830629.109543:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830629/real 1787830629] req@ffff985607df43c0 x1874669228478976/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830654 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830629.109559:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df43c0 x1874669228478976/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830654 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830629.109568:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830629.109571:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830629.109575:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228478976:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787830629.109581:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228479104:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830629.109604:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830629.109616:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830629/real 1787830629] req@ffff985607df5680 x1874669228479104/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830654 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830629.109628:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5680 x1874669228479104/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830654 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830629.109635:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830629.109639:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830629.109642:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228479104:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787830629.109647:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228479232:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830629.109666:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830629.109678:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830629/real 1787830629] req@ffff985607df70c0 x1874669228479232/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830654 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830629.109688:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df70c0 x1874669228479232/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830654 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830629.109722:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830629.109726:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830629.109729:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228479232:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830629.172900:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830630.070819:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830630.197149:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830631.092889:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830631.221008:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830632.116819:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830632.245056:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830633.140863:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830633.272525:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830634.165366:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787830634.229163:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830634.229176:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830634.229230:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830634.229242:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6779 00000100:00080000:2.0:1787830634.229262:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830634.229270:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787830634.229315:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8780 x1874669228479360/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830634.229347:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830634.229351:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830634.229353:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830634.229356:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6779 00000100:00080000:2.0:1787830634.229360:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830634.229364:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830634.229374:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830634.229388:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d9680 x1874669228479488/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830634.229396:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830634.229400:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830634.229401:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830634.229405:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6779 00000100:00080000:2.0:1787830634.229408:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830634.229411:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830634.229415:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830634.229423:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d83c0 x1874669228479616/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787830634.229551:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228479360:192.168.203.114@tcp:250:kworker.0 00000800:00000100:1.0:1787830634.229625:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830634.229664:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830634/real 1787830634] req@ffff9856140d8780 x1874669228479360/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830659 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830634.229679:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8780 x1874669228479360/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830659 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830634.229712:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830634.229730:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830634.229735:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228479360:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787830634.229742:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228479488:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830634.229776:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830634.229788:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830634/real 1787830634] req@ffff9856140d9680 x1874669228479488/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830659 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830634.229798:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d9680 x1874669228479488/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830659 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830634.229805:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830634.229813:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830634.229825:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228479488:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787830634.229830:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228479616:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830634.229848:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830634.229858:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830634/real 1787830634] req@ffff9856140d83c0 x1874669228479616/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830659 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830634.229868:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d83c0 x1874669228479616/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830659 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830634.229874:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830634.229877:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830634.229880:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228479616:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830634.293304:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830635.188968:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830635.317229:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830636.212823:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830636.341049:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830637.236844:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830637.364991:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830638.262621:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830638.389134:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830639.284809:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787830639.348999:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830639.349013:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830639.349018:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830639.349030:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6784 00000100:00080000:1.0:1787830639.349043:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830639.349051:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787830639.349098:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a1680 x1874669228479744/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830639.349264:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830639.349268:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830639.349270:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830639.349275:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6784 00000100:00080000:1.0:1787830639.349279:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830639.349283:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830639.349295:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830639.349308:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0780 x1874669228479872/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830639.349317:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830639.349321:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830639.349323:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830639.349326:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6784 00000100:00080000:1.0:1787830639.349330:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830639.349334:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830639.349339:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830639.349349:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3c00 x1874669228480000/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830639.349463:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228479744:192.168.203.114@tcp:250:kworker.0 00000800:00000100:0.0:1787830639.349544:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830639.349585:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830639/real 1787830639] req@ffff9856085a1680 x1874669228479744/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830664 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830639.349602:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a1680 x1874669228479744/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830664 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830639.349613:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830639.349617:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830639.349621:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228479744:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787830639.349628:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228479872:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830639.349653:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830639.349666:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830639/real 1787830639] req@ffff9856085a0780 x1874669228479872/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830664 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830639.349678:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0780 x1874669228479872/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830664 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830639.349735:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830639.349739:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830639.349743:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228479872:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787830639.349749:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228480000:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830639.349769:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830639.349781:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830639/real 1787830639] req@ffff9856085a3c00 x1874669228480000/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830664 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830639.349792:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3c00 x1874669228480000/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830664 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830639.349800:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830639.349803:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830639.349806:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228480000:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830639.412904:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830640.308807:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830640.437953:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830641.332889:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830641.464384:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830642.356853:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830642.486526:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830643.380864:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830643.509519:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830644.404809:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787830644.468847:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830644.468860:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830644.468865:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830644.468878:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6790 00000100:00080000:1.0:1787830644.468891:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830644.468900:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787830644.468958:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a03c0 x1874669228480128/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830644.469058:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830644.469063:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830644.469064:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830644.469069:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6790 00000100:00080000:1.0:1787830644.469073:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830644.469077:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830644.469088:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830644.469102:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3840 x1874669228480256/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830644.469111:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830644.469114:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830644.469116:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830644.469120:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6790 00000100:00080000:1.0:1787830644.469123:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830644.469126:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830644.469130:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830644.469139:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2580 x1874669228480384/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830644.469323:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228480128:192.168.203.114@tcp:250:kworker.0 00000800:00000100:0.0:1787830644.469410:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830644.469452:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830644/real 1787830644] req@ffff9856085a03c0 x1874669228480128/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830669 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830644.469470:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a03c0 x1874669228480128/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830669 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830644.469480:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830644.469484:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830644.469489:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228480128:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787830644.469496:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228480256:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830644.469520:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830644.469533:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830644/real 1787830644] req@ffff9856085a3840 x1874669228480256/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830669 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830644.469545:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3840 x1874669228480256/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830669 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830644.469553:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830644.469556:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830644.469559:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228480256:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787830644.469564:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228480384:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830644.469665:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830644.469677:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830644/real 1787830644] req@ffff9856085a2580 x1874669228480384/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830669 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830644.469722:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2580 x1874669228480384/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830669 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830644.469731:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830644.469735:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830644.469738:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228480384:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830644.533246:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830645.428857:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830645.557579:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830646.452817:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830646.581114:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830647.476851:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830647.605520:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830648.500896:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830648.629042:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830649.524857:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787830649.589281:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830649.589291:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830649.589295:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830649.589304:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6795 00000100:00080000:2.0:1787830649.589314:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830649.589321:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787830649.589356:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db480 x1874669228480512/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830649.589390:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830649.589394:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830649.589396:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830649.589399:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6795 00000100:00080000:2.0:1787830649.589403:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830649.589407:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830649.589415:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830649.589426:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8b40 x1874669228480640/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830649.589435:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830649.589438:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830649.589440:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830649.589443:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6795 00000100:00080000:2.0:1787830649.589446:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830649.589450:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830649.589454:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830649.589461:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d9a40 x1874669228480768/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830649.589688:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228480512:192.168.203.114@tcp:250:kworker.0 00000800:00000100:0.0:1787830649.589772:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830649.589804:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830649/real 1787830649] req@ffff9856140db480 x1874669228480512/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830674 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830649.589820:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db480 x1874669228480512/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830674 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830649.589828:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830649.589831:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830649.589834:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228480512:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787830649.589841:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228480640:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830649.589862:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830649.589873:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830649/real 1787830649] req@ffff9856140d8b40 x1874669228480640/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830674 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830649.589885:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8b40 x1874669228480640/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830674 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830649.589892:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830649.589895:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830649.589898:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228480640:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787830649.589903:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228480768:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830649.589922:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830649.589932:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830649/real 1787830649] req@ffff9856140d9a40 x1874669228480768/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830674 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830649.589949:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d9a40 x1874669228480768/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830674 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830649.589957:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830649.589960:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830649.589963:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228480768:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830649.652952:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830650.548896:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830650.676958:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830651.572849:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830651.701944:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830652.597688:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830652.726725:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830653.620848:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830653.749018:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830654.646080:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787830654.708891:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830654.708901:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830654.708905:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830654.708915:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6800 00000100:00080000:0.0:1787830654.708926:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830654.708934:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787830654.708966:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5a40 x1874669228480896/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787830654.709379:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228480896:192.168.203.114@tcp:250:kworker.0 00000800:00000100:1.0:1787830654.709435:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00080000:0.0:1787830654.709671:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830654.709676:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830654.709678:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830654.709683:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6800 00000100:00080000:0.0:1787830654.709688:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830654.709732:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787830654.709741:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830654.709757:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6580 x1874669228481024/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830654.710245:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830654.710250:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830654.710252:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830654.710257:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6800 00000100:00080000:0.0:1787830654.710261:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830654.710265:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787830654.710270:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830654.710283:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df70c0 x1874669228481152/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787830654.710414:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830654/real 1787830654] req@ffff985607df5a40 x1874669228480896/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830679 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830654.710432:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5a40 x1874669228480896/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830679 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830654.710442:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830654.710447:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830654.710451:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228480896:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787830654.710474:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228481024:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830654.710502:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830654.710516:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830654/real 1787830654] req@ffff985607df6580 x1874669228481024/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830679 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830654.710527:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6580 x1874669228481024/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830679 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830654.710532:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830654.710535:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830654.710537:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228481024:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787830654.710541:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228481152:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830654.710555:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830654.710562:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830654/real 1787830654] req@ffff985607df70c0 x1874669228481152/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830679 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830654.710569:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df70c0 x1874669228481152/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830679 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830654.710586:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830654.710589:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830654.710592:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228481152:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830654.773043:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830655.668853:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830655.799495:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830656.692816:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830656.821090:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830657.717281:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830657.845021:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830658.740868:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830658.869019:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830659.765138:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787830659.828846:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830659.828859:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830659.828863:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830659.828875:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6805 00000100:00080000:0.0:1787830659.828890:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830659.828900:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787830659.828946:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5680 x1874669228481280/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830659.828981:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830659.828984:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830659.828986:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830659.828989:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6805 00000100:00080000:0.0:1787830659.828992:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830659.828997:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787830659.829008:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830659.829021:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df43c0 x1874669228481408/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830659.829030:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830659.829034:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830659.829035:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830659.829039:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6805 00000100:00080000:0.0:1787830659.829042:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830659.829046:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787830659.829054:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830659.829057:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228481280:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787830659.829068:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df61c0 x1874669228481536/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787830659.829133:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830659.829180:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830659/real 1787830659] req@ffff985607df5680 x1874669228481280/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830684 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830659.829201:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5680 x1874669228481280/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830684 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830659.829213:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830659.829217:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830659.829221:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228481280:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787830659.829230:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228481408:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830659.829256:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830659.829269:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830659/real 1787830659] req@ffff985607df43c0 x1874669228481408/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830684 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830659.829280:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df43c0 x1874669228481408/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830684 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830659.829288:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830659.829292:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830659.829295:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228481408:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787830659.829318:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228481536:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830659.829338:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830659.829347:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830659/real 1787830659] req@ffff985607df61c0 x1874669228481536/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830684 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830659.829358:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df61c0 x1874669228481536/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830684 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830659.829365:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830659.829368:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830659.829371:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228481536:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830659.892905:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830660.788851:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830660.920974:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830661.814339:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830661.941823:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830662.837090:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830662.965439:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830663.860855:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830663.989572:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830664.886343:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787830664.949155:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830664.949169:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830664.949174:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830664.949188:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6810 00000100:00080000:2.0:1787830664.949203:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830664.949213:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787830664.949267:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140da940 x1874669228481664/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830664.949307:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830664.949311:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830664.949313:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830664.949327:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6810 00000100:00080000:2.0:1787830664.949331:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830664.949336:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830664.949347:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830664.949363:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db0c0 x1874669228481792/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787830664.949376:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228481664:192.168.203.114@tcp:250:kworker.0 00000100:00080000:2.0:1787830664.949377:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830664.949386:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830664.949389:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830664.949393:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6810 00000100:00080000:2.0:1787830664.949398:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830664.949402:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830664.949407:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830664.949419:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140dad00 x1874669228481920/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787830664.949472:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830664.949545:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830664/real 1787830664] req@ffff9856140da940 x1874669228481664/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830689 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830664.949565:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140da940 x1874669228481664/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830689 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830664.949577:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830664.949582:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830664.949586:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228481664:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787830664.949614:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228481792:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830664.949637:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830664.949660:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830664/real 1787830664] req@ffff9856140db0c0 x1874669228481792/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830689 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830664.949673:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db0c0 x1874669228481792/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830689 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830664.949682:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830664.949685:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830664.949688:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228481792:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787830664.949727:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228481920:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830664.949765:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830664.949791:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830664/real 1787830664] req@ffff9856140dad00 x1874669228481920/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830689 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830664.949804:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140dad00 x1874669228481920/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830689 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830664.949817:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830664.949821:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830664.949824:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228481920:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830665.013023:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830665.909089:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830666.040356:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830666.934035:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830667.060994:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830667.956896:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830668.085398:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830668.980843:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830669.109122:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830670.005727:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787830670.071771:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830670.071786:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830670.071791:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830670.071804:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6815 00000100:00080000:1.0:1787830670.071818:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830670.071827:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787830670.071874:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a1a40 x1874669228482048/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830670.071988:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228482048:192.168.203.114@tcp:250:kworker.0 00000100:00080000:1.0:1787830670.072166:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830670.072171:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830670.072174:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830670.072178:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6815 00000100:00080000:1.0:1787830670.072183:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830670.072187:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830670.072198:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830670.072212:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2940 x1874669228482176/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830670.074072:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228482176:192.168.203.114@tcp:38:kworker.0 00000100:00080000:1.0:1787830670.077045:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830670.077063:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830670.077069:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830670.077086:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6815 00000100:00080000:1.0:1787830670.077103:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830670.077113:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830670.077132:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830670.077427:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a12c0 x1874669228482304/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830670.077500:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228482304:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830670.133680:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830671.028855:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830671.156932:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830672.053299:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830672.181027:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830673.076927:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000400:00000100:3.0:1787830673.141058:0:2371:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.114:988 00000400:00000100:3.0:1787830673.141111:0:2371:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.114@tcp at host 192.168.203.114 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:3.0:1787830673.141131:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:3.0:1787830673.141173:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:3.0:1787830673.141181:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000100:00100000:0.0:1787830673.141296:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830670/real 1787830673] req@ffff9856085a12c0 x1874669228482304/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830695 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830673.141330:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a12c0 x1874669228482304/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830695 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830673.141341:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830673.141345:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830673.141349:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228482304:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787830673.141357:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830670/real 1787830673] req@ffff9856085a2940 x1874669228482176/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830695 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830673.141369:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2940 x1874669228482176/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830695 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830673.141377:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830673.141381:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830673.141385:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228482176:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787830673.141391:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830670/real 1787830673] req@ffff9856085a1a40 x1874669228482048/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830695 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830673.141402:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a1a40 x1874669228482048/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830695 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830673.141409:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830673.141412:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830673.141414:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228482048:192.168.203.114@tcp:250:kworker.0 00010000:00010000:2.0:1787830673.204891:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830674.100831:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830674.229066:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830675.125089:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787830675.189808:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830675.189819:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830675.189823:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830675.189833:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6820 00000100:00080000:1.0:1787830675.189843:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830675.189850:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787830675.189886:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a21c0 x1874669228482432/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830675.189921:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830675.189925:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830675.189926:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830675.189930:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6820 00000100:00080000:1.0:1787830675.189934:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830675.189938:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830675.189947:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830675.189960:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3480 x1874669228482560/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830675.189969:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830675.189972:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830675.189974:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830675.189977:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6820 00000100:00080000:1.0:1787830675.189980:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830675.189984:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830675.189988:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830675.189998:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a30c0 x1874669228482688/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830675.190113:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228482432:192.168.203.114@tcp:250:kworker.0 00000800:00000100:0.0:1787830675.190178:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830675.190209:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830675/real 1787830675] req@ffff9856085a21c0 x1874669228482432/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830700 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830675.190225:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a21c0 x1874669228482432/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830700 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830675.190235:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830675.190239:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830675.190242:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228482432:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787830675.190249:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228482560:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830675.190270:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830675.190281:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830675/real 1787830675] req@ffff9856085a3480 x1874669228482560/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830700 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830675.190291:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3480 x1874669228482560/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830700 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830675.190299:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830675.190301:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830675.190304:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228482560:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787830675.190309:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228482688:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830675.190328:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830675.190338:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830675/real 1787830675] req@ffff9856085a30c0 x1874669228482688/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830700 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830675.190349:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a30c0 x1874669228482688/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830700 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830675.190361:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830675.190364:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830675.190367:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228482688:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830675.253854:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830676.149592:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830676.277088:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830677.172855:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830677.301905:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830678.198920:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830678.325549:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830679.220848:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830679.349048:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830680.244840:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787830680.308863:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830680.308872:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830680.308876:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830680.308885:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6825 00000100:00080000:0.0:1787830680.308895:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830680.308902:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787830680.308936:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5e00 x1874669228482816/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830680.308971:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830680.308975:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830680.308976:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830680.308980:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6825 00000100:00080000:0.0:1787830680.308983:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830680.308987:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787830680.308996:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830680.309007:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6d00 x1874669228482944/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830680.309016:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830680.309019:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830680.309021:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830680.309024:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6825 00000100:00080000:0.0:1787830680.309027:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830680.309030:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787830680.309034:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830680.309045:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4f00 x1874669228483072/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787830680.309143:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228482816:192.168.203.114@tcp:250:kworker.0 00000800:00000100:1.0:1787830680.309203:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830680.309237:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830680/real 1787830680] req@ffff985607df5e00 x1874669228482816/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830705 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830680.309253:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5e00 x1874669228482816/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830705 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830680.309262:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830680.309266:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830680.309270:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228482816:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787830680.309277:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228482944:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830680.309301:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830680.309313:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830680/real 1787830680] req@ffff985607df6d00 x1874669228482944/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830705 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830680.309324:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6d00 x1874669228482944/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830705 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830680.309330:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830680.309333:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830680.309336:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228482944:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787830680.309342:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228483072:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830680.309360:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830680.309370:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830680/real 1787830680] req@ffff985607df4f00 x1874669228483072/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830705 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830680.309380:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4f00 x1874669228483072/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830705 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830680.309387:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830680.309390:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830680.309392:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228483072:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830680.372916:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830681.268810:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830681.397465:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830682.292926:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830682.424320:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830683.316874:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830683.445651:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830684.340993:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830684.469173:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830685.364849:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787830685.428965:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830685.428981:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830685.428986:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830685.428999:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6831 00000100:00080000:2.0:1787830685.429015:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830685.429026:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787830685.429079:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8f00 x1874669228483200/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830685.429117:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830685.429121:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830685.429123:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830685.429127:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6831 00000100:00080000:2.0:1787830685.429131:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830685.429135:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830685.429147:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830685.429160:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140da1c0 x1874669228483328/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830685.429171:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830685.429174:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830685.429176:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830685.429180:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6831 00000100:00080000:2.0:1787830685.429183:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830685.429187:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830685.429193:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830685.429196:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228483200:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787830685.429205:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140dbc00 x1874669228483456/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787830685.429561:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830685.429607:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830685/real 1787830685] req@ffff9856140d8f00 x1874669228483200/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830710 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830685.429627:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8f00 x1874669228483200/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830710 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830685.429638:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830685.429643:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830685.429646:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228483200:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787830685.429654:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228483328:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830685.429782:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830685.429796:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830685/real 1787830685] req@ffff9856140da1c0 x1874669228483328/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830710 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830685.429872:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140da1c0 x1874669228483328/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830710 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830685.429881:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830685.429883:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830685.429887:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228483328:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787830685.429919:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228483456:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830685.429941:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830685.429953:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830685/real 1787830685] req@ffff9856140dbc00 x1874669228483456/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830710 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830685.429965:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140dbc00 x1874669228483456/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830710 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830685.429979:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830685.429982:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830685.429986:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228483456:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830685.492896:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830686.388826:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830686.517377:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830687.412780:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830687.542418:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830688.436937:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830688.565374:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830689.460832:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830689.589432:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830690.484853:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787830690.549414:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830690.549429:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830690.549435:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830690.549552:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6836 00000100:00080000:1.0:1787830690.549567:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830690.549577:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787830690.549637:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0f00 x1874669228483584/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830690.549676:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830690.549681:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830690.549682:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830690.549686:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6836 00000100:00100000:0.0:1787830690.549773:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228483584:192.168.203.114@tcp:250:kworker.0 00000100:00080000:1.0:1787830690.549775:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830690.549780:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830690.549791:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830690.549807:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a30c0 x1874669228483712/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830690.549817:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830690.549821:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830690.549822:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830690.549825:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6836 00000100:00080000:1.0:1787830690.549836:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830690.549840:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830690.549844:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000800:00000100:0.0:1787830690.549850:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830690.549853:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3480 x1874669228483840/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830690.549892:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830690/real 1787830690] req@ffff9856085a0f00 x1874669228483584/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830715 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830690.549911:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0f00 x1874669228483584/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830715 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830690.549922:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830690.549926:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830690.549930:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228483584:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787830690.549956:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228483712:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830690.549979:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830690.549992:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830690/real 1787830690] req@ffff9856085a30c0 x1874669228483712/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830715 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830690.550009:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a30c0 x1874669228483712/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830715 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830690.550016:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830690.550020:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830690.550023:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228483712:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787830690.550028:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228483840:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830690.550046:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830690.550056:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830690/real 1787830690] req@ffff9856085a3480 x1874669228483840/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830715 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830690.550066:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3480 x1874669228483840/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830715 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830690.550073:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830690.550075:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830690.550078:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228483840:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830690.612920:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830691.508853:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830691.637676:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830692.532875:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830692.661272:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830693.557414:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830693.685495:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830694.581253:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830694.709266:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830695.605169:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787830695.669375:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830695.669385:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830695.669391:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830695.669402:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6841 00000100:00080000:0.0:1787830695.669415:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830695.669423:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787830695.669467:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7840 x1874669228483968/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830695.669497:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830695.669501:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830695.669504:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830695.669508:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6841 00000100:00080000:0.0:1787830695.669512:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830695.669517:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787830695.669527:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830695.669542:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df52c0 x1874669228484096/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830695.669551:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830695.669555:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830695.669556:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830695.669561:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6841 00000100:00080000:0.0:1787830695.669566:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830695.669569:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787830695.669574:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830695.669584:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7480 x1874669228484224/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787830695.669729:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228483968:192.168.203.114@tcp:250:kworker.0 00000800:00000100:1.0:1787830695.670528:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830695.670565:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830695/real 1787830695] req@ffff985607df7840 x1874669228483968/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830720 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830695.670584:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7840 x1874669228483968/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830720 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830695.670596:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830695.670601:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830695.670605:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228483968:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787830695.670613:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228484096:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830695.670639:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830695.670653:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830695/real 1787830695] req@ffff985607df52c0 x1874669228484096/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830720 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830695.670673:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df52c0 x1874669228484096/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830720 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830695.670683:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830695.670686:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830695.670689:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228484096:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787830695.670726:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228484224:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830695.670751:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830695.670779:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830695/real 1787830695] req@ffff985607df7480 x1874669228484224/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830720 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830695.670795:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7480 x1874669228484224/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830720 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830695.670805:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830695.670809:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830695.670812:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228484224:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830695.732906:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830696.628884:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830696.756925:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830697.652987:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830697.781315:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830698.676791:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830698.805143:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830699.700809:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830699.830222:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830700.724814:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787830700.788873:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830700.788884:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787830700.788888:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830700.788898:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6846 00000100:00080000:3.0:1787830700.788909:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830700.788916:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787830700.788952:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82580 x1874669228484352/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830700.788991:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830700.788995:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787830700.788997:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830700.789001:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6846 00000100:00080000:3.0:1787830700.789004:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830700.789008:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830700.789017:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830700.789028:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82940 x1874669228484480/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830700.789037:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830700.789040:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787830700.789042:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830700.789045:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6846 00000100:00080000:3.0:1787830700.789048:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830700.789052:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830700.789055:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830700.789064:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82d00 x1874669228484608/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787830700.789151:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228484352:192.168.203.114@tcp:250:kworker.0 00000800:00000100:1.0:1787830700.789218:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830700.789248:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830700/real 1787830700] req@ffff985610f82580 x1874669228484352/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830725 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830700.789278:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82580 x1874669228484352/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830725 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830700.789289:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830700.789293:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830700.789296:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228484352:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787830700.789303:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228484480:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830700.789326:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830700.789339:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830700/real 1787830700] req@ffff985610f82940 x1874669228484480/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830725 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830700.789356:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82940 x1874669228484480/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830725 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830700.789363:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830700.789367:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830700.789370:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228484480:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787830700.789375:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228484608:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830700.789394:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830700.789418:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830700/real 1787830700] req@ffff985610f82d00 x1874669228484608/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830725 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830700.789428:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82d00 x1874669228484608/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830725 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830700.789436:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830700.789439:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830700.789442:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228484608:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830700.852861:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830701.748822:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830701.877073:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830702.772842:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830702.900866:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830703.796887:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830703.925008:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830704.820928:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830704.948984:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830705.844801:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787830705.909098:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830705.909105:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830705.909108:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830705.909116:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6851 00000100:00080000:1.0:1787830705.909123:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830705.909128:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787830705.909163:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a21c0 x1874669228484736/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830705.909192:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830705.909196:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830705.909201:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830705.909205:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6851 00000100:00080000:1.0:1787830705.909209:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830705.909213:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830705.909230:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830705.909243:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a1a40 x1874669228484864/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830705.909250:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228484736:192.168.203.114@tcp:250:kworker.0 00000100:00080000:1.0:1787830705.909256:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830705.909262:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830705.909264:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830705.909268:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6851 00000100:00080000:1.0:1787830705.909271:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830705.909275:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830705.909280:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000800:00000100:0.0:1787830705.909330:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830705.909345:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2940 x1874669228484992/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830705.909370:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830705/real 1787830705] req@ffff9856085a21c0 x1874669228484736/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830730 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830705.909389:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a21c0 x1874669228484736/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830730 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830705.909399:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830705.909403:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830705.909406:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228484736:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787830705.909429:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228484864:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830705.909450:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830705.909461:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830705/real 1787830705] req@ffff9856085a1a40 x1874669228484864/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830730 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830705.909484:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a1a40 x1874669228484864/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830730 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830705.909492:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830705.909495:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830705.909497:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228484864:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787830705.909501:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228484992:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830705.909523:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830705.909530:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830705/real 1787830705] req@ffff9856085a2940 x1874669228484992/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830730 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830705.909541:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2940 x1874669228484992/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830730 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830705.909546:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830705.909548:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830705.909550:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228484992:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830705.972855:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830706.868944:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830706.997339:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830707.892978:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830708.020867:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830708.916801:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830709.045817:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830709.940806:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830710.071857:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830710.965251:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787830711.029158:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830711.029168:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787830711.029171:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830711.029191:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6856 00000100:00080000:3.0:1787830711.029202:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830711.029210:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787830711.029244:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f803c0 x1874669228485120/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830711.029271:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830711.029274:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787830711.029276:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830711.029279:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6856 00000100:00080000:3.0:1787830711.029283:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830711.029287:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830711.029295:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830711.029309:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83840 x1874669228485248/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830711.029320:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1787830711.029324:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228485120:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787830711.029325:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787830711.029329:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830711.029335:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6856 00000100:00080000:3.0:1787830711.029338:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830711.029342:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830711.029347:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830711.029359:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81e00 x1874669228485376/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787830711.029388:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830711.029418:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830711/real 1787830711] req@ffff985610f803c0 x1874669228485120/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830736 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830711.029434:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f803c0 x1874669228485120/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830736 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830711.029444:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830711.029448:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830711.029452:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228485120:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787830711.029472:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228485248:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830711.029491:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830711.029501:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830711/real 1787830711] req@ffff985610f83840 x1874669228485248/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830736 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830711.029511:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83840 x1874669228485248/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830736 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830711.029524:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830711.029527:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830711.029530:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228485248:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787830711.029535:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228485376:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830711.029550:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830711.029558:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830711/real 1787830711] req@ffff985610f81e00 x1874669228485376/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830736 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830711.029567:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81e00 x1874669228485376/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830736 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830711.029573:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830711.029575:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830711.029578:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228485376:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830711.092852:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830711.988878:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830712.116929:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830713.014743:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830713.141225:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830714.036863:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830714.165142:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830715.061253:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830715.189815:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830716.091214:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787830716.149675:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830716.149685:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787830716.149706:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830716.149716:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6861 00000100:00080000:3.0:1787830716.149726:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830716.149733:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787830716.149768:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81680 x1874669228485504/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830716.149852:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228485504:192.168.203.114@tcp:250:kworker.0 00000800:00000100:0.0:1787830716.149911:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00080000:3.0:1787830716.149942:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830716.149946:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787830716.149948:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830716.149953:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6861 00000100:00080000:3.0:1787830716.149957:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830716.149961:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830716.149972:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830716.149984:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f812c0 x1874669228485632/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830716.151090:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830716.151096:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787830716.151098:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830716.151104:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6861 00000100:00080000:3.0:1787830716.151109:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830716.151113:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830716.151121:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830716.151136:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80b40 x1874669228485760/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787830716.152439:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830716/real 1787830716] req@ffff985610f81680 x1874669228485504/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830741 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830716.152463:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81680 x1874669228485504/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830741 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830716.152473:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830716.152477:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830716.152480:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228485504:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787830716.152507:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228485632:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830716.152555:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830716.152572:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830716/real 1787830716] req@ffff985610f812c0 x1874669228485632/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830741 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830716.152583:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f812c0 x1874669228485632/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830741 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830716.152591:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830716.152595:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830716.152599:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228485632:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787830716.152605:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228485760:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830716.152624:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830716.152636:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830716/real 1787830716] req@ffff985610f80b40 x1874669228485760/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830741 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830716.152647:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80b40 x1874669228485760/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830741 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830716.152654:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830716.152657:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830716.152660:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228485760:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830716.213322:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830717.108830:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830717.237122:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830718.132900:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830718.261986:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830719.156848:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830719.285084:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830720.180915:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830720.309087:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830721.205066:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787830721.268846:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830721.268859:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787830721.268864:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830721.268875:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6866 00000100:00080000:3.0:1787830721.268889:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830721.268897:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787830721.268940:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80000 x1874669228485888/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830721.268976:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830721.268980:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787830721.268982:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830721.268986:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6866 00000100:00080000:3.0:1787830721.268990:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830721.268994:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830721.269005:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830721.269018:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81a40 x1874669228486016/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787830721.269028:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228485888:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787830721.269031:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830721.269037:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787830721.269039:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830721.269044:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6866 00000100:00080000:3.0:1787830721.269047:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830721.269050:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830721.269054:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830721.269064:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f830c0 x1874669228486144/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787830721.269107:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830721.269160:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830721/real 1787830721] req@ffff985610f80000 x1874669228485888/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830746 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830721.269177:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80000 x1874669228485888/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830746 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830721.269193:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830721.269197:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830721.269201:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228485888:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787830721.269225:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228486016:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830721.269246:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830721.269257:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830721/real 1787830721] req@ffff985610f81a40 x1874669228486016/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830746 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830721.269268:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81a40 x1874669228486016/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830746 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830721.269276:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830721.269279:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830721.269282:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228486016:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787830721.269288:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228486144:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830721.269302:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830721.269310:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830721/real 1787830721] req@ffff985610f830c0 x1874669228486144/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830746 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830721.269321:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f830c0 x1874669228486144/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830746 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830721.269329:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830721.269332:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830721.269335:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228486144:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830721.333062:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830722.228828:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830722.357276:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830723.252822:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830723.381823:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830724.276850:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830724.405597:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830725.302319:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830725.428943:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830726.324862:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787830726.388948:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830726.388960:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787830726.388965:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830726.388977:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6872 00000100:00080000:3.0:1787830726.388990:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830726.388998:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787830726.389043:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83c00 x1874669228486272/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830726.389082:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830726.389086:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787830726.389088:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830726.389092:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6872 00000100:00080000:3.0:1787830726.389095:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830726.389099:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830726.389109:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830726.389120:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80780 x1874669228486400/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830726.389129:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830726.389132:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787830726.389133:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830726.389136:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6872 00000100:00080000:3.0:1787830726.389139:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830726.389141:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830726.389145:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830726.389154:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f830c0 x1874669228486528/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787830726.389526:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228486272:192.168.203.114@tcp:250:kworker.0 00000800:00000100:1.0:1787830726.389604:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830726.389654:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830726/real 1787830726] req@ffff985610f83c00 x1874669228486272/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830751 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830726.389673:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83c00 x1874669228486272/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830751 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830726.389684:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830726.389689:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830726.389720:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228486272:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787830726.389728:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228486400:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830726.389752:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830726.389765:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830726/real 1787830726] req@ffff985610f80780 x1874669228486400/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830751 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830726.389776:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80780 x1874669228486400/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830751 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830726.389784:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830726.389787:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830726.389790:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228486400:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787830726.389799:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228486528:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830726.389819:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830726.389830:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830726/real 1787830726] req@ffff985610f830c0 x1874669228486528/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830751 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830726.389842:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f830c0 x1874669228486528/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830751 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830726.389849:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830726.389853:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830726.389856:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228486528:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830726.452922:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830727.348825:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830727.479370:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830728.372973:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830728.501757:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830729.396905:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830729.525089:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830730.420892:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830730.550529:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830731.444876:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787830731.509190:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830731.509199:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830731.509203:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830731.509211:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6877 00000100:00080000:1.0:1787830731.509220:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830731.509226:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787830731.509254:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a12c0 x1874669228486656/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830731.509279:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830731.509282:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830731.509283:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830731.509286:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6877 00000100:00080000:1.0:1787830731.509296:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830731.509304:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830731.509313:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830731.509326:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2580 x1874669228486784/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830731.509335:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1787830731.509338:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228486656:192.168.203.114@tcp:250:kworker.0 00000100:00080000:1.0:1787830731.509340:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830731.509344:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830731.509350:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6877 00000100:00080000:1.0:1787830731.509354:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830731.509358:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830731.509362:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830731.509371:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3840 x1874669228486912/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787830731.509409:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830731.509444:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830731/real 1787830731] req@ffff9856085a12c0 x1874669228486656/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830756 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830731.509460:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a12c0 x1874669228486656/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830756 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830731.509471:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830731.509474:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830731.509478:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228486656:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787830731.509500:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228486784:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787830731.509520:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830731.509531:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830731/real 1787830731] req@ffff9856085a2580 x1874669228486784/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830756 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830731.509543:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2580 x1874669228486784/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830756 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830731.509551:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830731.509554:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830731.509558:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228486784:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787830731.509564:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228486912:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787830731.509581:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830731.509591:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830731/real 1787830731] req@ffff9856085a3840 x1874669228486912/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830756 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830731.509602:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3840 x1874669228486912/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830756 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830731.509609:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830731.509612:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830731.509615:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228486912:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830731.572845:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830732.469999:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830732.600407:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830733.492844:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830733.621230:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830734.517016:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830734.645219:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830735.542164:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830735.669022:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830736.564896:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787830736.628812:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830736.628821:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830736.628825:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830736.628834:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6882 00000100:00080000:1.0:1787830736.628845:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830736.628851:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787830736.628885:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a03c0 x1874669228487040/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830736.628918:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830736.628921:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830736.628923:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830736.628927:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6882 00000100:00080000:1.0:1787830736.628930:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830736.628934:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830736.628942:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830736.628953:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3c00 x1874669228487168/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830736.628962:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830736.628966:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830736.628967:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830736.628971:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6882 00000100:00080000:1.0:1787830736.628975:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830736.628979:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830736.628985:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830736.628996:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0780 x1874669228487296/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787830736.629040:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228487040:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787830736.629166:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228487168:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787830736.629210:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228487296:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830736.692942:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830737.588811:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830737.716973:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830738.612887:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830738.741094:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830739.636843:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000400:00000100:0.0:1787830739.701841:0:2372:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.114:988 00000400:00000100:0.0:1787830739.701891:0:2372:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.114@tcp at host 192.168.203.114 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:0.0:1787830739.701919:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:0.0:1787830739.701962:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000100:00100000:2.0:1787830739.702252:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830736/real 1787830739] req@ffff9856085a03c0 x1874669228487040/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830761 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000800:00000100:0.0:1787830739.702274:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000100:00100000:2.0:1787830739.702297:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a03c0 x1874669228487040/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830761 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830739.702313:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830739.702321:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830739.702328:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228487040:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787830739.702339:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830736/real 1787830739] req@ffff9856085a3c00 x1874669228487168/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830761 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830739.702355:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3c00 x1874669228487168/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830761 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830739.702363:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830739.702366:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830739.702369:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228487168:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787830739.702410:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830736/real 1787830739] req@ffff9856085a0780 x1874669228487296/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830761 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830739.702423:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0780 x1874669228487296/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830761 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830739.702431:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830739.702435:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830739.702438:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228487296:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787830739.764905:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830740.661061:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830740.791409:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830741.685210:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787830741.749222:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830741.749236:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830741.749245:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830741.749257:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6887 00000100:00080000:0.0:1787830741.749271:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830741.749280:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787830741.749326:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4b40 x1874669228487424/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787830741.749514:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228487424:192.168.203.114@tcp:250:kworker.0 00000800:00000100:2.0:1787830741.749594:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00080000:0.0:1787830741.749629:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830741.749634:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830741.749636:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830741.749640:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6887 00000100:00080000:0.0:1787830741.749645:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830741.749650:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787830741.749661:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830741.749676:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6940 x1874669228487552/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787830741.750132:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830741/real 1787830741] req@ffff985607df4b40 x1874669228487424/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830766 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830741.750154:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4b40 x1874669228487424/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830766 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830741.750166:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830741.750170:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830741.750174:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228487424:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787830741.750206:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228487552:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787830741.750239:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830741.750251:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830741/real 1787830741] req@ffff985607df6940 x1874669228487552/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830766 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830741.750268:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6940 x1874669228487552/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830766 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830741.750277:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830741.750280:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830741.750283:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228487552:192.168.203.114@tcp:38:kworker.0 00000100:00080000:0.0:1787830741.751250:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830741.751260:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830741.751263:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830741.751273:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6887 00000100:00080000:0.0:1787830741.751284:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830741.751291:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787830741.751305:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830741.751334:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4780 x1874669228487680/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787830741.752515:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228487680:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787830741.752596:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830741.752631:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830741/real 1787830741] req@ffff985607df4780 x1874669228487680/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830766 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830741.752664:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4780 x1874669228487680/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830766 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830741.752675:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830741.752679:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830741.752683:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228487680:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830741.812887:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830742.708865:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830742.836978:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830743.733060:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830743.861285:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830744.756814:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830744.885251:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830745.780872:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830745.909260:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830746.804840:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787830746.869290:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830746.869300:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830746.869305:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830746.869315:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6892 00000100:00080000:1.0:1787830746.869325:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830746.869333:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787830746.869366:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a1680 x1874669228487808/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787830746.869438:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228487808:192.168.203.114@tcp:250:kworker.0 00000800:00000100:2.0:1787830746.869499:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00080000:1.0:1787830746.869530:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830746.869535:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830746.869537:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830746.869540:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6892 00000100:00080000:1.0:1787830746.869544:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830746.869547:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830746.869556:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830746.869569:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0000 x1874669228487936/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787830746.869624:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830746/real 1787830746] req@ffff9856085a1680 x1874669228487808/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830771 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830746.869646:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a1680 x1874669228487808/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830771 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830746.869655:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830746.869659:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830746.869663:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228487808:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787830746.869686:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228487936:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787830746.869860:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830746.869873:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830746/real 1787830746] req@ffff9856085a0000 x1874669228487936/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830771 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830746.869885:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0000 x1874669228487936/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830771 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830746.869893:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830746.869896:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830746.869899:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228487936:192.168.203.114@tcp:38:kworker.0 00000100:00080000:1.0:1787830746.870010:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830746.870014:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830746.870017:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830746.870021:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6892 00000100:00080000:1.0:1787830746.870026:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830746.870116:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830746.870123:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830746.870135:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0b40 x1874669228488064/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787830746.870996:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228488064:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787830746.871034:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830746.871050:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830746/real 1787830746] req@ffff9856085a0b40 x1874669228488064/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830771 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830746.871065:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0b40 x1874669228488064/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830771 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830746.871090:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830746.871094:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830746.871098:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228488064:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830746.932839:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830747.828878:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830747.957857:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830748.852860:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830748.980899:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830749.878799:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830750.005479:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830750.902442:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830751.029510:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830751.924920:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787830751.990887:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830751.990907:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830751.990911:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830751.990920:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6897 00000100:00080000:2.0:1787830751.990929:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830751.990936:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787830751.990972:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db840 x1874669228488192/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830751.991012:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830751.991016:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830751.991018:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830751.991022:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6897 00000100:00080000:2.0:1787830751.991029:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830751.991033:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830751.991042:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830751.991059:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140dbc00 x1874669228488320/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830751.991075:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830751.991078:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830751.991080:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830751.991083:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6897 00000100:00080000:2.0:1787830751.991087:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830751.991090:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830751.991094:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830751.991104:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140da1c0 x1874669228488448/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787830751.991380:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228488192:192.168.203.114@tcp:250:kworker.0 00000800:00000100:1.0:1787830751.991435:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830751.991466:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830751/real 1787830751] req@ffff9856140db840 x1874669228488192/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830776 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830751.991483:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db840 x1874669228488192/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830776 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830751.991492:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830751.991495:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830751.991499:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228488192:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787830751.991506:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228488320:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830751.991526:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830751.991538:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830751/real 1787830751] req@ffff9856140dbc00 x1874669228488320/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830776 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830751.991548:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140dbc00 x1874669228488320/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830776 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830751.991555:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830751.991558:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830751.991561:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228488320:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787830751.991566:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228488448:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830751.991582:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830751.991593:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830751/real 1787830751] req@ffff9856140da1c0 x1874669228488448/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830776 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830751.991603:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140da1c0 x1874669228488448/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830776 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830751.991610:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830751.991613:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830751.991616:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228488448:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830752.052993:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830752.948776:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830753.078915:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830753.973481:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830754.101122:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830754.996967:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830755.124948:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830756.020968:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830756.150321:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830757.044983:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787830757.108811:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830757.108821:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830757.108826:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830757.108836:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6902 00000100:00080000:2.0:1787830757.108847:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830757.108854:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787830757.108907:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8f00 x1874669228488576/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830757.108940:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830757.108943:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830757.108945:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830757.108949:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6902 00000100:00080000:2.0:1787830757.108952:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830757.108956:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830757.108965:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830757.108976:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140dad00 x1874669228488704/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830757.108986:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830757.108989:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830757.108991:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830757.108995:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6902 00000100:00080000:2.0:1787830757.108998:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830757.109001:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830757.109006:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830757.109019:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db0c0 x1874669228488832/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787830757.109182:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228488576:192.168.203.114@tcp:250:kworker.0 00000800:00000100:1.0:1787830757.109244:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830757.109275:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830757/real 1787830757] req@ffff9856140d8f00 x1874669228488576/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830782 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830757.109291:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8f00 x1874669228488576/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830782 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830757.109302:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830757.109306:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830757.109310:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228488576:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787830757.109316:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228488704:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830757.109338:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830757.109350:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830757/real 1787830757] req@ffff9856140dad00 x1874669228488704/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830782 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830757.109362:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140dad00 x1874669228488704/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830782 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830757.109371:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830757.109374:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830757.109377:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228488704:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787830757.109382:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228488832:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830757.109400:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830757.109411:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830757/real 1787830757] req@ffff9856140db0c0 x1874669228488832/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830782 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830757.109439:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db0c0 x1874669228488832/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830782 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830757.109446:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830757.109448:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830757.109451:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228488832:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830757.172841:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830758.068954:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830758.196953:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830759.092829:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830759.223896:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830760.117257:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830760.244993:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830761.140946:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830761.269167:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830762.165056:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787830762.229038:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830762.229049:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787830762.229053:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830762.229062:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6907 00000100:00080000:3.0:1787830762.229085:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830762.229092:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787830762.229251:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81a40 x1874669228488960/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830762.229276:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830762.229280:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787830762.229282:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830762.229286:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6907 00000100:00080000:3.0:1787830762.229290:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830762.229294:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830762.229303:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830762.229316:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80000 x1874669228489088/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787830762.229328:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228488960:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787830762.229329:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830762.229336:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787830762.229340:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830762.229344:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6907 00000100:00080000:3.0:1787830762.229347:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830762.229351:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830762.229356:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830762.229368:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80b40 x1874669228489216/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787830762.229390:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830762.229422:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830762/real 1787830762] req@ffff985610f81a40 x1874669228488960/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830787 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830762.229437:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81a40 x1874669228488960/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830787 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830762.229447:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830762.229451:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830762.229455:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228488960:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787830762.229476:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228489088:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830762.229497:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830762.229508:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830762/real 1787830762] req@ffff985610f80000 x1874669228489088/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830787 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830762.229519:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80000 x1874669228489088/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830787 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830762.229527:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830762.229530:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830762.229533:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228489088:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787830762.229537:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228489216:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830762.229555:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830762.229566:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830762/real 1787830762] req@ffff985610f80b40 x1874669228489216/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830787 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830762.229577:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80b40 x1874669228489216/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830787 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830762.229585:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830762.229588:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830762.229591:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228489216:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830762.292809:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830763.190383:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830763.316916:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830764.212828:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830764.341456:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830765.236881:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830765.365239:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830766.262195:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830766.389034:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830767.285209:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787830767.348947:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830767.348956:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830767.348961:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830767.348971:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6912 00000100:00080000:0.0:1787830767.348982:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830767.348990:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787830767.349024:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4000 x1874669228489344/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830767.349056:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830767.349060:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830767.349062:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830767.349065:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6912 00000100:00080000:0.0:1787830767.349069:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830767.349073:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787830767.349081:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830767.349095:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7c00 x1874669228489472/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830767.349106:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228489344:192.168.203.114@tcp:250:kworker.0 00000100:00080000:0.0:1787830767.349109:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830767.349115:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830767.349117:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830767.349121:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6912 00000100:00080000:0.0:1787830767.349124:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830767.349128:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787830767.349133:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830767.349145:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4780 x1874669228489600/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787830767.349168:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830767.349204:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830767/real 1787830767] req@ffff985607df4000 x1874669228489344/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830792 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830767.349223:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4000 x1874669228489344/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830792 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830767.349233:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830767.349236:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830767.349240:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228489344:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787830767.349262:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228489472:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830767.349285:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830767.349298:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830767/real 1787830767] req@ffff985607df7c00 x1874669228489472/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830792 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830767.349309:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7c00 x1874669228489472/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830792 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830767.349316:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830767.349319:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830767.349322:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228489472:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787830767.349328:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228489600:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830767.349346:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830767.349355:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830767/real 1787830767] req@ffff985607df4780 x1874669228489600/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830792 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830767.349863:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4780 x1874669228489600/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830792 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830767.349873:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830767.349877:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830767.349880:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228489600:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830767.412909:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830768.308825:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830768.436887:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830769.332830:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830769.460928:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830770.357174:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830770.484961:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830771.380895:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830771.509091:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830772.404870:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787830772.469097:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830772.469107:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830772.469112:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830772.469122:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6918 00000100:00080000:2.0:1787830772.469134:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830772.469141:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787830772.469185:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140da940 x1874669228489728/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830772.469213:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830772.469217:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830772.469219:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830772.469223:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6918 00000100:00080000:2.0:1787830772.469227:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830772.469231:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830772.469246:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830772.469266:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d9a40 x1874669228489856/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830772.469276:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830772.469279:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830772.469281:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830772.469285:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6918 00000100:00080000:2.0:1787830772.469289:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830772.469293:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830772.469298:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830772.469308:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8b40 x1874669228489984/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830772.470045:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228489728:192.168.203.114@tcp:250:kworker.0 00000800:00000100:3.0:1787830772.470135:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830772.470194:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830772/real 1787830772] req@ffff9856140da940 x1874669228489728/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830797 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830772.470214:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140da940 x1874669228489728/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830797 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830772.470227:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830772.470232:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830772.470236:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228489728:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787830772.470244:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228489856:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830772.470268:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830772.470281:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830772/real 1787830772] req@ffff9856140d9a40 x1874669228489856/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830797 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830772.470294:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d9a40 x1874669228489856/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830797 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830772.470308:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830772.470311:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830772.470411:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228489856:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787830772.470418:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228489984:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830772.470445:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830772.470458:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830772/real 1787830772] req@ffff9856140d8b40 x1874669228489984/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830797 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830772.470470:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8b40 x1874669228489984/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830797 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830772.470608:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830772.470614:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830772.470617:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228489984:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830772.532953:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830773.428846:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830773.556987:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830774.452865:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830774.583173:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830775.477834:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830775.605190:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830776.500842:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830776.629034:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830777.524862:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787830777.588853:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830777.588865:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830777.588870:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830777.588882:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6923 00000100:00080000:2.0:1787830777.588895:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830777.588903:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787830777.588957:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db480 x1874669228490112/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830777.588986:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830777.588990:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830777.588992:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830777.588995:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6923 00000100:00080000:2.0:1787830777.588999:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830777.589003:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830777.589014:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830777.589033:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d83c0 x1874669228490240/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830777.589043:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830777.589047:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830777.589049:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830777.589052:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6923 00000100:00080000:2.0:1787830777.589057:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830777.589062:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830777.589067:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830777.589080:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228490112:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787830777.589083:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d9680 x1874669228490368/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787830777.589175:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830777.589225:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830777/real 1787830777] req@ffff9856140db480 x1874669228490112/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830802 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830777.589246:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db480 x1874669228490112/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830802 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830777.589257:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830777.589261:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830777.589266:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228490112:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787830777.589273:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228490240:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830777.589295:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830777.589308:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830777/real 1787830777] req@ffff9856140d83c0 x1874669228490240/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830802 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830777.589319:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d83c0 x1874669228490240/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830802 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830777.589328:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830777.589331:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830777.589334:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228490240:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787830777.589360:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228490368:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830777.589381:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830777.589392:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830777/real 1787830777] req@ffff9856140d9680 x1874669228490368/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830802 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830777.589403:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d9680 x1874669228490368/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830802 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830777.589419:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830777.589423:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830777.589426:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228490368:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830777.652897:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830778.548825:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830778.676881:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830779.572975:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830779.700931:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830780.596805:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830780.725465:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830781.620821:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830781.749132:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830782.644875:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787830782.709205:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830782.709220:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830782.709225:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830782.709237:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6928 00000100:00080000:1.0:1787830782.709250:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830782.709257:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787830782.709303:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2d00 x1874669228490496/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830782.709388:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228490496:192.168.203.114@tcp:250:kworker.0 00000800:00000100:3.0:1787830782.709468:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00080000:1.0:1787830782.709506:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830782.709510:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830782.709513:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830782.709517:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6928 00000100:00080000:1.0:1787830782.709521:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830782.709526:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830782.709537:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830782.709552:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0b40 x1874669228490624/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830782.712210:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830782/real 1787830782] req@ffff9856085a2d00 x1874669228490496/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830807 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830782.712246:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2d00 x1874669228490496/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830807 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830782.712257:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830782.712263:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830782.712267:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228490496:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787830782.712311:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228490624:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830782.712389:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830782.712416:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830782/real 1787830782] req@ffff9856085a0b40 x1874669228490624/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830807 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830782.712442:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0b40 x1874669228490624/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830807 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830782.712450:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830782.712454:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830782.712457:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228490624:192.168.203.114@tcp:38:kworker.0 00000100:00080000:1.0:1787830782.713193:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830782.713200:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830782.713204:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830782.713211:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6928 00000100:00080000:1.0:1787830782.713218:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830782.713227:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830782.713241:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830782.713274:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0000 x1874669228490752/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830782.713374:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228490752:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830782.713411:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830782.713430:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830782/real 1787830782] req@ffff9856085a0000 x1874669228490752/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830807 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830782.713444:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0000 x1874669228490752/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830807 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830782.713460:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830782.713464:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830782.713468:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228490752:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830782.773109:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830783.668919:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830783.796843:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830784.694106:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830784.821391:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830785.716856:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830785.844959:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830786.740940:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830786.868929:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830787.764851:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787830787.828877:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830787.828887:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830787.828891:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830787.828900:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6933 00000100:00080000:2.0:1787830787.828911:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830787.828918:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787830787.828953:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8780 x1874669228490880/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830787.829038:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228490880:192.168.203.114@tcp:250:kworker.0 00000800:00000100:3.0:1787830787.829094:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00080000:2.0:1787830787.829126:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830787.829130:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830787.829132:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830787.829135:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6933 00000100:00080000:2.0:1787830787.829139:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830787.829144:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830787.829153:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830787.829166:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d9680 x1874669228491008/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830787.829494:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830787/real 1787830787] req@ffff9856140d8780 x1874669228490880/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830812 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830787.829509:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8780 x1874669228490880/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830812 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830787.829518:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830787.829521:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830787.829525:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228490880:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787830787.829547:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228491008:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830787.829568:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830787.829579:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830787/real 1787830787] req@ffff9856140d9680 x1874669228491008/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830812 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830787.829589:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d9680 x1874669228491008/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830812 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830787.829596:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830787.829599:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830787.829602:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228491008:192.168.203.114@tcp:38:kworker.0 00000100:00080000:2.0:1787830787.829751:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830787.829755:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830787.829757:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830787.829761:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6933 00000100:00080000:2.0:1787830787.829765:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830787.829769:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830787.829775:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830787.829788:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d83c0 x1874669228491136/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830787.829835:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228491136:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830787.829857:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830787.829918:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830787/real 1787830787] req@ffff9856140d83c0 x1874669228491136/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830812 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830787.829931:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d83c0 x1874669228491136/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830812 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830787.829938:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830787.829941:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830787.829945:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228491136:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830787.892810:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830788.789612:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830788.917096:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830789.812807:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830789.940893:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830790.837050:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830790.964975:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830791.861901:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830791.989009:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830792.884956:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787830792.949005:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830792.949019:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830792.949024:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830792.949037:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6938 00000100:00080000:1.0:1787830792.949051:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830792.949059:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787830792.949100:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a1680 x1874669228491264/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830792.949139:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830792.949143:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830792.949145:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830792.949149:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6938 00000100:00080000:1.0:1787830792.949153:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830792.949157:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830792.949168:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830792.949180:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0780 x1874669228491392/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830792.949191:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830792.949194:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830792.949196:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830792.949200:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6938 00000100:00080000:1.0:1787830792.949204:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830792.949207:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830792.949222:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830792.949233:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3c00 x1874669228491520/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830792.949794:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228491264:192.168.203.114@tcp:250:kworker.0 00000800:00000100:3.0:1787830792.949867:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830792.949959:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830792/real 1787830792] req@ffff9856085a1680 x1874669228491264/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830817 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830792.949981:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a1680 x1874669228491264/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830817 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830792.949993:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830792.949997:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830792.950001:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228491264:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787830792.950009:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228491392:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830792.950034:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830792.950177:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830792/real 1787830792] req@ffff9856085a0780 x1874669228491392/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830817 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830792.950192:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0780 x1874669228491392/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830817 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830792.950200:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830792.950203:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830792.950207:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228491392:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787830792.950213:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228491520:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830792.950236:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830792.950248:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830792/real 1787830792] req@ffff9856085a3c00 x1874669228491520/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830817 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830792.950261:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3c00 x1874669228491520/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830817 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830792.950269:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830792.950272:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830792.950276:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228491520:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830793.013918:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830793.908842:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830794.037321:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830794.932870:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830795.061145:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830795.958070:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830796.086128:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830796.981313:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830797.109212:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830798.005854:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787830798.069428:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830798.069438:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787830798.069443:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830798.069458:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6943 00000100:00080000:3.0:1787830798.069469:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830798.069476:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787830798.069512:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f812c0 x1874669228491648/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830798.069543:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830798.069547:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787830798.069549:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830798.069553:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6943 00000100:00080000:3.0:1787830798.069556:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830798.069560:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830798.069569:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830798.069591:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81680 x1874669228491776/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830798.069600:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830798.069603:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787830798.069605:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830798.069608:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6943 00000100:00080000:3.0:1787830798.069612:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830798.069615:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830798.069620:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830798.069629:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81e00 x1874669228491904/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787830798.069721:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228491648:192.168.203.114@tcp:250:kworker.0 00000800:00000100:1.0:1787830798.069791:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830798.069822:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830798/real 1787830798] req@ffff985610f812c0 x1874669228491648/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830823 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830798.069838:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f812c0 x1874669228491648/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830823 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830798.069848:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830798.069852:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830798.069856:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228491648:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787830798.069862:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228491776:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830798.069884:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830798.069896:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830798/real 1787830798] req@ffff985610f81680 x1874669228491776/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830823 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830798.069908:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81680 x1874669228491776/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830823 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830798.069917:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830798.069920:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830798.069923:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228491776:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787830798.069929:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228491904:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830798.069949:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830798.069961:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830798/real 1787830798] req@ffff985610f81e00 x1874669228491904/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830823 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830798.069973:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81e00 x1874669228491904/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830823 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830798.069980:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830798.069983:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830798.069986:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228491904:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830798.133418:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830799.029812:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830799.156991:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830800.052830:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830800.188485:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830801.077414:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830801.205031:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830802.101777:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830802.231729:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830803.124846:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787830803.188867:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830803.188880:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830803.188885:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830803.188897:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6948 00000100:00080000:1.0:1787830803.188910:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830803.188919:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787830803.188971:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a03c0 x1874669228492032/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830803.189007:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830803.189011:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830803.189013:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830803.189017:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6948 00000100:00080000:1.0:1787830803.189020:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830803.189024:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830803.189035:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830803.189047:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3840 x1874669228492160/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830803.189057:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830803.189060:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830803.189062:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830803.189065:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6948 00000100:00080000:1.0:1787830803.189069:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830803.189072:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830803.189076:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830803.189095:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2580 x1874669228492288/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830803.189360:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228492032:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787830803.189475:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228492160:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787830803.189502:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228492288:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830803.252901:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830804.165062:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830804.277449:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830805.172906:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830805.302998:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830806.197177:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000400:00000100:0.0:1787830806.264038:0:2373:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.114:988 00000400:00000100:0.0:1787830806.264114:0:2373:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.114@tcp at host 192.168.203.114 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:0.0:1787830806.264148:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:0.0:1787830806.264199:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:0.0:1787830806.264212:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000100:00100000:3.0:1787830806.264354:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830803/real 1787830806] req@ffff9856085a03c0 x1874669228492032/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830828 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830806.264383:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a03c0 x1874669228492032/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830828 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830806.264397:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830806.264403:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830806.264407:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228492032:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787830806.264417:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830803/real 1787830806] req@ffff9856085a3840 x1874669228492160/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830828 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830806.264444:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3840 x1874669228492160/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830828 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830806.264459:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830806.264463:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830806.264467:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228492160:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787830806.264474:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830803/real 1787830806] req@ffff9856085a2580 x1874669228492288/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830828 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830806.264494:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2580 x1874669228492288/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830828 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830806.264502:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830806.264506:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830806.264509:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228492288:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787830806.330355:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830807.220820:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830807.349023:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830808.245352:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787830808.310624:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830808.310638:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787830808.310643:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830808.310656:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6953 00000100:00080000:3.0:1787830808.310671:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830808.310682:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787830808.310770:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83840 x1874669228492416/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830808.310809:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830808.310813:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787830808.310814:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830808.310819:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6953 00000100:00080000:3.0:1787830808.310823:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830808.310826:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830808.310847:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830808.310864:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f803c0 x1874669228492544/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830808.310874:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830808.310878:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787830808.310880:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830808.310884:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6953 00000100:00080000:3.0:1787830808.310888:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830808.310891:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830808.310896:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830808.310907:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82d00 x1874669228492672/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787830808.310977:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228492416:192.168.203.114@tcp:250:kworker.0 00000800:00000100:1.0:1787830808.311068:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830808.311112:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830808/real 1787830808] req@ffff985610f83840 x1874669228492416/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830833 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830808.311133:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83840 x1874669228492416/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830833 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830808.311146:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830808.311151:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830808.311156:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228492416:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787830808.311164:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228492544:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830808.311190:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830808.311203:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830808/real 1787830808] req@ffff985610f803c0 x1874669228492544/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830833 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830808.311215:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f803c0 x1874669228492544/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830833 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830808.311224:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830808.311228:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830808.311231:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228492544:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787830808.311237:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228492672:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830808.311257:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830808.311269:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830808/real 1787830808] req@ffff985610f82d00 x1874669228492672/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830833 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830808.311288:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82d00 x1874669228492672/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830833 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830808.311296:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830808.311299:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830808.311302:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228492672:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830808.372911:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830809.270107:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830809.397685:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830810.293065:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830810.421024:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830811.317584:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830811.446162:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830812.340906:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830812.470122:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830813.366568:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787830813.428904:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830813.428912:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787830813.428916:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830813.428926:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6959 00000100:00080000:3.0:1787830813.428935:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830813.428942:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787830813.428975:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82940 x1874669228492800/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787830813.429042:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228492800:192.168.203.114@tcp:250:kworker.0 00000800:00000100:1.0:1787830813.429106:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00080000:3.0:1787830813.429131:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830813.429135:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787830813.429137:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830813.429141:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6959 00000100:00080000:3.0:1787830813.429144:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830813.429148:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830813.429156:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830813.429167:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82580 x1874669228492928/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787830813.430345:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830813/real 1787830813] req@ffff985610f82940 x1874669228492800/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830838 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830813.430366:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82940 x1874669228492800/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830838 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830813.430376:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830813.430380:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830813.430389:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228492800:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787830813.430669:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228492928:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830813.430729:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830813.430744:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830813/real 1787830813] req@ffff985610f82580 x1874669228492928/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830838 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830813.430757:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82580 x1874669228492928/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830838 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830813.430766:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830813.430770:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830813.430773:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228492928:192.168.203.114@tcp:38:kworker.0 00000100:00080000:3.0:1787830813.431311:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830813.431315:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787830813.431317:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830813.431321:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6959 00000100:00080000:3.0:1787830813.431328:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830813.431333:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830813.431343:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830813.431363:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80f00 x1874669228493056/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787830813.431530:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228493056:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830813.431626:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830813.431656:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830813/real 1787830813] req@ffff985610f80f00 x1874669228493056/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830838 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830813.431668:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80f00 x1874669228493056/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830838 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830813.431676:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830813.431679:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830813.431682:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228493056:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830813.494422:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830814.389338:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830814.516988:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830815.413240:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830815.541374:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830816.436853:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830816.565153:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830817.460857:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830817.588921:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830818.484858:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787830818.551185:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830818.551196:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830818.551200:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830818.551209:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6964 00000100:00080000:1.0:1787830818.551219:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830818.551226:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787830818.551260:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a12c0 x1874669228493184/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830818.551335:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228493184:192.168.203.114@tcp:250:kworker.0 00000800:00000100:3.0:1787830818.551393:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00080000:1.0:1787830818.551430:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830818.551443:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830818.551445:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830818.551449:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6964 00000100:00080000:1.0:1787830818.551453:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830818.551457:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830818.551465:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830818.551478:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2940 x1874669228493312/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830818.558311:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830818/real 1787830818] req@ffff9856085a12c0 x1874669228493184/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830843 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830818.558342:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a12c0 x1874669228493184/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830843 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830818.558353:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830818.558358:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830818.558362:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228493184:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787830818.558394:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228493312:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830818.558825:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830818.558844:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830818/real 1787830818] req@ffff9856085a2940 x1874669228493312/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830843 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830818.558858:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2940 x1874669228493312/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830843 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830818.558867:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830818.558871:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830818.558874:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228493312:192.168.203.114@tcp:38:kworker.0 00000100:00080000:1.0:1787830818.559425:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830818.559430:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830818.559440:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830818.559446:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6964 00000100:00080000:1.0:1787830818.559453:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830818.559459:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830818.559470:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830818.559496:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a1a40 x1874669228493440/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830818.560355:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228493440:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830818.560426:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830818.560467:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830818/real 1787830818] req@ffff9856085a1a40 x1874669228493440/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830843 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830818.560482:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a1a40 x1874669228493440/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830843 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830818.560496:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830818.560500:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830818.560504:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228493440:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830818.612854:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830819.508987:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830819.638005:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830820.532870:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830820.660866:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830821.556887:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830821.686192:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830822.580826:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830822.709060:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830823.606431:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787830823.668921:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830823.668935:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787830823.668940:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830823.668953:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6969 00000100:00080000:3.0:1787830823.668968:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830823.668977:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787830823.669526:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f821c0 x1874669228493568/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787830823.669606:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f821c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228493568:192.168.203.114@tcp:250:kworker.0 00000800:00000100:1.0:1787830823.669688:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00080000:3.0:1787830823.669826:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830823.669830:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787830823.669833:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830823.669837:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6969 00000100:00080000:3.0:1787830823.669842:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830823.669847:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830823.669858:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830823.669871:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83480 x1874669228493696/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787830823.670653:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830823/real 1787830823] req@ffff985610f821c0 x1874669228493568/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830848 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830823.670677:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f821c0 x1874669228493568/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830848 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830823.670690:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830823.670722:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830823.670727:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f821c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228493568:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787830823.670760:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228493696:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830823.670806:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830823.670822:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830823/real 1787830823] req@ffff985610f83480 x1874669228493696/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830848 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830823.670835:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83480 x1874669228493696/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830848 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830823.670844:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830823.670848:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830823.670851:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228493696:192.168.203.114@tcp:38:kworker.0 00000100:00080000:3.0:1787830823.670937:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830823.670941:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787830823.670944:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830823.670950:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6969 00000100:00080000:3.0:1787830823.670955:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830823.670960:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830823.670969:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830823.671497:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83480 x1874669228493824/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787830823.671590:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228493824:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787830823.671633:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830823.671650:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830823/real 1787830823] req@ffff985610f83480 x1874669228493824/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830848 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830823.671664:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83480 x1874669228493824/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830848 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830823.671677:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830823.671681:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787830823.671684:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228493824:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830823.732888:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830824.628977:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830824.756914:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830825.652771:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830825.780903:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830826.676902:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830826.804893:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830827.700855:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830827.829884:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830828.724804:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787830828.794877:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830828.794891:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830828.794896:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830828.794909:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6974 00000100:00080000:1.0:1787830828.794923:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830828.794933:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787830828.794977:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a21c0 x1874669228493952/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830828.795010:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830828.795015:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830828.795016:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830828.795021:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6974 00000100:00080000:1.0:1787830828.795025:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830828.795029:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830828.795040:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830828.795053:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3480 x1874669228494080/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830828.795063:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830828.795068:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830828.795070:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830828.795074:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6974 00000100:00080000:1.0:1787830828.795077:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830828.795082:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830828.795087:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830828.795098:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a30c0 x1874669228494208/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830828.795384:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228493952:192.168.203.114@tcp:250:kworker.0 00000800:00000100:3.0:1787830828.795458:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830828.795496:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830828/real 1787830828] req@ffff9856085a21c0 x1874669228493952/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830853 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830828.795516:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a21c0 x1874669228493952/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830853 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830828.795527:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830828.795530:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830828.795535:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228493952:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787830828.795542:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228494080:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830828.795562:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830828.795574:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830828/real 1787830828] req@ffff9856085a3480 x1874669228494080/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830853 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830828.795585:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3480 x1874669228494080/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830853 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830828.795593:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830828.795596:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830828.795600:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228494080:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787830828.795604:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228494208:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830828.795619:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830828.795638:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830828/real 1787830828] req@ffff9856085a30c0 x1874669228494208/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830853 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830828.795650:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a30c0 x1874669228494208/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830853 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830828.795658:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830828.795661:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830828.795664:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228494208:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830828.853578:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830829.749048:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830829.876874:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830830.772957:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830830.902867:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830831.796841:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830831.924968:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830832.820843:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830832.949222:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830833.846470:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787830833.910863:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830833.910876:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830833.910881:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830833.910893:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6979 00000100:00080000:0.0:1787830833.910906:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830833.910914:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787830833.910961:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6940 x1874669228494336/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830833.910992:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830833.910995:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830833.910998:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830833.911001:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6979 00000100:00080000:0.0:1787830833.911005:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830833.911010:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787830833.911021:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830833.911033:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4b40 x1874669228494464/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830833.911042:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830833.911046:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830833.911048:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830833.911051:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6979 00000100:00080000:0.0:1787830833.911054:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830833.911059:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787830833.911063:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830833.911074:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7480 x1874669228494592/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830833.911175:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228494336:192.168.203.114@tcp:250:kworker.0 00000800:00000100:3.0:1787830833.911251:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830833.911288:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830833/real 1787830833] req@ffff985607df6940 x1874669228494336/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830858 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830833.911318:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6940 x1874669228494336/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830858 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830833.911340:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830833.911345:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830833.911349:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228494336:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787830833.911357:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228494464:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830833.911382:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830833.911395:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830833/real 1787830833] req@ffff985607df4b40 x1874669228494464/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830858 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830833.911417:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4b40 x1874669228494464/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830858 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830833.911425:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830833.911428:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830833.911431:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228494464:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787830833.911437:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228494592:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830833.911457:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830833.911468:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830833/real 1787830833] req@ffff985607df7480 x1874669228494592/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830858 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830833.911487:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7480 x1874669228494592/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830858 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830833.911494:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830833.911497:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830833.911500:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228494592:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830833.972919:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830834.868860:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830834.997279:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830835.906384:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830836.022426:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830836.917164:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830837.045265:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830837.940969:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830838.069426:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830838.964932:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787830839.029425:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830839.029440:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830839.029445:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830839.029458:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6984 00000100:00080000:0.0:1787830839.029473:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830839.029481:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787830839.029528:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df52c0 x1874669228494720/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830839.030447:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228494720:192.168.203.114@tcp:250:kworker.0 00000800:00000100:3.0:1787830839.030539:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830839.031607:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830839/real 1787830839] req@ffff985607df52c0 x1874669228494720/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830864 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830839.032903:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830839.032913:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830839.032916:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830839.032925:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6984 00000100:00080000:0.0:1787830839.032936:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830839.032945:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787830839.032960:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830839.032999:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7840 x1874669228494848/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830839.033018:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830839.033021:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830839.033024:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830839.033092:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6984 00000100:00080000:0.0:1787830839.033097:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830839.033101:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787830839.033106:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830839.033119:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4f00 x1874669228494976/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830839.033551:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df52c0 x1874669228494720/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830864 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830839.033564:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830839.033568:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830839.033574:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228494720:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787830839.033610:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228494848:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830839.033901:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830839.033933:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830839/real 1787830839] req@ffff985607df7840 x1874669228494848/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830864 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830839.033946:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7840 x1874669228494848/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830864 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830839.033961:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830839.033964:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830839.033967:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228494848:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787830839.033974:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228494976:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830839.034024:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830839.034037:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830839/real 1787830839] req@ffff985607df4f00 x1874669228494976/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830864 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830839.034050:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4f00 x1874669228494976/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830864 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830839.034058:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830839.034061:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830839.034065:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228494976:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830839.092902:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830839.988876:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830840.117618:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830841.012824:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830841.142072:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830842.038372:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830842.166041:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830843.061075:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830843.189105:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830844.085944:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787830844.150125:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830844.150140:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830844.150145:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830844.150157:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6989 00000100:00080000:2.0:1787830844.150173:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830844.150182:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787830844.150234:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db480 x1874669228495104/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830844.150283:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830844.150288:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830844.150290:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830844.150295:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6989 00000100:00080000:2.0:1787830844.150299:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830844.150303:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830844.150314:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830844.150330:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8b40 x1874669228495232/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830844.150340:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830844.150344:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830844.150345:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830844.150350:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6989 00000100:00080000:2.0:1787830844.150354:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830844.150360:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830844.150364:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830844.150374:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d9a40 x1874669228495360/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830844.150474:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228495104:192.168.203.114@tcp:250:kworker.0 00000800:00000100:3.0:1787830844.150555:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830844.150600:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830844/real 1787830844] req@ffff9856140db480 x1874669228495104/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830869 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830844.150620:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db480 x1874669228495104/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830869 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830844.150632:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830844.150648:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830844.150652:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228495104:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787830844.150660:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228495232:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830844.150685:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830844.150725:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830844/real 1787830844] req@ffff9856140d8b40 x1874669228495232/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830869 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830844.150737:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8b40 x1874669228495232/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830869 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830844.150746:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830844.150749:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830844.150752:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228495232:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787830844.150758:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228495360:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830844.150777:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830844.150789:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830844/real 1787830844] req@ffff9856140d9a40 x1874669228495360/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830869 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830844.150800:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d9a40 x1874669228495360/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830869 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830844.150809:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830844.150812:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830844.150815:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228495360:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830844.213077:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830845.108892:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830845.239303:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830846.132854:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830846.265612:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830847.157434:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830847.285140:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830848.180913:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830848.309147:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830849.205138:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787830849.269355:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830849.269372:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830849.269379:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830849.269392:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 6994 00000100:00080000:1.0:1787830849.269407:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830849.269418:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787830849.269469:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0f00 x1874669228495488/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830849.269573:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830849.269577:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830849.269579:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830849.269584:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6994 00000100:00080000:1.0:1787830849.269588:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830849.269593:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830849.269605:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830849.269624:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a30c0 x1874669228495616/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830849.269634:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830849.269637:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830849.269639:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830849.269643:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 6994 00000100:00080000:1.0:1787830849.269647:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830849.269651:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830849.269655:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830849.269666:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3480 x1874669228495744/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830849.269860:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228495488:192.168.203.114@tcp:250:kworker.0 00000800:00000100:3.0:1787830849.273192:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830849.273262:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830849/real 1787830849] req@ffff9856085a0f00 x1874669228495488/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830874 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830849.273288:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0f00 x1874669228495488/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830874 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830849.273299:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830849.273304:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830849.273308:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228495488:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787830849.273320:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228495616:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830849.273369:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830849.273381:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830849/real 1787830849] req@ffff9856085a30c0 x1874669228495616/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830874 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830849.273392:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a30c0 x1874669228495616/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830874 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830849.273399:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830849.273408:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830849.273411:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228495616:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787830849.273418:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228495744:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830849.274140:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830849.274163:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830849/real 1787830849] req@ffff9856085a3480 x1874669228495744/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830874 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830849.274179:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3480 x1874669228495744/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830874 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830849.274187:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830849.274191:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830849.274194:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228495744:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830849.334004:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830850.228831:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830850.356879:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830851.252815:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830851.381466:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830852.276822:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830852.404887:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830853.300805:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830853.429808:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830854.324865:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787830854.388800:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830854.388809:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830854.388813:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830854.388823:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 7000 00000100:00080000:1.0:1787830854.388833:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830854.388840:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787830854.388874:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a21c0 x1874669228495872/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830854.388900:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830854.388903:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830854.388905:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830854.388909:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7000 00000100:00080000:1.0:1787830854.388913:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830854.388918:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830854.388935:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830854.388936:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228495872:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787830854.388951:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a1a40 x1874669228496000/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830854.388962:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830854.388966:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830854.388968:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830854.388972:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7000 00000100:00080000:1.0:1787830854.388976:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830854.388980:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830854.388985:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000800:00000100:3.0:1787830854.388997:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787830854.389000:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2940 x1874669228496128/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830854.389042:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830854/real 1787830854] req@ffff9856085a21c0 x1874669228495872/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830879 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830854.389057:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a21c0 x1874669228495872/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830879 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830854.389065:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830854.389069:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830854.389072:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228495872:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787830854.389094:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228496000:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830854.389123:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830854.389136:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830854/real 1787830854] req@ffff9856085a1a40 x1874669228496000/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830879 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830854.389147:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a1a40 x1874669228496000/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830879 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830854.389154:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830854.389158:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830854.389161:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228496000:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787830854.389167:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228496128:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830854.389200:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830854.389210:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830854/real 1787830854] req@ffff9856085a2940 x1874669228496128/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830879 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830854.389217:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2940 x1874669228496128/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830879 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830854.389225:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830854.389228:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830854.389231:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228496128:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830854.452894:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830855.348810:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830855.476857:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830856.373140:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830856.500895:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830857.396883:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830857.525055:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830858.420807:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830858.549321:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830859.444889:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787830859.511636:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830859.511650:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830859.511656:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830859.511669:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 7005 00000100:00080000:1.0:1787830859.511683:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830859.511711:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787830859.511760:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a12c0 x1874669228496256/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830859.511850:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830859.511855:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830859.511857:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830859.511862:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7005 00000100:00080000:1.0:1787830859.511867:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830859.511871:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830859.511884:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830859.511899:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2580 x1874669228496384/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830859.511909:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830859.511912:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830859.511914:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830859.511918:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7005 00000100:00080000:1.0:1787830859.511923:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830859.511926:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830859.511931:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830859.511942:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3840 x1874669228496512/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830859.512607:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228496256:192.168.203.114@tcp:250:kworker.0 00000800:00000100:3.0:1787830859.512735:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830859.512782:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830859/real 1787830859] req@ffff9856085a12c0 x1874669228496256/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830884 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830859.512802:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a12c0 x1874669228496256/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830884 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830859.512812:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830859.512816:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830859.512820:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228496256:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787830859.512829:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228496384:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830859.512855:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830859.512870:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830859/real 1787830859] req@ffff9856085a2580 x1874669228496384/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830884 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830859.512882:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2580 x1874669228496384/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830884 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830859.512890:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830859.512893:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830859.512896:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228496384:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787830859.512903:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228496512:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830859.512925:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830859.512938:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830859/real 1787830859] req@ffff9856085a3840 x1874669228496512/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830884 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830859.512957:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3840 x1874669228496512/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830884 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830859.512966:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830859.512970:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830859.512973:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228496512:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830859.573191:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830860.468853:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830860.597636:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830861.495617:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830861.620890:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830862.519602:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830862.648134:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830863.541105:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830863.669678:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830864.565100:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787830864.630773:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830864.630783:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830864.630787:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830864.630796:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 7010 00000100:00080000:0.0:1787830864.630807:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830864.630814:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787830864.630852:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6d00 x1874669228496640/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830864.630884:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830864.630888:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830864.630890:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830864.630893:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7010 00000100:00080000:0.0:1787830864.630896:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830864.630899:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787830864.630908:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830864.630918:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5e00 x1874669228496768/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830864.630926:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830864.630929:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830864.630930:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830864.630933:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7010 00000100:00080000:0.0:1787830864.630936:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830864.630939:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787830864.630943:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830864.630951:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df61c0 x1874669228496896/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830864.631384:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228496640:192.168.203.114@tcp:250:kworker.0 00000800:00000100:3.0:1787830864.631440:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830864.631470:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830864/real 1787830864] req@ffff985607df6d00 x1874669228496640/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830889 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830864.631487:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6d00 x1874669228496640/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830889 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830864.631495:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830864.631499:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830864.631502:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228496640:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787830864.631521:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228496768:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830864.631542:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830864.631554:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830864/real 1787830864] req@ffff985607df5e00 x1874669228496768/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830889 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830864.631565:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5e00 x1874669228496768/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830889 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830864.631572:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830864.631575:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830864.631577:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228496768:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787830864.631595:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228496896:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830864.631673:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830864.631685:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830864/real 1787830864] req@ffff985607df61c0 x1874669228496896/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830889 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830864.631724:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df61c0 x1874669228496896/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830889 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830864.631731:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830864.631734:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830864.631737:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228496896:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830864.693767:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830865.588816:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830865.718871:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830866.614260:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830866.743762:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830867.637373:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830867.775833:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830868.660828:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830868.789574:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830869.684804:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787830869.749116:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830869.749125:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830869.749129:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830869.749139:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 7015 00000100:00080000:1.0:1787830869.749149:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830869.749156:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787830869.749190:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a03c0 x1874669228497024/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830869.749221:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830869.749225:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830869.749227:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830869.749231:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7015 00000100:00080000:1.0:1787830869.749234:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830869.749238:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830869.749247:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830869.749275:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3c00 x1874669228497152/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830869.749285:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830869.749293:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830869.749294:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830869.749298:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7015 00000100:00080000:1.0:1787830869.749302:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830869.749305:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830869.749309:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830869.749319:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0780 x1874669228497280/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830869.749599:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228497024:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787830869.749682:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228497152:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787830869.749735:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228497280:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830869.812844:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830870.709835:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830870.837546:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830871.732918:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830871.860944:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830872.756876:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000400:00000100:1.0:1787830872.824161:0:2370:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.114:988 00000400:00000100:1.0:1787830872.824208:0:2370:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.114@tcp at host 192.168.203.114 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1787830872.824223:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:1.0:1787830872.824267:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:1.0:1787830872.824277:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000100:00100000:0.0:1787830872.827861:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830869/real 1787830872] req@ffff9856085a03c0 x1874669228497024/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830894 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830872.827895:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a03c0 x1874669228497024/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830894 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830872.827906:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830872.827911:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830872.827915:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228497024:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787830872.827922:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830869/real 1787830872] req@ffff9856085a3c00 x1874669228497152/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830894 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830872.827934:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3c00 x1874669228497152/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830894 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830872.827941:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830872.827944:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830872.827946:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228497152:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787830872.827952:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830869/real 1787830872] req@ffff9856085a0780 x1874669228497280/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830894 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830872.827962:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0780 x1874669228497280/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830894 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830872.827967:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830872.827970:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830872.827973:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228497280:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787830872.884965:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830873.785070:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830873.909218:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830874.805222:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787830874.869960:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830874.869975:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830874.869979:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830874.870135:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 7020 00000100:00080000:0.0:1787830874.870150:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830874.870159:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787830874.870202:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df43c0 x1874669228497408/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830874.870240:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830874.870245:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830874.870247:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830874.870251:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7020 00000100:00080000:0.0:1787830874.870254:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830874.870259:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787830874.870271:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830874.870283:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5680 x1874669228497536/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830874.870292:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830874.870295:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830874.870297:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830874.870301:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7020 00000100:00080000:0.0:1787830874.870304:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830874.870308:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787830874.870312:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830874.870321:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df70c0 x1874669228497664/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830874.870424:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228497408:192.168.203.114@tcp:250:kworker.0 00000800:00000100:3.0:1787830874.870503:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830874.870540:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830874/real 1787830874] req@ffff985607df43c0 x1874669228497408/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830899 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830874.870559:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df43c0 x1874669228497408/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830899 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830874.870570:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830874.870574:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830874.870579:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228497408:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787830874.870586:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228497536:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830874.870607:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830874.870619:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830874/real 1787830874] req@ffff985607df5680 x1874669228497536/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830899 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830874.870629:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5680 x1874669228497536/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830899 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830874.870636:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830874.870639:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830874.870643:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228497536:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787830874.870649:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228497664:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830874.870667:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830874.870677:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830874/real 1787830874] req@ffff985607df70c0 x1874669228497664/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830899 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830874.870687:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df70c0 x1874669228497664/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830899 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830874.870738:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830874.870742:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830874.870747:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228497664:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830874.934274:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830875.828847:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830875.957397:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830876.853302:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830876.987114:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830877.877225:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830878.005055:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830878.901004:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830879.038415:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830879.925092:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787830879.988947:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830879.988958:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830879.988963:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830879.988974:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 7025 00000100:00080000:2.0:1787830879.988987:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830879.988995:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787830879.989059:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140da940 x1874669228497792/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830879.989098:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830879.989102:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830879.989104:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830879.989108:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7025 00000100:00080000:2.0:1787830879.989111:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830879.989115:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830879.989126:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830879.989137:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db0c0 x1874669228497920/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830879.989146:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830879.989149:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830879.989159:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830879.989162:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7025 00000100:00080000:2.0:1787830879.989166:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830879.989170:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830879.989174:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830879.989183:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140dad00 x1874669228498048/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830879.989309:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228497792:192.168.203.114@tcp:250:kworker.0 00000800:00000100:0.0:1787830879.989386:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830879.989427:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830879/real 1787830879] req@ffff9856140da940 x1874669228497792/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830904 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830879.989445:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140da940 x1874669228497792/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830904 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830879.989467:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830879.989471:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830879.989475:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228497792:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787830879.989482:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228497920:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830879.989502:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830879.989513:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830879/real 1787830879] req@ffff9856140db0c0 x1874669228497920/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830904 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830879.989525:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db0c0 x1874669228497920/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830904 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830879.989533:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830879.989536:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830879.989539:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228497920:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787830879.989544:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228498048:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830879.989559:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830879.989567:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830879/real 1787830879] req@ffff9856140dad00 x1874669228498048/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830904 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830879.989576:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140dad00 x1874669228498048/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830904 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830879.989584:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830879.989587:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830879.989591:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228498048:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830880.052821:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830880.949150:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830881.077088:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830881.989839:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830882.102723:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830882.997652:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830883.125767:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830884.020854:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830884.149768:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830885.044837:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787830885.110320:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830885.110331:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830885.110335:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830885.110344:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 7030 00000100:00080000:1.0:1787830885.110355:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830885.110362:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787830885.110406:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a1680 x1874669228498176/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830885.110470:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228498176:192.168.203.114@tcp:250:kworker.0 00000800:00000100:0.0:1787830885.110524:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00080000:1.0:1787830885.110552:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830885.110555:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830885.110557:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830885.110561:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7030 00000100:00080000:1.0:1787830885.110564:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830885.110568:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830885.110576:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830885.110587:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0000 x1874669228498304/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830885.116026:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830885/real 1787830885] req@ffff9856085a1680 x1874669228498176/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830910 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830885.116164:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a1680 x1874669228498176/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830910 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830885.116177:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830885.116181:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830885.116186:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228498176:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787830885.116216:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228498304:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830885.116274:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830885.116289:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830885/real 1787830885] req@ffff9856085a0000 x1874669228498304/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830910 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830885.116300:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0000 x1874669228498304/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830910 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830885.116308:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830885.116311:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830885.116314:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228498304:192.168.203.114@tcp:38:kworker.0 00000100:00080000:1.0:1787830885.119786:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830885.119794:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830885.119798:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830885.119806:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7030 00000100:00080000:1.0:1787830885.119815:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830885.119822:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830885.119833:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830885.119864:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0b40 x1874669228498432/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830885.120021:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228498432:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830885.120139:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830885.120164:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830885/real 1787830885] req@ffff9856085a0b40 x1874669228498432/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830910 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830885.120180:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0b40 x1874669228498432/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830910 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830885.120190:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830885.120194:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830885.120197:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228498432:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830885.172826:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830886.068843:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830886.197172:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830887.093434:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830887.221563:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830888.116864:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830888.245080:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830889.141742:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830889.269900:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830890.164925:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787830890.230391:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830890.230404:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830890.230410:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830890.230423:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 7035 00000100:00080000:1.0:1787830890.230437:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830890.230446:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787830890.230492:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2d00 x1874669228498560/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830890.230526:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830890.230530:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830890.230532:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830890.230535:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7035 00000100:00080000:1.0:1787830890.230539:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830890.230543:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830890.230554:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830890.230568:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0b40 x1874669228498688/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830890.230577:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830890.230581:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830890.230582:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830890.230586:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7035 00000100:00080000:1.0:1787830890.230588:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830890.230591:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830890.230596:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830890.230604:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0000 x1874669228498816/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830890.230731:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228498560:192.168.203.114@tcp:250:kworker.0 00000800:00000100:0.0:1787830890.230811:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830890.230852:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830890/real 1787830890] req@ffff9856085a2d00 x1874669228498560/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830915 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830890.230870:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2d00 x1874669228498560/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830915 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830890.230881:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830890.230885:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830890.230889:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228498560:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787830890.230895:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228498688:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830890.230916:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830890.230928:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830890/real 1787830890] req@ffff9856085a0b40 x1874669228498688/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830915 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830890.230951:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0b40 x1874669228498688/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830915 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830890.230960:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830890.230963:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830890.230966:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228498688:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787830890.230971:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228498816:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830890.230988:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830890.231001:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830890/real 1787830890] req@ffff9856085a0000 x1874669228498816/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830915 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830890.231011:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0000 x1874669228498816/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830915 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830890.231019:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830890.231021:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830890.231024:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228498816:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830890.292890:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830891.188885:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830891.316918:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830892.212824:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830892.340884:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830893.237006:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830893.364948:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830894.260800:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830894.389819:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830895.284906:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787830895.348856:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830895.348870:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787830895.348876:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830895.348889:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 7040 00000100:00080000:3.0:1787830895.348911:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830895.348921:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787830895.348968:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f821c0 x1874669228498944/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830895.349003:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830895.349007:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787830895.349009:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830895.349013:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7040 00000100:00080000:3.0:1787830895.349017:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830895.349022:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830895.349034:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830895.349049:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80f00 x1874669228499072/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830895.349059:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830895.349063:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787830895.349064:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830895.349069:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7040 00000100:00080000:3.0:1787830895.349074:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830895.349078:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830895.349082:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830895.349093:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82580 x1874669228499200/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830895.349202:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f821c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228498944:192.168.203.114@tcp:250:kworker.0 00000800:00000100:0.0:1787830895.349274:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830895.349322:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830895/real 1787830895] req@ffff985610f821c0 x1874669228498944/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830920 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830895.349338:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f821c0 x1874669228498944/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830920 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830895.349362:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830895.349369:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830895.349374:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f821c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228498944:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787830895.349381:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228499072:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830895.349403:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830895.349414:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830895/real 1787830895] req@ffff985610f80f00 x1874669228499072/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830920 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830895.349424:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80f00 x1874669228499072/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830920 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830895.349431:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830895.349434:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830895.349437:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228499072:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787830895.349443:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228499200:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830895.349461:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830895.349471:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830895/real 1787830895] req@ffff985610f82580 x1874669228499200/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830920 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830895.349480:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82580 x1874669228499200/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830920 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830895.349487:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830895.349490:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830895.349493:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228499200:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830895.412874:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830896.309946:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830896.436886:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830897.332822:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830897.462120:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830898.356811:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830898.485171:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830899.380835:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830899.509223:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830900.404810:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787830900.469256:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830900.469265:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830900.469269:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830900.469278:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 7046 00000100:00080000:1.0:1787830900.469288:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830900.469295:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787830900.469329:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a1680 x1874669228499328/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830900.469356:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830900.469360:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830900.469362:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830900.469365:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7046 00000100:00080000:1.0:1787830900.469368:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830900.469372:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830900.469381:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830900.469403:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0780 x1874669228499456/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830900.469413:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830900.469416:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830900.469418:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830900.469421:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7046 00000100:00080000:1.0:1787830900.469424:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830900.469427:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830900.469432:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830900.469442:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3c00 x1874669228499584/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830900.469666:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228499328:192.168.203.114@tcp:250:kworker.0 00000800:00000100:0.0:1787830900.469759:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830900.469800:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830900/real 1787830900] req@ffff9856085a1680 x1874669228499328/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830925 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830900.469816:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a1680 x1874669228499328/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830925 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830900.469826:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830900.469829:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830900.469834:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228499328:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787830900.469850:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228499456:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830900.469889:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830900.469902:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830900/real 1787830900] req@ffff9856085a0780 x1874669228499456/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830925 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830900.469914:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0780 x1874669228499456/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830925 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830900.469922:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830900.469926:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830900.469929:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228499456:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787830900.469935:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228499584:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830900.469955:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830900.469965:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830900/real 1787830900] req@ffff9856085a3c00 x1874669228499584/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830925 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830900.469981:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3c00 x1874669228499584/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830925 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830900.469989:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830900.469992:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830900.469995:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228499584:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830900.532852:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830901.428837:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830901.558160:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830902.452845:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830902.582222:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830903.476966:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830903.605972:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830904.500924:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830904.630401:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830905.524962:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787830905.588883:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830905.588897:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830905.588902:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830905.588916:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 7051 00000100:00080000:1.0:1787830905.588936:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830905.588945:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787830905.589006:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a03c0 x1874669228499712/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830905.589033:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830905.589036:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830905.589038:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830905.589041:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7051 00000100:00080000:1.0:1787830905.589045:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830905.589049:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787830905.589076:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228499712:192.168.203.114@tcp:250:kworker.0 00000800:00000100:0.0:1787830905.589158:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830905.589204:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830905/real 1787830905] req@ffff9856085a03c0 x1874669228499712/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830930 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830905.589223:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a03c0 x1874669228499712/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830930 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830905.589235:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830905.589239:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830905.589244:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228499712:192.168.203.114@tcp:250:kworker.0 00000008:00100000:1.0:1787830905.590583:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830905.590629:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3840 x1874669228499840/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830905.590662:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830905.590667:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830905.590669:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830905.590674:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7051 00000100:00080000:1.0:1787830905.590684:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830905.590704:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830905.590711:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830905.590729:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2580 x1874669228499968/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830905.590765:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228499840:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830905.590862:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830905.590896:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830905/real 1787830905] req@ffff9856085a3840 x1874669228499840/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830930 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830905.590915:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3840 x1874669228499840/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830930 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830905.590933:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830905.590949:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830905.590954:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228499840:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787830905.590962:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228499968:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830905.590995:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830905.591008:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830905/real 1787830905] req@ffff9856085a2580 x1874669228499968/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830930 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830905.591020:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2580 x1874669228499968/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830930 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830905.591028:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830905.591032:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830905.591035:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228499968:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830905.653050:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830906.549100:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830906.679074:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830907.572950:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830907.701427:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830908.596958:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830908.725211:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830909.621173:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830909.750532:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830910.644945:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787830910.708908:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830910.708921:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830910.708927:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830910.708939:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 7056 00000100:00080000:0.0:1787830910.708953:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830910.708962:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787830910.709011:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6580 x1874669228500096/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830910.709660:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228500096:192.168.203.114@tcp:250:kworker.0 00000800:00000100:3.0:1787830910.709775:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00080000:0.0:1787830910.709848:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830910.709853:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830910.709855:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830910.709861:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7056 00000100:00080000:0.0:1787830910.709866:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830910.709872:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787830910.709883:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830910.709901:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5a40 x1874669228500224/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830910.713777:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830910/real 1787830910] req@ffff985607df6580 x1874669228500096/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830935 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830910.713861:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6580 x1874669228500096/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830935 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830910.713989:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830910.713997:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830910.714002:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228500096:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787830910.714046:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228500224:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830910.714242:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830910.714263:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830910/real 1787830910] req@ffff985607df5a40 x1874669228500224/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830935 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830910.714275:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5a40 x1874669228500224/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830935 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830910.714282:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830910.714286:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830910.714290:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228500224:192.168.203.114@tcp:38:kworker.0 00000100:00080000:0.0:1787830910.716640:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830910.716652:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830910.716656:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830910.716667:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7056 00000100:00080000:0.0:1787830910.716678:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830910.716686:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787830910.716716:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830910.716758:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5a40 x1874669228500352/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830910.717153:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228500352:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830910.718264:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830910.718305:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830910/real 1787830910] req@ffff985607df5a40 x1874669228500352/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830935 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830910.718861:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5a40 x1874669228500352/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830935 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830910.718875:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830910.718879:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830910.718884:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228500352:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830910.772937:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830911.668847:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830911.797595:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830912.694463:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830912.822277:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830913.716863:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830913.845959:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830914.740864:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830914.868978:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830915.764860:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787830915.830280:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830915.830294:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787830915.830299:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830915.830381:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 7061 00000100:00080000:3.0:1787830915.830396:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830915.830405:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787830915.830458:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82940 x1874669228500480/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830915.830679:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228500480:192.168.203.114@tcp:250:kworker.0 00000800:00000100:0.0:1787830915.830812:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00080000:3.0:1787830915.831005:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830915.831009:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787830915.831012:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830915.831016:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7061 00000100:00080000:3.0:1787830915.831021:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830915.831027:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830915.831039:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830915.831056:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82d00 x1874669228500608/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830915.831247:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830915/real 1787830915] req@ffff985610f82940 x1874669228500480/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830940 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830915.831267:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82940 x1874669228500480/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830940 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830915.831280:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830915.831284:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830915.831288:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228500480:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787830915.831328:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228500608:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830915.831357:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830915.831371:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830915/real 1787830915] req@ffff985610f82d00 x1874669228500608/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830940 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830915.831383:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82d00 x1874669228500608/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830940 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830915.831391:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830915.831394:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830915.831397:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228500608:192.168.203.114@tcp:38:kworker.0 00000100:00080000:3.0:1787830915.831685:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830915.831712:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787830915.831714:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830915.831719:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7061 00000100:00080000:3.0:1787830915.831724:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830915.831730:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830915.831736:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830915.831752:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f803c0 x1874669228500736/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830915.831959:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228500736:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830915.832116:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830915.832256:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830915/real 1787830915] req@ffff985610f803c0 x1874669228500736/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830940 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830915.832271:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f803c0 x1874669228500736/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830940 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830915.832280:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830915.832283:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830915.832287:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228500736:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830915.893219:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830916.788872:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830916.917087:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830917.812817:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830917.941547:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830918.836873:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830918.965545:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830919.862713:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830919.989084:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830920.884942:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787830920.949753:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830920.949763:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787830920.949767:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830920.949777:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 7066 00000100:00080000:3.0:1787830920.949790:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830920.949798:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787830920.949838:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83840 x1874669228500864/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830920.949857:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830920.949860:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787830920.949862:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830920.949865:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7066 00000100:00080000:3.0:1787830920.949868:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830920.949871:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830920.949880:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830920.949891:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81e00 x1874669228500992/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830920.949899:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830920.949901:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787830920.949903:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830920.949906:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7066 00000100:00080000:3.0:1787830920.949909:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830920.949912:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830920.949916:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830920.949925:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81680 x1874669228501120/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830920.950035:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228500864:192.168.203.114@tcp:250:kworker.0 00000800:00000100:0.0:1787830920.950092:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830920.950122:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830920/real 1787830920] req@ffff985610f83840 x1874669228500864/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830945 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830920.950137:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83840 x1874669228500864/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830945 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830920.950147:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830920.950150:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830920.950154:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228500864:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787830920.950160:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228500992:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830920.950176:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830920.950185:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830920/real 1787830920] req@ffff985610f81e00 x1874669228500992/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830945 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830920.950196:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81e00 x1874669228500992/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830945 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830920.950203:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830920.950206:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830920.950209:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228500992:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787830920.950213:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228501120:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830920.950228:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830920.950239:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830920/real 1787830920] req@ffff985610f81680 x1874669228501120/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830945 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830920.950248:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81680 x1874669228501120/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830945 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830920.950256:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830920.950259:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830920.950262:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228501120:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830921.013478:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830921.908860:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830922.037099:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830922.932811:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830923.062549:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830923.956858:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830924.085247:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830924.980883:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830925.109017:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830926.004852:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787830926.068961:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830926.068977:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830926.068983:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830926.068998:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 7071 00000100:00080000:1.0:1787830926.069012:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830926.069022:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787830926.069210:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a12c0 x1874669228501248/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830926.069251:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830926.069256:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830926.069259:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830926.069265:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7071 00000100:00080000:1.0:1787830926.069273:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830926.069280:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830926.069289:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830926.069301:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2940 x1874669228501376/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830926.069310:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830926.069314:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830926.069315:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830926.069319:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7071 00000100:00080000:1.0:1787830926.069323:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830926.069326:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830926.069330:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830926.069339:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a1a40 x1874669228501504/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830926.069465:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228501248:192.168.203.114@tcp:250:kworker.0 00000800:00000100:0.0:1787830926.069526:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830926.069557:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830926/real 1787830926] req@ffff9856085a12c0 x1874669228501248/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830951 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830926.069574:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a12c0 x1874669228501248/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830951 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830926.069584:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830926.069588:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830926.069591:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228501248:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787830926.069597:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228501376:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830926.069620:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830926.069632:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830926/real 1787830926] req@ffff9856085a2940 x1874669228501376/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830951 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830926.069643:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2940 x1874669228501376/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830951 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830926.069652:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830926.069655:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830926.069658:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228501376:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787830926.069663:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228501504:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787830926.069681:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787830926.069746:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830926/real 1787830926] req@ffff9856085a1a40 x1874669228501504/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830951 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830926.069758:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a1a40 x1874669228501504/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830951 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830926.069765:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830926.069768:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787830926.069770:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228501504:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830926.132837:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830927.028930:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830927.157043:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830928.053187:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830928.184425:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830929.076951:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830929.205001:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830930.100912:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830930.229058:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830931.128260:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787830931.188904:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830931.188918:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830931.188923:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830931.188936:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 7076 00000100:00080000:0.0:1787830931.188951:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830931.188960:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787830931.189008:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6580 x1874669228501632/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830931.189094:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228501632:192.168.203.114@tcp:250:kworker.0 00000800:00000100:3.0:1787830931.189172:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00080000:0.0:1787830931.189208:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830931.189213:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830931.189215:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830931.189220:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7076 00000100:00080000:0.0:1787830931.189224:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830931.189228:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787830931.189240:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830931.189253:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df70c0 x1874669228501760/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830931.189314:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830931/real 1787830931] req@ffff985607df6580 x1874669228501632/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830956 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830931.189336:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6580 x1874669228501632/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830956 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830931.189348:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830931.189353:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830931.189357:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228501632:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787830931.189397:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228501760:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830931.189422:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830931.189436:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830931/real 1787830931] req@ffff985607df70c0 x1874669228501760/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830956 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830931.189449:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df70c0 x1874669228501760/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830956 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830931.189457:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830931.189461:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830931.189464:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228501760:192.168.203.114@tcp:38:kworker.0 00000100:00080000:0.0:1787830931.192068:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830931.192081:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830931.192086:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830931.192099:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7076 00000100:00080000:0.0:1787830931.192111:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830931.192120:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787830931.192136:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830931.192181:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5680 x1874669228501888/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830931.192323:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228501888:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830931.192473:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830931.192508:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830931/real 1787830931] req@ffff985607df5680 x1874669228501888/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830956 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830931.192528:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5680 x1874669228501888/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830956 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830931.192538:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830931.192542:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830931.192546:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228501888:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830931.252892:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830932.149175:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830932.278194:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830933.172961:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830933.302884:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830934.196965:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830934.326989:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830935.221137:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830935.348943:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830936.244819:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787830936.315229:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830936.315243:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830936.315249:0:37:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830936.315264:0:37:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 7081 00000100:00080000:0.0:1787830936.315279:0:37:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830936.315288:0:37:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787830936.315341:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df43c0 x1874669228502016/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830936.315377:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830936.315389:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830936.315391:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830936.315395:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7081 00000100:00080000:0.0:1787830936.315400:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830936.315405:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787830936.315417:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830936.315436:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df61c0 x1874669228502144/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787830936.315440:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228502016:192.168.203.114@tcp:250:kworker.0 00000100:00080000:0.0:1787830936.315459:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830936.315464:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830936.315466:0:37:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830936.315473:0:37:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7081 00000100:00080000:0.0:1787830936.315477:0:37:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830936.315482:0:37:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787830936.315488:0:37:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830936.315501:0:37:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5e00 x1874669228502272/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787830936.315564:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228502144:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787830936.315590:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228502272:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830936.373929:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830937.268850:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830937.398078:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830938.292813:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830938.420951:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830939.316827:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000400:00000100:2.0:1787830939.384052:0:2371:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.114:988 00000400:00000100:2.0:1787830939.384095:0:2371:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.114@tcp at host 192.168.203.114 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:2.0:1787830939.384111:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:2.0:1787830939.384163:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:2.0:1787830939.384171:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000100:00100000:3.0:1787830939.384224:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830936/real 1787830939] req@ffff985607df61c0 x1874669228502144/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830961 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830939.384260:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df61c0 x1874669228502144/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830961 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830939.384270:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830939.384274:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830939.384278:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228502144:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787830939.384285:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830936/real 1787830939] req@ffff985607df5e00 x1874669228502272/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830961 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830939.384296:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5e00 x1874669228502272/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830961 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830939.384302:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830939.384305:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830939.384308:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228502272:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787830939.384314:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830936/real 1787830939] req@ffff985607df43c0 x1874669228502016/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830961 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830939.384325:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df43c0 x1874669228502016/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830961 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830939.384334:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830939.384337:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830939.384340:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228502016:192.168.203.114@tcp:250:kworker.0 00010000:00010000:2.0:1787830939.444884:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830940.340783:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830940.469100:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830941.365004:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787830941.433738:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830941.433747:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830941.433751:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830941.433760:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 7087 00000100:00080000:1.0:1787830941.433770:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830941.433778:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787830941.433823:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a21c0 x1874669228502400/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830941.433854:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830941.433857:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830941.433858:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830941.433862:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7087 00000100:00080000:1.0:1787830941.433866:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830941.433869:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830941.433877:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830941.433944:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228502400:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787830941.433967:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3480 x1874669228502528/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830941.433981:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830941.433984:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830941.433986:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830941.433990:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7087 00000100:00080000:1.0:1787830941.433995:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830941.434000:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830941.434005:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830941.434020:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a30c0 x1874669228502656/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787830941.434045:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830941.434093:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830941/real 1787830941] req@ffff9856085a21c0 x1874669228502400/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830966 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830941.434113:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a21c0 x1874669228502400/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830966 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830941.434125:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830941.434129:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830941.434133:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228502400:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787830941.434164:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228502528:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830941.434190:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830941.434203:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830941/real 1787830941] req@ffff9856085a3480 x1874669228502528/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830966 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830941.434216:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3480 x1874669228502528/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830966 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830941.434225:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830941.434228:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830941.434231:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228502528:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787830941.434237:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228502656:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830941.434257:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830941.434268:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830941/real 1787830941] req@ffff9856085a30c0 x1874669228502656/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830966 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830941.434279:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a30c0 x1874669228502656/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830966 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830941.434287:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830941.434290:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830941.434293:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228502656:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830941.492820:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830942.388851:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830942.517626:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830943.412808:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830943.540888:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830944.436824:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830944.564990:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830945.460823:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830945.589077:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830946.485593:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787830946.549276:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830946.549287:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830946.549290:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830946.549299:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 7092 00000100:00080000:0.0:1787830946.549309:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830946.549316:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787830946.549349:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6d00 x1874669228502784/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830946.549383:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830946.549386:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830946.549388:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830946.549391:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7092 00000100:00080000:0.0:1787830946.549394:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830946.549397:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787830946.549406:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830946.549416:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4f00 x1874669228502912/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830946.549426:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830946.549429:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830946.549430:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830946.549434:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7092 00000100:00080000:0.0:1787830946.549437:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830946.549441:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787830946.549446:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830946.549455:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7840 x1874669228503040/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830946.549482:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228502784:192.168.203.114@tcp:250:kworker.0 00000800:00000100:3.0:1787830946.549586:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830946.549637:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830946/real 1787830946] req@ffff985607df6d00 x1874669228502784/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830971 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830946.549660:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6d00 x1874669228502784/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830971 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830946.549672:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830946.549677:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830946.549682:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228502784:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787830946.549737:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228502912:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830946.549765:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830946.549779:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830946/real 1787830946] req@ffff985607df4f00 x1874669228502912/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830971 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830946.549792:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4f00 x1874669228502912/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830971 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830946.549806:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830946.549809:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830946.549813:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228502912:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787830946.549845:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228503040:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830946.549866:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830946.549876:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830946/real 1787830946] req@ffff985607df7840 x1874669228503040/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830971 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830946.549887:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7840 x1874669228503040/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830971 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830946.549895:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830946.549898:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830946.549901:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228503040:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830946.612860:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830947.508838:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830947.637061:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830948.532815:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830948.660982:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830949.557320:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830949.684976:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830950.581189:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830950.708917:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830951.605251:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787830951.669461:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830951.669471:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830951.669475:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830951.669484:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 7097 00000100:00080000:0.0:1787830951.669495:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830951.669502:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787830951.669551:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df52c0 x1874669228503168/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830951.669641:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228503168:192.168.203.114@tcp:250:kworker.0 00000800:00000100:3.0:1787830951.669739:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00080000:0.0:1787830951.671077:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830951.671084:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830951.671086:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830951.671092:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7097 00000100:00080000:0.0:1787830951.671100:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830951.671107:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787830951.671118:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830951.671143:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7480 x1874669228503296/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830951.675151:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830951/real 1787830951] req@ffff985607df52c0 x1874669228503168/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830976 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830951.675178:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df52c0 x1874669228503168/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830976 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830951.675189:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830951.675193:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830951.675203:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228503168:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787830951.675232:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228503296:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830951.675284:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830951.675299:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830951/real 1787830951] req@ffff985607df7480 x1874669228503296/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830976 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830951.675310:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7480 x1874669228503296/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830976 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830951.675457:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830951.675461:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830951.675464:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228503296:192.168.203.114@tcp:38:kworker.0 00000100:00080000:0.0:1787830951.675644:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830951.675649:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830951.675651:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830951.675656:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7097 00000100:00080000:0.0:1787830951.675662:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830951.675668:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787830951.675680:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830951.675715:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4b40 x1874669228503424/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830951.675989:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228503424:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830951.676018:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830951.676031:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830951/real 1787830951] req@ffff985607df4b40 x1874669228503424/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830976 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830951.676044:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4b40 x1874669228503424/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830976 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830951.676052:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830951.676055:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830951.676058:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228503424:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830951.732862:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830952.628812:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830952.757024:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830953.653432:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830953.781006:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830954.677084:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830954.805371:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830955.701046:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830955.829414:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830956.724860:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787830956.789065:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830956.789080:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830956.789085:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830956.789097:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 7102 00000100:00080000:0.0:1787830956.789110:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830956.789119:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787830956.789164:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6940 x1874669228503552/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830956.789348:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228503552:192.168.203.114@tcp:250:kworker.0 00000800:00000100:3.0:1787830956.789430:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00080000:0.0:1787830956.789486:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830956.789491:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830956.789493:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830956.789498:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7102 00000100:00080000:0.0:1787830956.789503:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830956.789507:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787830956.789520:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830956.789535:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4780 x1874669228503680/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830956.789584:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830956.789588:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830956.789591:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830956.789596:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7102 00000100:00080000:0.0:1787830956.789601:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830956.789606:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787830956.789612:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830956.789639:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830956/real 1787830956] req@ffff985607df6940 x1874669228503552/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830981 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830956.789666:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6940 x1874669228503552/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830981 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830956.789676:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830956.789681:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830956.789684:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228503552:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787830956.789782:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228503680:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830956.789865:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830956.789884:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830956/real 1787830956] req@ffff985607df4780 x1874669228503680/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830981 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830956.789895:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4780 x1874669228503680/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830981 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830956.789903:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830956.789907:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830956.789910:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228503680:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787830956.789934:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7c00 x1874669228503808/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830956.790099:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228503808:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830956.790118:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830956.790127:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830956/real 1787830956] req@ffff985607df7c00 x1874669228503808/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830981 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830956.790134:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7c00 x1874669228503808/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830981 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830956.790144:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830956.790146:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830956.790148:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228503808:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830956.853571:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830957.748865:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830957.877134:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830958.772855:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830958.901517:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830959.796856:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830959.924994:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830960.820936:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830960.951221:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830961.844852:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787830961.910229:0:93375:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830961.910238:0:93375:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787830961.910242:0:93375:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830961.910253:0:93375:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 7107 00000100:00080000:3.0:1787830961.910262:0:93375:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830961.910269:0:93375:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787830961.910301:0:93375:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f812c0 x1874669228503936/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830961.910330:0:93375:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830961.910333:0:93375:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787830961.910335:0:93375:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830961.910338:0:93375:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7107 00000100:00080000:3.0:1787830961.910342:0:93375:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830961.910345:0:93375:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830961.910354:0:93375:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830961.910364:0:93375:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80b40 x1874669228504064/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830961.910374:0:93375:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830961.910377:0:93375:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787830961.910380:0:93375:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830961.910384:0:93375:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7107 00000100:00080000:3.0:1787830961.910387:0:93375:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830961.910390:0:93375:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830961.910393:0:93375:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830961.910402:0:93375:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80000 x1874669228504192/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787830961.910816:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228503936:192.168.203.114@tcp:250:kworker.0 00000800:00000100:2.0:1787830961.910955:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830961.911033:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830961/real 1787830961] req@ffff985610f812c0 x1874669228503936/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830986 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830961.911052:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f812c0 x1874669228503936/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830986 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830961.911063:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830961.911066:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830961.911070:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228503936:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787830961.911076:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228504064:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787830961.911095:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830961.911104:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830961/real 1787830961] req@ffff985610f80b40 x1874669228504064/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830986 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830961.911111:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80b40 x1874669228504064/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830986 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830961.911116:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830961.911118:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830961.911121:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228504064:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787830961.911124:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228504192:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787830961.911134:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830961.911139:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830961/real 1787830961] req@ffff985610f80000 x1874669228504192/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830986 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830961.911145:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80000 x1874669228504192/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830986 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830961.911150:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830961.911151:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830961.911153:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228504192:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830961.972863:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830962.868802:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830962.996875:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830963.892816:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830964.023710:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830964.917270:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830965.045607:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830965.940793:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830966.068993:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830966.964826:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787830967.028850:0:93375:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830967.028864:0:93375:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787830967.028869:0:93375:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830967.028883:0:93375:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 7112 00000100:00080000:3.0:1787830967.028898:0:93375:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830967.028909:0:93375:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787830967.028959:0:93375:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81a40 x1874669228504320/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830967.028998:0:93375:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830967.029002:0:93375:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787830967.029004:0:93375:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830967.029008:0:93375:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7112 00000100:00080000:3.0:1787830967.029012:0:93375:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830967.029016:0:93375:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830967.029028:0:93375:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830967.029043:0:93375:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f830c0 x1874669228504448/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830967.029062:0:93375:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830967.029067:0:93375:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:2.0:1787830967.029068:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228504320:192.168.203.114@tcp:250:kworker.0 00000100:00080000:3.0:1787830967.029069:0:93375:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830967.029078:0:93375:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7112 00000100:00080000:3.0:1787830967.029083:0:93375:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830967.029087:0:93375:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830967.029091:0:93375:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830967.029104:0:93375:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80780 x1874669228504576/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787830967.029156:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830967.029334:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830967/real 1787830967] req@ffff985610f81a40 x1874669228504320/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830992 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830967.029355:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81a40 x1874669228504320/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830992 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830967.029366:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830967.029371:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830967.029374:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228504320:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787830967.029417:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228504448:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787830967.029443:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830967.029456:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830967/real 1787830967] req@ffff985610f830c0 x1874669228504448/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830992 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830967.029467:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f830c0 x1874669228504448/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830992 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830967.029474:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830967.029477:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830967.029480:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228504448:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787830967.029485:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228504576:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787830967.029503:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830967.029514:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830967/real 1787830967] req@ffff985610f80780 x1874669228504576/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830992 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830967.029524:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80780 x1874669228504576/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830992 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830967.029532:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830967.029535:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830967.029538:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228504576:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830967.093123:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830967.989082:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830968.116965:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830969.012882:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830969.141772:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830970.038090:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830970.165573:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830971.060862:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830971.188897:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830972.085606:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787830972.149525:0:93375:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830972.149538:0:93375:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787830972.149543:0:93375:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830972.149556:0:93375:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 7117 00000100:00080000:3.0:1787830972.149572:0:93375:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830972.149581:0:93375:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787830972.149633:0:93375:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83c00 x1874669228504704/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830972.149672:0:93375:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830972.149676:0:93375:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787830972.149679:0:93375:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830972.149683:0:93375:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7117 00000100:00080000:3.0:1787830972.149687:0:93375:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830972.149724:0:93375:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830972.149736:0:93375:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830972.149752:0:93375:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80780 x1874669228504832/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830972.149762:0:93375:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830972.149765:0:93375:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787830972.149768:0:93375:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830972.149771:0:93375:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7117 00000100:00080000:3.0:1787830972.149776:0:93375:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830972.149780:0:93375:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787830972.149785:0:93375:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830972.149795:0:93375:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f830c0 x1874669228504960/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787830972.150210:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228504704:192.168.203.114@tcp:250:kworker.0 00000800:00000100:2.0:1787830972.150288:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830972.150326:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830972/real 1787830972] req@ffff985610f83c00 x1874669228504704/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830997 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830972.150342:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83c00 x1874669228504704/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830997 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830972.150358:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830972.150370:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830972.150374:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228504704:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787830972.150382:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228504832:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787830972.150404:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830972.150415:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830972/real 1787830972] req@ffff985610f80780 x1874669228504832/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830997 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830972.150426:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80780 x1874669228504832/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830997 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830972.150433:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830972.150436:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830972.150439:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228504832:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787830972.150444:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228504960:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787830972.150463:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787830972.150474:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830972/real 1787830972] req@ffff985610f830c0 x1874669228504960/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830997 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830972.150486:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f830c0 x1874669228504960/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830997 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830972.150499:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830972.150503:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787830972.150506:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228504960:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830972.212944:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830973.108999:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830973.236847:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830974.136849:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830974.260850:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830975.156946:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830975.284921:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830976.181107:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830976.308917:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830977.204949:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787830977.269000:0:93375:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830977.269011:0:93375:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830977.269015:0:93375:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830977.269025:0:93375:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 7122 00000100:00080000:2.0:1787830977.269034:0:93375:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830977.269042:0:93375:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787830977.269077:0:93375:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8f00 x1874669228505088/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830977.269103:0:93375:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830977.269106:0:93375:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830977.269108:0:93375:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830977.269111:0:93375:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7122 00000100:00080000:2.0:1787830977.269115:0:93375:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830977.269119:0:93375:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830977.269128:0:93375:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830977.269139:0:93375:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140da1c0 x1874669228505216/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830977.269148:0:93375:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830977.269151:0:93375:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830977.269153:0:93375:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830977.269156:0:93375:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7122 00000100:00080000:2.0:1787830977.269159:0:93375:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830977.269162:0:93375:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830977.269166:0:93375:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830977.269175:0:93375:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140dbc00 x1874669228505344/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830977.269303:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228505088:192.168.203.114@tcp:250:kworker.0 00000800:00000100:3.0:1787830977.269376:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830977.269407:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830977/real 1787830977] req@ffff9856140d8f00 x1874669228505088/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787831002 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830977.269430:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8f00 x1874669228505088/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787831002 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830977.269439:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830977.269455:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830977.269459:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228505088:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787830977.269465:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228505216:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830977.269487:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830977.269499:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830977/real 1787830977] req@ffff9856140da1c0 x1874669228505216/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831002 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830977.269509:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140da1c0 x1874669228505216/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831002 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830977.269515:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830977.269519:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830977.269521:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228505216:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787830977.269527:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228505344:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830977.269543:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830977.269554:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830977/real 1787830977] req@ffff9856140dbc00 x1874669228505344/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831002 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830977.269564:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140dbc00 x1874669228505344/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831002 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830977.269577:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830977.269580:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830977.269583:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228505344:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830977.335058:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830978.229202:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830978.357594:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830979.252822:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830979.382769:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830980.276894:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830980.405005:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830981.302032:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830981.429494:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830982.324850:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787830982.390793:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830982.390809:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830982.390814:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830982.390828:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 7128 00000100:00080000:0.0:1787830982.390841:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830982.390850:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787830982.390902:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4000 x1874669228505472/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830982.390945:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830982.390949:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830982.390951:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830982.390955:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7128 00000100:00080000:0.0:1787830982.390958:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830982.390963:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787830982.390974:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830982.390987:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7c00 x1874669228505600/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830982.390996:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830982.391000:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830982.391001:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830982.391005:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7128 00000100:00080000:0.0:1787830982.391009:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830982.391013:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787830982.391017:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830982.391028:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4780 x1874669228505728/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830982.391134:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228505472:192.168.203.114@tcp:250:kworker.0 00000800:00000100:3.0:1787830982.391212:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830982.391256:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830982/real 1787830982] req@ffff985607df4000 x1874669228505472/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787831007 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830982.391275:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4000 x1874669228505472/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787831007 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830982.391286:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830982.391299:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830982.391304:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228505472:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787830982.391311:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228505600:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830982.391335:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830982.391349:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830982/real 1787830982] req@ffff985607df7c00 x1874669228505600/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831007 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830982.391361:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7c00 x1874669228505600/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831007 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830982.391369:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830982.391372:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830982.391376:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228505600:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787830982.391381:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228505728:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830982.391410:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830982.391421:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830982/real 1787830982] req@ffff985607df4780 x1874669228505728/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831007 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830982.391432:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4780 x1874669228505728/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831007 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830982.391440:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830982.391443:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830982.391446:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228505728:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830982.452907:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830983.348859:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830983.477168:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830984.372856:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830984.500987:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830985.396959:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830985.527935:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830986.421453:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830986.548925:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830987.444821:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787830987.509106:0:93375:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830987.509119:0:93375:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830987.509124:0:93375:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830987.509137:0:93375:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 7133 00000100:00080000:2.0:1787830987.509160:0:93375:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830987.509169:0:93375:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787830987.509218:0:93375:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db840 x1874669228505856/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830987.509264:0:93375:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830987.509268:0:93375:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830987.509269:0:93375:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830987.509274:0:93375:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7133 00000100:00080000:2.0:1787830987.509278:0:93375:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830987.509283:0:93375:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830987.509294:0:93375:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830987.509310:0:93375:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140dbc00 x1874669228505984/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830987.509319:0:93375:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830987.509323:0:93375:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830987.509324:0:93375:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830987.509328:0:93375:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7133 00000100:00080000:2.0:1787830987.509332:0:93375:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830987.509335:0:93375:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830987.509341:0:93375:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830987.509351:0:93375:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140da1c0 x1874669228506112/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830987.509399:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228505856:192.168.203.114@tcp:250:kworker.0 00000800:00000100:3.0:1787830987.509470:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830987.509505:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830987/real 1787830987] req@ffff9856140db840 x1874669228505856/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787831012 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830987.509520:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db840 x1874669228505856/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787831012 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830987.509530:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830987.509534:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830987.509538:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228505856:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787830987.509545:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228505984:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830987.509622:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830987.509636:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830987/real 1787830987] req@ffff9856140dbc00 x1874669228505984/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831012 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830987.509934:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140dbc00 x1874669228505984/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831012 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830987.509941:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830987.509945:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830987.509948:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228505984:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787830987.509954:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228506112:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830987.509978:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830987.509990:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830987/real 1787830987] req@ffff9856140da1c0 x1874669228506112/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831012 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830987.510001:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140da1c0 x1874669228506112/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831012 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830987.510008:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830987.510011:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830987.510014:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228506112:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830987.575411:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830988.468889:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830988.603447:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830989.493081:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830989.621408:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830990.517381:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830990.647056:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830991.541760:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830991.671257:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830992.564831:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787830992.629321:0:93375:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830992.629332:0:93375:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830992.629336:0:93375:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830992.629345:0:93375:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 7138 00000100:00080000:2.0:1787830992.629356:0:93375:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830992.629364:0:93375:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787830992.629401:0:93375:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8f00 x1874669228506240/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830992.629430:0:93375:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830992.629435:0:93375:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830992.629436:0:93375:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830992.629440:0:93375:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7138 00000100:00080000:2.0:1787830992.629444:0:93375:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830992.629448:0:93375:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830992.629457:0:93375:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830992.629470:0:93375:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140dad00 x1874669228506368/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830992.629481:0:93375:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830992.629484:0:93375:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830992.629486:0:93375:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830992.629490:0:93375:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7138 00000100:00080000:2.0:1787830992.629493:0:93375:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830992.629497:0:93375:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787830992.629502:0:93375:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830992.629513:0:93375:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db0c0 x1874669228506496/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830992.629514:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228506240:192.168.203.114@tcp:250:kworker.0 00000800:00000100:3.0:1787830992.629580:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830992.629610:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830992/real 1787830992] req@ffff9856140d8f00 x1874669228506240/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787831017 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830992.629628:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8f00 x1874669228506240/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787831017 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830992.629638:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830992.629642:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830992.629646:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228506240:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787830992.629653:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228506368:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830992.629672:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830992.629683:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830992/real 1787830992] req@ffff9856140dad00 x1874669228506368/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831017 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830992.629936:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140dad00 x1874669228506368/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831017 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830992.629945:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830992.629948:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830992.629951:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228506368:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787830992.629975:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228506496:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830992.629996:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830992.630006:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830992/real 1787830992] req@ffff9856140db0c0 x1874669228506496/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831017 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830992.630017:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db0c0 x1874669228506496/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831017 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830992.630025:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830992.630028:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830992.630032:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228506496:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830992.692829:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830993.588798:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830993.717055:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830994.612922:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830994.746149:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830995.636838:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830995.765258:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830996.660846:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787830996.788969:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830997.686252:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787830997.748886:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830997.748899:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830997.748905:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830997.748918:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 7143 00000100:00080000:1.0:1787830997.748932:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830997.748942:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787830997.748990:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0f00 x1874669228506624/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830997.749027:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830997.749031:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830997.749033:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830997.749037:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7143 00000100:00080000:1.0:1787830997.749041:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830997.749045:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830997.749057:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830997.749072:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a30c0 x1874669228506752/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830997.749081:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830997.749084:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830997.749086:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830997.749089:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7143 00000100:00080000:1.0:1787830997.749093:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830997.749097:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787830997.749103:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830997.749112:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3480 x1874669228506880/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830997.749228:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228506624:192.168.203.114@tcp:250:kworker.0 00000800:00000100:3.0:1787830997.749311:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830997.749355:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830997/real 1787830997] req@ffff9856085a0f00 x1874669228506624/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787831022 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830997.749375:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0f00 x1874669228506624/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787831022 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830997.749387:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830997.749392:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830997.749397:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228506624:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787830997.749405:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228506752:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830997.749431:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830997.749444:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830997/real 1787830997] req@ffff9856085a30c0 x1874669228506752/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831022 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830997.749454:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a30c0 x1874669228506752/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831022 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830997.749461:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830997.749464:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830997.749467:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228506752:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787830997.749472:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228506880:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787830997.749489:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787830997.749499:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830997/real 1787830997] req@ffff9856085a3480 x1874669228506880/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831022 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830997.749509:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3480 x1874669228506880/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831022 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830997.749516:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830997.749519:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787830997.749522:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228506880:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787830997.812883:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830998.708832:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830998.836968:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830999.733509:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787830999.861037:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787831000.757173:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787831000.885029:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787831001.781103:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787831001.913116:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787831002.805216:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787831002.868922:0:93375:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787831002.868947:0:93375:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787831002.868952:0:93375:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787831002.868968:0:93375:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 7148 00000100:00080000:0.0:1787831002.868983:0:93375:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787831002.868992:0:93375:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787831002.869044:0:93375:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6940 x1874669228507008/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787831002.869086:0:93375:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787831002.869090:0:93375:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787831002.869092:0:93375:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787831002.869096:0:93375:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7148 00000100:00080000:0.0:1787831002.869099:0:93375:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787831002.869104:0:93375:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787831002.869115:0:93375:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787831002.869128:0:93375:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4b40 x1874669228507136/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787831002.869137:0:93375:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787831002.869140:0:93375:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787831002.869142:0:93375:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787831002.869145:0:93375:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7148 00000100:00080000:0.0:1787831002.869158:0:93375:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787831002.869163:0:93375:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787831002.869168:0:93375:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787831002.869177:0:93375:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7480 x1874669228507264/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787831002.869333:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228507008:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787831002.869442:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228507136:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787831002.869471:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228507264:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787831002.933904:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787831003.832484:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787831003.960095:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787831004.853054:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787831004.983513:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787831005.876840:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000400:00000100:0.0:1787831005.941271:0:2372:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.114:988 00000400:00000100:0.0:1787831005.941332:0:2372:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.114@tcp at host 192.168.203.114 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:0.0:1787831005.941354:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000100:00100000:3.0:1787831005.943187:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831002/real 1787831005] req@ffff985607df6940 x1874669228507008/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787831027 ref 2 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787831005.943241:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6940 x1874669228507008/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787831027 ref 2 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787831005.943257:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787831005.943262:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787831005.943266:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228507008:192.168.203.114@tcp:250:kworker.0 00000800:00000100:0.0:1787831005.943879:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:0.0:1787831005.943910:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000100:00100000:1.0:1787831005.945484:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831002/real 1787831005] req@ffff985607df4b40 x1874669228507136/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831027 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787831005.945551:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4b40 x1874669228507136/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831027 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787831005.945629:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787831005.945636:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787831005.945640:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228507136:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787831005.945651:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831002/real 1787831005] req@ffff985607df7480 x1874669228507264/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831027 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787831005.945668:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7480 x1874669228507264/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831027 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787831005.945676:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787831005.945680:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787831005.945683:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228507264:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787831006.005015:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787831006.900934:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787831007.030979:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787831007.926053:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787831007.989306:0:93375:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787831007.989316:0:93375:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787831007.989320:0:93375:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787831007.989330:0:93375:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 7153 00000100:00080000:0.0:1787831007.989341:0:93375:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787831007.989348:0:93375:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787831007.989382:0:93375:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6940 x1874669228507392/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787831007.989407:0:93375:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787831007.989411:0:93375:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787831007.989412:0:93375:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787831007.989416:0:93375:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7153 00000100:00080000:0.0:1787831007.989419:0:93375:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787831007.989423:0:93375:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787831007.989431:0:93375:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787831007.989456:0:93375:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df52c0 x1874669228507520/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787831007.989466:0:93375:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787831007.989469:0:93375:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787831007.989471:0:93375:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787831007.989474:0:93375:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7153 00000100:00080000:0.0:1787831007.989477:0:93375:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787831007.989481:0:93375:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787831007.989486:0:93375:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787831007.989495:0:93375:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df7840 x1874669228507648/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787831007.990831:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228507392:192.168.203.114@tcp:250:kworker.0 00000800:00000100:1.0:1787831007.991597:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787831007.991643:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831007/real 1787831007] req@ffff985607df6940 x1874669228507392/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787831032 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787831007.991661:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6940 x1874669228507392/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787831032 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787831007.991680:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787831007.991685:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787831007.991689:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228507392:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787831007.991724:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228507520:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787831007.991751:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787831007.991763:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831007/real 1787831007] req@ffff985607df52c0 x1874669228507520/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831032 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787831007.991775:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df52c0 x1874669228507520/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831032 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787831007.991784:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787831007.991787:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787831007.991790:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228507520:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787831007.991796:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228507648:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787831007.991814:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787831007.991824:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831007/real 1787831007] req@ffff985607df7840 x1874669228507648/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831032 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787831007.991835:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df7840 x1874669228507648/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831032 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787831007.994492:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787831007.994501:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787831007.994506:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228507648:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787831008.052847:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787831008.948794:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787831009.077366:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787831009.973312:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787831010.101311:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787831010.996891:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787831011.126021:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787831012.020842:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787831012.152962:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787831013.045333:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787831013.110651:0:93375:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787831013.110664:0:93375:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787831013.110670:0:93375:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787831013.110682:0:93375:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 7158 00000100:00080000:2.0:1787831013.110725:0:93375:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787831013.110734:0:93375:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787831013.110784:0:93375:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140da940 x1874669228507776/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787831013.110829:0:93375:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787831013.110833:0:93375:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787831013.110834:0:93375:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787831013.110838:0:93375:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7158 00000100:00080000:2.0:1787831013.110842:0:93375:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787831013.110846:0:93375:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787831013.110857:0:93375:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787831013.110870:0:93375:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d9a40 x1874669228507904/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787831013.110879:0:93375:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787831013.110882:0:93375:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787831013.110883:0:93375:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787831013.110887:0:93375:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7158 00000100:00080000:2.0:1787831013.110891:0:93375:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787831013.110896:0:93375:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787831013.110900:0:93375:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787831013.110909:0:93375:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8b40 x1874669228508032/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787831013.111188:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228507776:192.168.203.114@tcp:250:kworker.0 00000800:00000100:1.0:1787831013.111268:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787831013.111311:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831013/real 1787831013] req@ffff9856140da940 x1874669228507776/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787831038 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787831013.111330:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140da940 x1874669228507776/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787831038 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787831013.111341:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787831013.111345:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787831013.111348:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228507776:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787831013.111368:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228507904:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787831013.111393:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787831013.111404:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831013/real 1787831013] req@ffff9856140d9a40 x1874669228507904/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831038 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787831013.111416:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d9a40 x1874669228507904/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831038 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787831013.111424:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787831013.111428:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787831013.111431:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228507904:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787831013.111437:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228508032:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787831013.111456:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787831013.111468:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831013/real 1787831013] req@ffff9856140d8b40 x1874669228508032/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831038 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787831013.111479:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8b40 x1874669228508032/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831038 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787831013.111488:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787831013.111491:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787831013.111494:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228508032:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787831013.172940:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787831014.068926:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787831014.197147:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787831015.092933:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787831015.224155:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787831016.119675:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787831016.248840:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787831017.141554:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787831017.272717:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787831018.166197:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787831018.228935:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787831018.228950:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787831018.228955:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787831018.228968:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 7163 00000100:00080000:2.0:1787831018.228983:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787831018.228991:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787831018.229259:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db480 x1874669228508160/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787831018.229402:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228508160:192.168.203.114@tcp:250:kworker.0 00000800:00000100:1.0:1787831018.229484:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00080000:2.0:1787831018.229521:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787831018.229525:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787831018.229528:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787831018.229532:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7163 00000100:00080000:2.0:1787831018.229537:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787831018.229541:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787831018.229553:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787831018.229567:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d83c0 x1874669228508288/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787831018.229753:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831018/real 1787831018] req@ffff9856140db480 x1874669228508160/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787831043 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787831018.229772:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db480 x1874669228508160/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787831043 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787831018.229782:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787831018.229787:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787831018.229791:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228508160:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787831018.229821:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228508288:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787831018.234112:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787831018.234163:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831018/real 1787831018] req@ffff9856140d83c0 x1874669228508288/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831043 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787831018.234187:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d83c0 x1874669228508288/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831043 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787831018.234203:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787831018.234208:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787831018.234213:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228508288:192.168.203.114@tcp:38:kworker.0 00000100:00080000:2.0:1787831018.239017:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787831018.239031:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787831018.239036:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787831018.239058:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7163 00000100:00080000:2.0:1787831018.239071:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787831018.239081:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787831018.239098:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787831018.239383:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d9680 x1874669228508416/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787831018.239864:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228508416:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787831018.239953:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787831018.239989:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831018/real 1787831018] req@ffff9856140d9680 x1874669228508416/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831043 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787831018.240007:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d9680 x1874669228508416/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831043 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787831018.240018:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787831018.240023:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787831018.240027:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228508416:192.168.203.114@tcp:38:kworker.0 00010000:00010000:0.0:1787831018.293131:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787831019.188850:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787831019.316934:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787831020.212816:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787831020.344297:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787831021.236840:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787831021.301244:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787831021.301254:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787831021.301258:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787831021.301268:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 7168 00000100:00080000:1.0:1787831021.301278:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787831021.301286:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787831021.301334:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a21c0 x1874669228508544/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787831021.301367:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787831021.301372:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787831021.301374:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787831021.301377:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7168 00000100:00080000:1.0:1787831021.301381:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787831021.301385:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787831021.301394:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787831021.301407:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a1a40 x1874669228508672/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787831021.301416:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787831021.301419:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787831021.301421:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787831021.301424:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7169 00000100:00080000:1.0:1787831021.301428:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787831021.301431:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787831021.301435:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787831021.301445:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2940 x1874669228508800/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787831021.301556:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228508544:192.168.203.114@tcp:250:kworker.0 00000800:00000100:3.0:1787831021.301615:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787831021.301650:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831021/real 1787831021] req@ffff9856085a21c0 x1874669228508544/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787831046 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787831021.301677:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a21c0 x1874669228508544/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787831046 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787831021.301687:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787831021.301713:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787831021.301717:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228508544:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787831021.301724:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228508672:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787831021.301746:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787831021.301758:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831021/real 1787831021] req@ffff9856085a1a40 x1874669228508672/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831046 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787831021.301770:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a1a40 x1874669228508672/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831046 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787831021.301778:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787831021.301781:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787831021.301783:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228508672:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787831021.301788:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228508800:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787831021.301809:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787831021.301820:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831021/real 1787831021] req@ffff9856085a2940 x1874669228508800/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831046 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787831021.301831:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2940 x1874669228508800/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831046 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787831021.301839:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787831021.301842:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787831021.301845:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228508800:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787831021.364901:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787831022.260827:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787831022.389533:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787831023.284806:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787831023.414045:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787831024.308837:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787831024.438985:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787831025.334597:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787831025.460980:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787831026.357009:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787831026.421110:0:93375:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787831026.421120:0:93375:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787831026.421125:0:93375:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787831026.421135:0:93375:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 7172 00000100:00080000:0.0:1787831026.421146:0:93375:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787831026.421153:0:93375:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787831026.421186:0:93375:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df4f00 x1874669228508928/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787831026.421216:0:93375:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787831026.421220:0:93375:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787831026.421221:0:93375:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787831026.421225:0:93375:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7172 00000100:00080000:0.0:1787831026.421228:0:93375:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787831026.421232:0:93375:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787831026.421240:0:93375:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787831026.421251:0:93375:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6d00 x1874669228509056/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787831026.421266:0:93375:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787831026.421268:0:93375:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787831026.421270:0:93375:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787831026.421274:0:93375:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7172 00000100:00080000:0.0:1787831026.421277:0:93375:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787831026.421281:0:93375:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787831026.421285:0:93375:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787831026.421294:0:93375:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df43c0 x1874669228509184/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787831026.421648:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df4f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228508928:192.168.203.114@tcp:250:kworker.0 00000800:00000100:3.0:1787831026.421735:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787831026.421771:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831026/real 1787831026] req@ffff985607df4f00 x1874669228508928/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787831051 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787831026.421787:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df4f00 x1874669228508928/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787831051 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787831026.421797:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787831026.421801:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787831026.421805:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df4f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228508928:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787831026.421812:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228509056:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787831026.421835:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787831026.421847:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831026/real 1787831026] req@ffff985607df6d00 x1874669228509056/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831051 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787831026.421858:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6d00 x1874669228509056/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831051 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787831026.421865:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787831026.421867:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787831026.421870:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228509056:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787831026.421875:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228509184:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787831026.421906:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787831026.421917:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831026/real 1787831026] req@ffff985607df43c0 x1874669228509184/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831051 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787831026.421928:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df43c0 x1874669228509184/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831051 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787831026.421934:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787831026.421937:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787831026.421940:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228509184:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787831026.484891:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787831027.380818:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787831027.508932:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787831028.404868:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787831028.535051:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787831029.429344:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787831029.556932:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787831030.452812:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787831030.582739:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787831031.477108:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787831031.541344:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787831031.541355:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787831031.541360:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787831031.541370:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 7177 00000100:00080000:3.0:1787831031.541382:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787831031.541390:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787831031.541438:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81a40 x1874669228509312/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787831031.541465:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787831031.541468:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787831031.541470:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787831031.541472:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7177 00000100:00080000:3.0:1787831031.541475:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787831031.541477:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787831031.541486:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787831031.541494:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80000 x1874669228509440/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787831031.541500:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787831031.541503:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787831031.541505:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787831031.541507:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7177 00000100:00080000:3.0:1787831031.541508:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787831031.541510:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787831031.541514:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787831031.541523:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80b40 x1874669228509568/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787831031.542107:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228509312:192.168.203.114@tcp:250:kworker.0 00000800:00000100:1.0:1787831031.542179:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787831031.542215:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831031/real 1787831031] req@ffff985610f81a40 x1874669228509312/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787831056 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787831031.542240:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81a40 x1874669228509312/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787831056 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787831031.542250:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787831031.542254:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787831031.542258:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228509312:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787831031.542266:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228509440:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787831031.542287:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787831031.542299:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831031/real 1787831031] req@ffff985610f80000 x1874669228509440/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831056 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787831031.542310:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80000 x1874669228509440/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831056 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787831031.542318:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787831031.542321:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787831031.542324:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228509440:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787831031.542330:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228509568:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787831031.542350:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787831031.542360:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831031/real 1787831031] req@ffff985610f80b40 x1874669228509568/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831056 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787831031.542370:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80b40 x1874669228509568/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831056 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787831031.542382:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787831031.542386:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787831031.542389:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228509568:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787831031.605128:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831032.500883:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787831032.629079:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787831033.526672:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787831033.652845:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787831034.548913:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787831034.676961:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787831035.572804:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787831035.701959:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787831036.597124:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787831036.660803:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787831036.660812:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787831036.660817:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787831036.660826:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 7182 00000100:00080000:3.0:1787831036.660836:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787831036.660843:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787831036.660882:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f812c0 x1874669228509696/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787831036.660922:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787831036.660926:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787831036.660927:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787831036.660931:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7182 00000100:00080000:3.0:1787831036.660934:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787831036.660937:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787831036.660946:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787831036.660956:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81680 x1874669228509824/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787831036.660965:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787831036.660967:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787831036.660969:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787831036.660972:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7182 00000100:00080000:3.0:1787831036.660974:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787831036.660977:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787831036.660981:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787831036.660990:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81e00 x1874669228509952/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787831036.661556:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228509696:192.168.203.114@tcp:250:kworker.0 00000800:00000100:2.0:1787831036.661617:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787831036.661645:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831036/real 1787831036] req@ffff985610f812c0 x1874669228509696/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787831061 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787831036.661666:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f812c0 x1874669228509696/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787831061 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787831036.661675:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787831036.661680:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787831036.661684:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228509696:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787831036.661720:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228509824:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787831036.661745:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787831036.661759:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831036/real 1787831036] req@ffff985610f81680 x1874669228509824/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831061 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787831036.661770:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81680 x1874669228509824/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831061 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787831036.661779:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787831036.661782:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787831036.661786:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228509824:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787831036.661795:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228509952:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787831036.661814:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787831036.661829:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831036/real 1787831036] req@ffff985610f81e00 x1874669228509952/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831061 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787831036.661842:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81e00 x1874669228509952/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831061 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787831036.661855:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787831036.661859:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787831036.661863:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228509952:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787831036.724878:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787831037.620817:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787831037.749070:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787831038.644819:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787831038.772857:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787831039.668820:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787831039.797210:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787831040.693739:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787831040.820883:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787831041.716809:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787831041.780891:0:93375:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787831041.780904:0:93375:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787831041.780909:0:93375:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787831041.780921:0:93375:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 7187 00000100:00080000:2.0:1787831041.780935:0:93375:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787831041.780944:0:93375:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787831041.780990:0:93375:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8780 x1874669228510080/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787831041.781090:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228510080:192.168.203.114@tcp:250:kworker.0 00000800:00000100:1.0:1787831041.781183:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00080000:2.0:1787831041.781219:0:93375:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787831041.781223:0:93375:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787831041.781225:0:93375:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787831041.781238:0:93375:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7187 00000100:00080000:2.0:1787831041.781243:0:93375:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787831041.781247:0:93375:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787831041.781265:0:93375:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787831041.781282:0:93375:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d9680 x1874669228510208/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787831041.781616:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831041/real 1787831041] req@ffff9856140d8780 x1874669228510080/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787831066 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787831041.781637:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8780 x1874669228510080/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787831066 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787831041.781648:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787831041.781653:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787831041.781657:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228510080:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787831041.781683:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228510208:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787831041.781768:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787831041.781782:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831041/real 1787831041] req@ffff9856140d9680 x1874669228510208/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831066 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787831041.781794:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d9680 x1874669228510208/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831066 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787831041.781803:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787831041.781806:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787831041.781809:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228510208:192.168.203.114@tcp:38:kworker.0 00000100:00080000:2.0:1787831041.782000:0:93375:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787831041.782005:0:93375:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787831041.782007:0:93375:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787831041.782012:0:93375:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7187 00000100:00080000:2.0:1787831041.782016:0:93375:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787831041.782021:0:93375:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787831041.782027:0:93375:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787831041.782043:0:93375:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d83c0 x1874669228510336/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787831041.782084:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228510336:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787831041.782111:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787831041.782210:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831041/real 1787831041] req@ffff9856140d83c0 x1874669228510336/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831066 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787831041.782222:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d83c0 x1874669228510336/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831066 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787831041.782230:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787831041.782234:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787831041.782237:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228510336:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787831041.844934:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787831042.740908:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787831042.868957:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787831043.766189:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787831043.892844:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787831044.788837:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787831044.916911:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787831045.816729:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787831045.942206:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787831046.836863:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787831046.900897:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787831046.900907:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787831046.900911:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787831046.900921:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 7192 00000100:00080000:3.0:1787831046.900931:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787831046.900939:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787831046.900976:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83840 x1874669228510464/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787831046.901006:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787831046.901010:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787831046.901012:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787831046.901015:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7192 00000100:00080000:3.0:1787831046.901019:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787831046.901022:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787831046.901031:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787831046.901042:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f803c0 x1874669228510592/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787831046.901051:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787831046.901054:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787831046.901056:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787831046.901058:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7192 00000100:00080000:3.0:1787831046.901062:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787831046.901066:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787831046.901070:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787831046.901078:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82d00 x1874669228510720/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787831046.901172:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228510464:192.168.203.114@tcp:250:kworker.0 00000800:00000100:1.0:1787831046.901228:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787831046.901256:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831046/real 1787831046] req@ffff985610f83840 x1874669228510464/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787831071 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787831046.901271:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83840 x1874669228510464/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787831071 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787831046.901280:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787831046.901284:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787831046.901288:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228510464:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787831046.901293:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228510592:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787831046.901313:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787831046.901326:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831046/real 1787831046] req@ffff985610f803c0 x1874669228510592/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831071 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787831046.901337:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f803c0 x1874669228510592/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831071 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787831046.901343:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787831046.901346:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787831046.901349:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228510592:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787831046.901354:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228510720:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787831046.901373:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787831046.901382:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831046/real 1787831046] req@ffff985610f82d00 x1874669228510720/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831071 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787831046.901399:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82d00 x1874669228510720/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831071 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787831046.901407:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787831046.901410:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787831046.901412:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228510720:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787831046.966748:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787831047.860806:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787831047.989784:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787831048.887318:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787831049.013976:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787831049.909459:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787831050.037720:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787831050.934490:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787831051.060925:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787831051.956850:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787831052.021035:0:93375:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787831052.021044:0:93375:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787831052.021048:0:93375:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787831052.021058:0:93375:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 7197 00000100:00080000:1.0:1787831052.021068:0:93375:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787831052.021075:0:93375:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787831052.021181:0:93375:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a12c0 x1874669228510848/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787831052.021213:0:93375:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787831052.021217:0:93375:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787831052.021219:0:93375:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787831052.021222:0:93375:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7197 00000100:00080000:1.0:1787831052.021227:0:93375:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787831052.021231:0:93375:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787831052.021240:0:93375:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787831052.021252:0:93375:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2580 x1874669228510976/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787831052.021339:0:93375:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787831052.021342:0:93375:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787831052.021344:0:93375:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787831052.021348:0:93375:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7197 00000100:00080000:1.0:1787831052.021351:0:93375:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787831052.021355:0:93375:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787831052.021489:0:93375:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787831052.021503:0:93375:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3840 x1874669228511104/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787831052.027182:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228510848:192.168.203.114@tcp:250:kworker.0 00000800:00000100:2.0:1787831052.027263:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787831052.027300:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831052/real 1787831052] req@ffff9856085a12c0 x1874669228510848/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787831077 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787831052.027318:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a12c0 x1874669228510848/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787831077 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787831052.027328:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787831052.027333:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787831052.027337:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228510848:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787831052.027344:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228510976:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787831052.027366:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787831052.027377:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831052/real 1787831052] req@ffff9856085a2580 x1874669228510976/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831077 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787831052.027387:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2580 x1874669228510976/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831077 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787831052.027395:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787831052.027398:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787831052.027401:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228510976:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787831052.027407:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228511104:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787831052.027426:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787831052.027436:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831052/real 1787831052] req@ffff9856085a3840 x1874669228511104/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831077 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787831052.027446:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3840 x1874669228511104/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831077 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787831052.027453:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787831052.027456:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787831052.027459:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228511104:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787831052.088802:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787831052.980817:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787831053.109445:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787831054.004819:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787831054.133122:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787831055.028866:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787831055.157405:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787831056.052885:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787831056.181374:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787831057.076923:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787831057.141792:0:93375:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787831057.141805:0:93375:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787831057.141811:0:93375:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787831057.141824:0:93375:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 7202 00000100:00080000:2.0:1787831057.141838:0:93375:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787831057.141847:0:93375:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787831057.141899:0:93375:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db480 x1874669228511232/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787831057.141937:0:93375:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787831057.141941:0:93375:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787831057.141943:0:93375:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787831057.141947:0:93375:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7202 00000100:00080000:2.0:1787831057.141951:0:93375:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787831057.141955:0:93375:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787831057.141966:0:93375:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787831057.141978:0:93375:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8b40 x1874669228511360/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787831057.141987:0:93375:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787831057.141990:0:93375:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787831057.141992:0:93375:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787831057.141996:0:93375:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7202 00000100:00080000:2.0:1787831057.141999:0:93375:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787831057.142002:0:93375:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787831057.142007:0:93375:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787831057.142017:0:93375:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d9a40 x1874669228511488/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787831057.142167:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228511232:192.168.203.114@tcp:250:kworker.0 00000800:00000100:1.0:1787831057.142256:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787831057.142299:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831057/real 1787831057] req@ffff9856140db480 x1874669228511232/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787831082 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787831057.142328:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db480 x1874669228511232/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787831082 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787831057.142340:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787831057.142344:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787831057.142348:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228511232:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787831057.142355:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228511360:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787831057.142378:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787831057.142390:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831057/real 1787831057] req@ffff9856140d8b40 x1874669228511360/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831082 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787831057.142400:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8b40 x1874669228511360/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831082 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787831057.142407:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787831057.142410:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787831057.142413:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228511360:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787831057.142418:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228511488:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787831057.142438:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787831057.142449:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831057/real 1787831057] req@ffff9856140d9a40 x1874669228511488/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831082 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787831057.142459:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d9a40 x1874669228511488/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831082 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787831057.142466:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787831057.142469:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787831057.142472:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228511488:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787831057.205484:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787831058.100849:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787831058.231095:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787831059.124988:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787831059.253391:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787831060.149811:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787831060.277048:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787831061.173225:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787831061.302072:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831062.198681:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787831062.261971:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787831062.261986:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787831062.261990:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787831062.262000:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 7207 00000100:00080000:3.0:1787831062.262011:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787831062.262018:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787831062.262052:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82940 x1874669228511616/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787831062.262079:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787831062.262082:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787831062.262084:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787831062.262087:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7207 00000100:00080000:3.0:1787831062.262090:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787831062.262094:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787831062.262103:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787831062.262114:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82580 x1874669228511744/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787831062.262329:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787831062.262333:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787831062.262335:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787831062.262340:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7207 00000100:00080000:3.0:1787831062.262343:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787831062.262347:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787831062.262353:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787831062.262365:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80f00 x1874669228511872/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787831062.266111:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228511616:192.168.203.114@tcp:250:kworker.0 00000800:00000100:1.0:1787831062.266245:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787831062.266310:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831062/real 1787831062] req@ffff985610f82940 x1874669228511616/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787831087 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787831062.266332:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82940 x1874669228511616/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787831087 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787831062.266345:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787831062.266351:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787831062.266356:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228511616:192.168.203.114@tcp:250:kworker.0 00000100:00100000:1.0:1787831062.266365:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228511744:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787831062.266391:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787831062.266405:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831062/real 1787831062] req@ffff985610f82580 x1874669228511744/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831087 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787831062.266418:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82580 x1874669228511744/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831087 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787831062.266427:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787831062.266431:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787831062.266435:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228511744:192.168.203.114@tcp:38:kworker.0 00000100:00100000:1.0:1787831062.266442:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228511872:192.168.203.114@tcp:38:kworker.0 00000800:00000100:1.0:1787831062.266465:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:1.0:1787831062.266476:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831062/real 1787831062] req@ffff985610f80f00 x1874669228511872/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831087 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787831062.266487:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80f00 x1874669228511872/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831087 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787831062.266494:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787831062.266497:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:1.0:1787831062.266501:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228511872:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787831062.324971:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831063.220993:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787831063.349093:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831064.245738:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787831064.373200:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831065.269221:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787831065.397064:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831066.293039:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787831066.421649:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831067.317064:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787831067.381574:0:93375:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787831067.381584:0:93375:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787831067.381588:0:93375:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787831067.381598:0:93375:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 7213 00000100:00080000:2.0:1787831067.381608:0:93375:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787831067.381614:0:93375:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787831067.381650:0:93375:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140da940 x1874669228512000/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787831067.381741:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228512000:192.168.203.114@tcp:250:kworker.0 00000100:00080000:2.0:1787831067.381877:0:93375:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787831067.381882:0:93375:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787831067.381885:0:93375:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787831067.381889:0:93375:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7213 00000100:00080000:2.0:1787831067.381894:0:93375:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787831067.381898:0:93375:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787831067.381906:0:93375:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787831067.381922:0:93375:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db0c0 x1874669228512128/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787831067.386951:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228512128:192.168.203.114@tcp:38:kworker.0 00000100:00080000:2.0:1787831067.387890:0:93375:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787831067.387897:0:93375:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787831067.387901:0:93375:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787831067.387909:0:93375:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7213 00000100:00080000:2.0:1787831067.387916:0:93375:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787831067.387924:0:93375:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787831067.387935:0:93375:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787831067.387964:0:93375:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140dad00 x1874669228512256/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787831067.388182:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228512256:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787831067.445513:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831068.341624:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787831068.468949:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831069.365167:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787831069.492873:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831070.388814:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000400:00000100:0.0:1787831070.456786:0:2373:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.114:988 00000400:00000100:0.0:1787831070.456846:0:2373:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.114@tcp at host 192.168.203.114 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:0.0:1787831070.456864:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:0.0:1787831070.456914:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:0.0:1787831070.456922:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000100:00100000:2.0:1787831070.457029:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831067/real 1787831070] req@ffff9856140dad00 x1874669228512256/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831092 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787831070.457052:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140dad00 x1874669228512256/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831092 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787831070.457063:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787831070.457068:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787831070.457072:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228512256:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787831070.457079:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831067/real 1787831070] req@ffff9856140db0c0 x1874669228512128/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831092 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787831070.457090:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db0c0 x1874669228512128/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831092 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787831070.457097:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787831070.457100:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787831070.457103:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228512128:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787831070.457108:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831067/real 1787831070] req@ffff9856140da940 x1874669228512000/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787831092 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787831070.457120:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140da940 x1874669228512000/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787831092 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787831070.457137:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787831070.457141:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787831070.457144:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228512000:192.168.203.114@tcp:250:kworker.0 00010000:00010000:2.0:1787831070.518577:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831071.415056:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787831071.540953:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831072.436829:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787831072.503239:0:93375:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787831072.503248:0:93375:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787831072.503252:0:93375:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787831072.503263:0:93375:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 7218 00000100:00080000:1.0:1787831072.503273:0:93375:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787831072.503281:0:93375:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787831072.503322:0:93375:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a03c0 x1874669228512384/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787831072.503361:0:93375:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787831072.503365:0:93375:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787831072.503367:0:93375:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787831072.503372:0:93375:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7218 00000100:00080000:1.0:1787831072.503376:0:93375:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787831072.503381:0:93375:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787831072.503390:0:93375:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787831072.503402:0:93375:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3c00 x1874669228512512/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787831072.503411:0:93375:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787831072.503414:0:93375:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787831072.503415:0:93375:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787831072.503419:0:93375:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7218 00000100:00080000:1.0:1787831072.503422:0:93375:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787831072.503426:0:93375:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787831072.503430:0:93375:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787831072.503454:0:93375:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0780 x1874669228512640/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787831072.505797:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228512384:192.168.203.114@tcp:250:kworker.0 00000800:00000100:3.0:1787831072.505872:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787831072.505909:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831072/real 1787831072] req@ffff9856085a03c0 x1874669228512384/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787831097 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787831072.505928:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a03c0 x1874669228512384/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787831097 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787831072.505939:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787831072.505944:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787831072.505948:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228512384:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787831072.505956:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228512512:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787831072.505979:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787831072.505992:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831072/real 1787831072] req@ffff9856085a3c00 x1874669228512512/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831097 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787831072.506002:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3c00 x1874669228512512/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831097 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787831072.506009:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787831072.506012:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787831072.506015:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228512512:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787831072.506021:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228512640:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787831072.506039:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787831072.506049:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831072/real 1787831072] req@ffff9856085a0780 x1874669228512640/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831097 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787831072.506058:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0780 x1874669228512640/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831097 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787831072.506064:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787831072.506067:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787831072.506069:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228512640:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787831072.564921:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831073.460818:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787831073.590543:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831074.484966:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787831074.612914:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831075.509080:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787831075.638837:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831076.535926:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787831076.661255:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831077.556908:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787831077.620877:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787831077.620887:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787831077.620892:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787831077.620902:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 7223 00000100:00080000:3.0:1787831077.620913:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787831077.620919:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787831077.620953:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f821c0 x1874669228512768/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787831077.620986:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787831077.620990:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787831077.620992:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787831077.620995:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7223 00000100:00080000:3.0:1787831077.620998:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787831077.621001:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787831077.621010:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787831077.621021:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83480 x1874669228512896/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787831077.621031:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787831077.621033:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787831077.621035:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787831077.621038:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7223 00000100:00080000:3.0:1787831077.621042:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787831077.621045:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787831077.621049:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787831077.621061:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80f00 x1874669228513024/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787831077.621150:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f821c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228512768:192.168.203.114@tcp:250:kworker.0 00000800:00000100:2.0:1787831077.621278:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787831077.621309:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831077/real 1787831077] req@ffff985610f821c0 x1874669228512768/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787831102 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787831077.621322:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f821c0 x1874669228512768/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787831102 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787831077.621331:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787831077.621335:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787831077.621338:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f821c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228512768:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787831077.621345:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228512896:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787831077.621364:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787831077.621374:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831077/real 1787831077] req@ffff985610f83480 x1874669228512896/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831102 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787831077.621385:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83480 x1874669228512896/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831102 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787831077.621392:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787831077.621395:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787831077.621398:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228512896:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787831077.621403:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228513024:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787831077.621421:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787831077.621430:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831077/real 1787831077] req@ffff985610f80f00 x1874669228513024/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831102 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787831077.621440:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80f00 x1874669228513024/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831102 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787831077.621447:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787831077.621450:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787831077.621453:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228513024:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787831077.686119:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831078.580825:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787831078.709055:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831079.605252:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787831079.732963:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831080.629128:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787831080.760470:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831081.653184:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787831081.782248:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831082.676808:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787831082.740912:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787831082.740923:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787831082.740927:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787831082.740938:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 7228 00000100:00080000:0.0:1787831082.740948:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787831082.740956:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787831082.740994:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5e00 x1874669228513152/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787831082.741021:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787831082.741024:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787831082.741026:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787831082.741030:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7228 00000100:00080000:0.0:1787831082.741033:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787831082.741037:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787831082.741047:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787831082.741060:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df61c0 x1874669228513280/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787831082.741069:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787831082.741073:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787831082.741075:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787831082.741078:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7228 00000100:00080000:0.0:1787831082.741081:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787831082.741084:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787831082.741088:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787831082.741098:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5680 x1874669228513408/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787831082.741139:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228513152:192.168.203.114@tcp:250:kworker.0 00000800:00000100:2.0:1787831082.741199:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787831082.741232:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831082/real 1787831082] req@ffff985607df5e00 x1874669228513152/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787831107 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787831082.741247:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5e00 x1874669228513152/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787831107 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787831082.741256:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787831082.741260:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787831082.741265:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228513152:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787831082.741376:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228513280:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787831082.741404:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787831082.741418:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831082/real 1787831082] req@ffff985607df61c0 x1874669228513280/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831107 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787831082.741429:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df61c0 x1874669228513280/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831107 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787831082.741437:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787831082.741440:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787831082.741443:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228513280:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787831082.741448:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228513408:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787831082.741467:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787831082.741477:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831082/real 1787831082] req@ffff985607df5680 x1874669228513408/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831107 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787831082.741486:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5680 x1874669228513408/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831107 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787831082.741492:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787831082.741495:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787831082.741498:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228513408:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787831082.806258:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831083.701852:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787831083.829008:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831084.724914:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787831084.853983:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831085.748840:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787831085.876977:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787831086.772933:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787831086.905051:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787831087.796973:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787831087.861096:0:93375:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787831087.861106:0:93375:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787831087.861110:0:93375:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787831087.861121:0:93375:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 7233 00000100:00080000:1.0:1787831087.861131:0:93375:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787831087.861139:0:93375:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787831087.861176:0:93375:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a1680 x1874669228513536/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787831087.861265:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228513536:192.168.203.114@tcp:250:kworker.0 00000800:00000100:2.0:1787831087.861325:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00080000:1.0:1787831087.861357:0:93375:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787831087.861361:0:93375:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787831087.861363:0:93375:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787831087.861367:0:93375:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7233 00000100:00080000:1.0:1787831087.861372:0:93375:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787831087.861375:0:93375:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787831087.861384:0:93375:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787831087.861397:0:93375:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0000 x1874669228513664/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787831087.863087:0:93375:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787831087.863096:0:93375:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787831087.863099:0:93375:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787831087.863106:0:93375:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7233 00000100:00080000:1.0:1787831087.863114:0:93375:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787831087.863120:0:93375:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787831087.863141:0:93375:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787831087.863168:0:93375:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0b40 x1874669228513792/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787831087.863373:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831087/real 1787831087] req@ffff9856085a1680 x1874669228513536/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787831112 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787831087.863393:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a1680 x1874669228513536/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787831112 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787831087.863403:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787831087.863407:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787831087.863411:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228513536:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787831087.863437:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228513664:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787831087.863493:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787831087.863514:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831087/real 1787831087] req@ffff9856085a0000 x1874669228513664/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831112 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787831087.863527:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0000 x1874669228513664/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831112 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787831087.863535:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787831087.863539:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787831087.863542:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228513664:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787831087.863548:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228513792:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787831087.863581:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787831087.863596:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831087/real 1787831087] req@ffff9856085a0b40 x1874669228513792/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831112 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787831087.863607:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0b40 x1874669228513792/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831112 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787831087.863614:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787831087.863617:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787831087.863620:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228513792:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787831087.924901:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787831088.820918:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787831088.949444:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787831089.846724:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787831089.972900:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787831090.869111:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787831090.997978:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787831091.894522:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787831092.021057:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787831092.917255:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787831092.980843:0:93375:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787831092.980858:0:93375:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787831092.980863:0:93375:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787831092.980875:0:93375:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 7238 00000100:00080000:1.0:1787831092.980888:0:93375:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787831092.980897:0:93375:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787831092.980947:0:93375:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2d00 x1874669228513920/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787831092.980983:0:93375:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787831092.980987:0:93375:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787831092.980989:0:93375:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787831092.980993:0:93375:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7238 00000100:00080000:1.0:1787831092.980997:0:93375:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787831092.981002:0:93375:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787831092.981013:0:93375:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787831092.981026:0:93375:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0b40 x1874669228514048/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787831092.981040:0:93375:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787831092.981043:0:93375:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787831092.981045:0:93375:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787831092.981048:0:93375:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7238 00000100:00080000:1.0:1787831092.981052:0:93375:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787831092.981055:0:93375:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787831092.981060:0:93375:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787831092.981070:0:93375:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0000 x1874669228514176/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787831092.981806:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228513920:192.168.203.114@tcp:250:kworker.0 00000800:00000100:2.0:1787831092.981884:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787831092.981928:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831092/real 1787831092] req@ffff9856085a2d00 x1874669228513920/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787831117 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787831092.981948:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2d00 x1874669228513920/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787831117 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787831092.981960:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787831092.981964:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787831092.981968:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228513920:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787831092.981975:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228514048:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787831092.981998:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787831092.982010:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831092/real 1787831092] req@ffff9856085a0b40 x1874669228514048/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831117 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787831092.982021:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0b40 x1874669228514048/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831117 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787831092.982029:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787831092.982032:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787831092.982035:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228514048:192.168.203.114@tcp:38:kworker.0 00000100:00100000:2.0:1787831092.982040:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228514176:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787831092.982059:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787831092.982070:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831092/real 1787831092] req@ffff9856085a0000 x1874669228514176/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831117 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787831092.982080:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0000 x1874669228514176/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831117 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787831092.982087:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787831092.982090:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787831092.982092:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228514176:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787831093.045201:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787831093.940872:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787831094.068912:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787831094.964942:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787831095.092845:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787831095.988808:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787831096.117016:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831097.015371:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787831097.142663:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831098.036795:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787831098.101081:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787831098.101090:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787831098.101094:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787831098.101104:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 7243 00000100:00080000:1.0:1787831098.101115:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787831098.101122:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787831098.101161:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a1680 x1874669228514304/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787831098.101242:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228514304:192.168.203.114@tcp:250:kworker.0 00000800:00000100:2.0:1787831098.101310:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00080000:1.0:1787831098.101340:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787831098.101343:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787831098.101346:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787831098.101350:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7243 00000100:00080000:1.0:1787831098.101353:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787831098.101358:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787831098.101366:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787831098.101379:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a0780 x1874669228514432/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787831098.101942:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831098/real 1787831098] req@ffff9856085a1680 x1874669228514304/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787831123 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787831098.101960:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a1680 x1874669228514304/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787831123 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787831098.101970:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787831098.101974:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787831098.101977:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228514304:192.168.203.114@tcp:250:kworker.0 00000100:00100000:2.0:1787831098.102002:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228514432:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787831098.102029:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787831098.102040:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831098/real 1787831098] req@ffff9856085a0780 x1874669228514432/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831123 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787831098.102050:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a0780 x1874669228514432/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831123 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787831098.102057:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787831098.102059:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787831098.102062:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228514432:192.168.203.114@tcp:38:kworker.0 00000100:00080000:1.0:1787831098.102421:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787831098.102426:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787831098.102428:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787831098.102433:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7243 00000100:00080000:1.0:1787831098.102438:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787831098.102443:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787831098.102451:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787831098.102563:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3c00 x1874669228514560/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787831098.102779:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228514560:192.168.203.114@tcp:38:kworker.0 00000800:00000100:2.0:1787831098.102929:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:2.0:1787831098.102949:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831098/real 1787831098] req@ffff9856085a3c00 x1874669228514560/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831123 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787831098.102963:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3c00 x1874669228514560/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831123 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787831098.102977:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787831098.102980:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:2.0:1787831098.102983:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228514560:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787831098.165728:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831099.061239:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787831099.189002:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831100.084920:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787831100.212853:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787831101.109015:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787831101.245356:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787831102.132862:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787831102.262761:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787831103.156846:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787831103.222111:0:93375:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787831103.222125:0:93375:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787831103.222131:0:93375:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787831103.222144:0:93375:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 7248 00000100:00080000:2.0:1787831103.222158:0:93375:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787831103.222168:0:93375:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787831103.222218:0:93375:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140da940 x1874669228514688/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787831103.222254:0:93375:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787831103.222258:0:93375:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787831103.222260:0:93375:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787831103.222264:0:93375:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7248 00000100:00080000:2.0:1787831103.222269:0:93375:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787831103.222273:0:93375:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787831103.222287:0:93375:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787831103.222302:0:93375:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140db0c0 x1874669228514816/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787831103.222314:0:93375:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1787831103.222316:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228514688:192.168.203.114@tcp:250:kworker.0 00000100:00080000:2.0:1787831103.222320:0:93375:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787831103.222324:0:93375:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787831103.222330:0:93375:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7248 00000100:00080000:2.0:1787831103.222334:0:93375:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787831103.222338:0:93375:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787831103.222343:0:93375:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787831103.222356:0:93375:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140dad00 x1874669228514944/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787831103.222405:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787831103.222448:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831103/real 1787831103] req@ffff9856140da940 x1874669228514688/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787831128 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787831103.222481:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140da940 x1874669228514688/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787831128 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787831103.222510:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787831103.222515:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787831103.222520:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228514688:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787831103.222552:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228514816:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787831103.222576:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787831103.222589:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831103/real 1787831103] req@ffff9856140db0c0 x1874669228514816/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831128 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787831103.222602:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140db0c0 x1874669228514816/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831128 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787831103.222610:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787831103.222614:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787831103.222617:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228514816:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787831103.222623:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228514944:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787831103.222642:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787831103.222654:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831103/real 1787831103] req@ffff9856140dad00 x1874669228514944/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831128 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787831103.222665:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140dad00 x1874669228514944/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831128 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787831103.222673:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787831103.222676:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787831103.222679:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228514944:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787831103.287333:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787831104.181616:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787831104.309104:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787831105.204944:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787831105.334092:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787831106.230291:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787831106.357002:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787831107.252879:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787831107.381270:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787831108.276875:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787831108.341070:0:93375:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787831108.341084:0:93375:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787831108.341089:0:93375:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787831108.341102:0:93375:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 7253 00000100:00080000:1.0:1787831108.341116:0:93375:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787831108.341126:0:93375:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787831108.341387:0:93375:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a03c0 x1874669228515072/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787831108.341486:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228515072:192.168.203.114@tcp:250:kworker.0 00000800:00000100:3.0:1787831108.341577:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00080000:1.0:1787831108.341614:0:93375:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787831108.341618:0:93375:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787831108.341621:0:93375:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787831108.341626:0:93375:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7253 00000100:00080000:1.0:1787831108.341630:0:93375:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787831108.341635:0:93375:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787831108.341646:0:93375:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787831108.341660:0:93375:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3840 x1874669228515200/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787831108.341764:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831108/real 1787831108] req@ffff9856085a03c0 x1874669228515072/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787831133 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787831108.341786:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a03c0 x1874669228515072/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787831133 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787831108.341797:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787831108.341802:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787831108.341806:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228515072:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787831108.341835:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228515200:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787831108.341862:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787831108.341875:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831108/real 1787831108] req@ffff9856085a3840 x1874669228515200/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831133 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787831108.341888:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3840 x1874669228515200/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831133 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787831108.341896:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787831108.341900:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787831108.341903:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228515200:192.168.203.114@tcp:38:kworker.0 00000100:00080000:1.0:1787831108.342728:0:93375:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787831108.342843:0:93375:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787831108.342847:0:93375:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787831108.342856:0:93375:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7253 00000100:00080000:1.0:1787831108.342867:0:93375:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787831108.342875:0:93375:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787831108.342887:0:93375:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787831108.342923:0:93375:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2580 x1874669228515328/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787831108.343196:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228515328:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787831108.343262:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787831108.343292:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831108/real 1787831108] req@ffff9856085a2580 x1874669228515328/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831133 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787831108.343309:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2580 x1874669228515328/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831133 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787831108.343320:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787831108.343324:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787831108.343328:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228515328:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787831108.405847:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831109.306372:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787831109.429129:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831110.329384:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787831110.453054:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831111.349095:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787831111.477483:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831112.372856:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787831112.502301:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831113.396901:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787831113.460948:0:93375:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787831113.460958:0:93375:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787831113.460962:0:93375:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787831113.460972:0:93375:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 7259 00000100:00080000:1.0:1787831113.460992:0:93375:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787831113.461000:0:93375:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787831113.461032:0:93375:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a12c0 x1874669228515456/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787831113.461064:0:93375:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787831113.461068:0:93375:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787831113.461069:0:93375:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787831113.461073:0:93375:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7259 00000100:00080000:1.0:1787831113.461076:0:93375:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787831113.461079:0:93375:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787831113.461087:0:93375:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787831113.461097:0:93375:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a2940 x1874669228515584/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787831113.461105:0:93375:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787831113.461109:0:93375:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787831113.461110:0:93375:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787831113.461114:0:93375:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7259 00000100:00080000:1.0:1787831113.461118:0:93375:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787831113.461121:0:93375:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787831113.461126:0:93375:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787831113.461137:0:93375:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a1a40 x1874669228515712/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787831113.461274:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228515456:192.168.203.114@tcp:250:kworker.0 00000800:00000100:3.0:1787831113.461329:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787831113.461362:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831113/real 1787831113] req@ffff9856085a12c0 x1874669228515456/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787831138 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787831113.461377:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a12c0 x1874669228515456/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787831138 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787831113.461395:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787831113.461399:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787831113.461402:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228515456:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787831113.461408:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228515584:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787831113.461429:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787831113.461451:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831113/real 1787831113] req@ffff9856085a2940 x1874669228515584/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831138 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787831113.461462:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a2940 x1874669228515584/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831138 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787831113.461474:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787831113.461477:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787831113.461480:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228515584:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787831113.461485:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228515712:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787831113.461503:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787831113.461513:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831113/real 1787831113] req@ffff9856085a1a40 x1874669228515712/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831138 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787831113.461523:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a1a40 x1874669228515712/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831138 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787831113.461530:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787831113.461533:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787831113.461535:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228515712:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787831113.526716:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831114.420813:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787831114.549420:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831115.444819:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787831115.572857:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831116.469024:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787831116.596918:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831117.492788:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787831117.624427:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831118.516809:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787831118.580867:0:93375:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787831118.580877:0:93375:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787831118.580881:0:93375:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787831118.580892:0:93375:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 7264 00000100:00080000:3.0:1787831118.580902:0:93375:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787831118.580910:0:93375:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787831118.580948:0:93375:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82580 x1874669228515840/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787831118.581023:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228515840:192.168.203.114@tcp:250:kworker.0 00000800:00000100:0.0:1787831118.581083:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00080000:3.0:1787831118.581116:0:93375:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787831118.581120:0:93375:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787831118.581122:0:93375:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787831118.581126:0:93375:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7264 00000100:00080000:3.0:1787831118.581130:0:93375:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787831118.581135:0:93375:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787831118.581144:0:93375:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787831118.581160:0:93375:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82940 x1874669228515968/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787831118.581255:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831118/real 1787831118] req@ffff985610f82580 x1874669228515840/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787831143 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787831118.581273:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82580 x1874669228515840/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787831143 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787831118.581283:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787831118.581287:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787831118.581307:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228515840:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787831118.581330:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228515968:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787831118.581370:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787831118.581384:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831118/real 1787831118] req@ffff985610f82940 x1874669228515968/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831143 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787831118.581397:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82940 x1874669228515968/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831143 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787831118.581411:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787831118.581414:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787831118.581417:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228515968:192.168.203.114@tcp:38:kworker.0 00000100:00080000:3.0:1787831118.581798:0:93375:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787831118.581803:0:93375:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787831118.581805:0:93375:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787831118.581810:0:93375:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7264 00000100:00080000:3.0:1787831118.581814:0:93375:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787831118.581819:0:93375:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787831118.581825:0:93375:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787831118.581841:0:93375:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82d00 x1874669228516096/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787831118.581885:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228516096:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787831118.581914:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787831118.582037:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831118/real 1787831118] req@ffff985610f82d00 x1874669228516096/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831143 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787831118.582049:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82d00 x1874669228516096/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831143 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787831118.582058:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787831118.582061:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787831118.582065:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228516096:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787831118.645200:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831119.540909:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787831119.668999:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831120.565181:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787831120.692826:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831121.589730:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787831121.717318:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831122.613133:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787831122.741231:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831123.636838:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787831123.700851:0:93375:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787831123.700865:0:93375:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787831123.700870:0:93375:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787831123.700884:0:93375:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 7269 00000100:00080000:3.0:1787831123.700898:0:93375:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787831123.700906:0:93375:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787831123.700955:0:93375:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f803c0 x1874669228516224/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787831123.700993:0:93375:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787831123.700996:0:93375:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787831123.700998:0:93375:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787831123.701002:0:93375:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7269 00000100:00080000:3.0:1787831123.701005:0:93375:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787831123.701009:0:93375:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787831123.701020:0:93375:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787831123.701033:0:93375:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83840 x1874669228516352/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787831123.701042:0:93375:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787831123.701046:0:93375:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787831123.701048:0:93375:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787831123.701052:0:93375:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7269 00000100:00080000:3.0:1787831123.701055:0:93375:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787831123.701059:0:93375:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787831123.701064:0:93375:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787831123.701073:0:93375:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81e00 x1874669228516480/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787831123.701183:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228516224:192.168.203.114@tcp:250:kworker.0 00000800:00000100:0.0:1787831123.701263:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787831123.701303:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831123/real 1787831123] req@ffff985610f803c0 x1874669228516224/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787831148 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787831123.701331:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f803c0 x1874669228516224/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787831148 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787831123.701341:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787831123.701345:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787831123.701349:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228516224:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787831123.701356:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228516352:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787831123.701380:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787831123.701393:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831123/real 1787831123] req@ffff985610f83840 x1874669228516352/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831148 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787831123.701404:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83840 x1874669228516352/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831148 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787831123.701411:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787831123.701414:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787831123.701417:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228516352:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787831123.701423:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228516480:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787831123.701442:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787831123.701453:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831123/real 1787831123] req@ffff985610f81e00 x1874669228516480/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831148 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787831123.701463:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81e00 x1874669228516480/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831148 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787831123.701471:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787831123.701474:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787831123.701477:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228516480:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787831123.766051:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831124.661206:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787831124.791208:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831125.684817:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787831125.812983:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831126.709000:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787831126.837578:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831127.733174:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787831127.860924:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831128.756813:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787831128.821287:0:93375:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787831128.821303:0:93375:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787831128.821309:0:93375:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787831128.821322:0:93375:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 7274 00000100:00080000:3.0:1787831128.821338:0:93375:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787831128.821347:0:93375:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787831128.821395:0:93375:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81680 x1874669228516608/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787831128.821491:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228516608:192.168.203.114@tcp:250:kworker.0 00000800:00000100:0.0:1787831128.821596:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00080000:3.0:1787831128.821632:0:93375:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787831128.821643:0:93375:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787831128.821645:0:93375:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787831128.821649:0:93375:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7274 00000100:00080000:3.0:1787831128.821653:0:93375:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787831128.821658:0:93375:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787831128.821670:0:93375:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787831128.821723:0:93375:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f812c0 x1874669228516736/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787831128.823407:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831128/real 1787831128] req@ffff985610f81680 x1874669228516608/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787831153 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787831128.823436:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81680 x1874669228516608/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787831153 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787831128.823448:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787831128.823453:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787831128.823468:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228516608:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787831128.823504:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228516736:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787831128.823567:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787831128.823651:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831128/real 1787831128] req@ffff985610f812c0 x1874669228516736/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831153 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787831128.823666:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f812c0 x1874669228516736/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831153 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787831128.823674:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787831128.823677:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787831128.823681:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228516736:192.168.203.114@tcp:38:kworker.0 00000100:00080000:3.0:1787831128.826757:0:93375:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787831128.826770:0:93375:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787831128.826774:0:93375:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787831128.826786:0:93375:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7274 00000100:00080000:3.0:1787831128.826798:0:93375:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787831128.826808:0:93375:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787831128.827090:0:93375:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787831128.827136:0:93375:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80b40 x1874669228516864/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787831128.827339:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228516864:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787831128.827422:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787831128.827455:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831128/real 1787831128] req@ffff985610f80b40 x1874669228516864/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831153 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787831128.827495:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80b40 x1874669228516864/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831153 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787831128.827507:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787831128.827511:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787831128.827515:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228516864:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787831128.885422:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831129.780863:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787831129.908989:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831130.805084:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787831130.932888:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831131.828845:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787831131.962291:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831132.853298:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787831132.981956:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831133.876849:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787831133.940893:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787831133.940917:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787831133.940922:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787831133.940942:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 7279 00000100:00080000:3.0:1787831133.940966:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787831133.940976:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787831133.941034:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80000 x1874669228516992/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787831133.941077:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787831133.941081:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787831133.941083:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787831133.941087:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7279 00000100:00080000:3.0:1787831133.941090:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787831133.941094:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787831133.941106:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787831133.941117:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81a40 x1874669228517120/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787831133.941126:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787831133.941130:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787831133.941132:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787831133.941136:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7279 00000100:00080000:3.0:1787831133.941139:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787831133.941143:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787831133.941148:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787831133.941158:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f830c0 x1874669228517248/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787831133.941492:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228516992:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787831133.942088:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228517120:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787831133.942134:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228517248:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787831134.006422:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831134.900861:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787831135.028973:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831135.925430:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787831136.053148:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831136.948872:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000400:00000100:1.0:1787831137.022287:0:2370:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.114:988 00000400:00000100:1.0:1787831137.022348:0:2370:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.114@tcp at host 192.168.203.114 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1787831137.022370:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000100:00100000:0.0:1787831137.023410:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831133/real 1787831137] req@ffff985610f80000 x1874669228516992/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787831158 ref 2 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787831137.023447:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80000 x1874669228516992/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787831158 ref 2 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787831137.025645:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787831137.025667:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787831137.025676:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228516992:192.168.203.114@tcp:250:kworker.0 00000800:00000100:1.0:1787831137.026647:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000800:00000100:1.0:1787831137.026675:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.114@tcp 00000100:00100000:0.0:1787831137.026892:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831133/real 1787831137] req@ffff985610f81a40 x1874669228517120/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831158 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787831137.026920:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81a40 x1874669228517120/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831158 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787831137.026935:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787831137.026940:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787831137.026944:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228517120:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787831137.026952:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831133/real 1787831137] req@ffff985610f830c0 x1874669228517248/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831158 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787831137.026965:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f830c0 x1874669228517248/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831158 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787831137.026975:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787831137.026979:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787831137.026983:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228517248:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787831137.077397:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831137.972810:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787831138.101492:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831138.997817:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787831139.062844:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787831139.062859:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787831139.062865:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787831139.062890:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 7284 00000100:00080000:3.0:1787831139.062904:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787831139.062913:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787831139.062968:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80780 x1874669228517376/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787831139.063012:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787831139.063016:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787831139.063120:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787831139.063125:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7284 00000100:00080000:3.0:1787831139.063130:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787831139.063135:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787831139.063147:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787831139.063161:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83c00 x1874669228517504/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787831139.063171:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787831139.063174:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787831139.063176:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787831139.063180:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7284 00000100:00080000:3.0:1787831139.063183:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787831139.063187:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787831139.063193:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787831139.063233:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f830c0 x1874669228517632/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787831139.063356:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228517376:192.168.203.114@tcp:250:kworker.0 00000800:00000100:0.0:1787831139.063552:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787831139.063597:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831139/real 1787831139] req@ffff985610f80780 x1874669228517376/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787831164 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787831139.063617:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80780 x1874669228517376/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787831164 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787831139.063628:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787831139.063632:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787831139.063636:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228517376:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787831139.063644:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228517504:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787831139.063669:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787831139.063681:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831139/real 1787831139] req@ffff985610f83c00 x1874669228517504/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831164 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787831139.063727:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83c00 x1874669228517504/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831164 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787831139.063736:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787831139.063739:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787831139.063743:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228517504:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787831139.063749:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228517632:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787831139.063771:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787831139.063782:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831139/real 1787831139] req@ffff985610f830c0 x1874669228517632/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831164 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787831139.063792:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f830c0 x1874669228517632/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831164 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787831139.063799:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787831139.063803:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787831139.063806:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f830c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228517632:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787831139.126583:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831140.020871:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787831140.148992:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831141.045155:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787831141.173574:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831142.068873:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787831142.197214:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831143.092813:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787831143.221149:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831144.116822:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:0.0:1787831144.181644:0:93375:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787831144.181652:0:93375:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787831144.181656:0:93375:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787831144.181665:0:93375:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 7289 00000100:00080000:0.0:1787831144.181675:0:93375:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787831144.181682:0:93375:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:0.0:1787831144.181943:0:93375:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df70c0 x1874669228517760/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787831144.181975:0:93375:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787831144.181979:0:93375:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787831144.181981:0:93375:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787831144.181986:0:93375:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7289 00000100:00080000:0.0:1787831144.181991:0:93375:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787831144.181997:0:93375:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787831144.182007:0:93375:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787831144.182022:0:93375:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df6580 x1874669228517888/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787831144.182033:0:93375:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787831144.182037:0:93375:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787831144.182039:0:93375:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787831144.182043:0:93375:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7289 00000100:00080000:0.0:1787831144.182048:0:93375:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787831144.182053:0:93375:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:0.0:1787831144.182058:0:93375:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787831144.182070:0:93375:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985607df5a40 x1874669228518016/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787831144.182287:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228517760:192.168.203.114@tcp:250:kworker.0 00000800:00000100:3.0:1787831144.182604:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787831144.182672:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831144/real 1787831144] req@ffff985607df70c0 x1874669228517760/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787831169 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787831144.183229:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df70c0 x1874669228517760/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787831169 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787831144.183248:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787831144.183258:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787831144.183267:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228517760:192.168.203.114@tcp:250:kworker.0 00000100:00100000:3.0:1787831144.183288:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228517888:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787831144.183348:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787831144.183378:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831144/real 1787831144] req@ffff985607df6580 x1874669228517888/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831169 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787831144.183399:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df6580 x1874669228517888/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831169 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787831144.183415:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787831144.183421:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787831144.183431:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228517888:192.168.203.114@tcp:38:kworker.0 00000100:00100000:3.0:1787831144.183504:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985607df5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228518016:192.168.203.114@tcp:38:kworker.0 00000800:00000100:3.0:1787831144.183540:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:3.0:1787831144.183571:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831144/real 1787831144] req@ffff985607df5a40 x1874669228518016/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831169 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787831144.183592:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985607df5a40 x1874669228518016/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831169 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787831144.183604:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787831144.183611:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:3.0:1787831144.183618:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985607df5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228518016:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787831144.247474:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831145.140805:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787831145.268995:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831146.164942:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787831146.293603:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831147.189633:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787831147.316933:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831148.212992:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787831148.343275:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831149.236871:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787831149.301037:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787831149.301045:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787831149.301049:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787831149.301057:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 7294 00000100:00080000:3.0:1787831149.301067:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787831149.301074:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787831149.301106:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81a40 x1874669228518144/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787831149.301176:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228518144:192.168.203.114@tcp:250:kworker.0 00000800:00000100:0.0:1787831149.301232:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00080000:3.0:1787831149.301260:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787831149.301263:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787831149.301265:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787831149.301268:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7294 00000100:00080000:3.0:1787831149.301272:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787831149.301276:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787831149.301285:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787831149.301297:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80000 x1874669228518272/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787831149.301344:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831149/real 1787831149] req@ffff985610f81a40 x1874669228518144/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787831174 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787831149.301360:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81a40 x1874669228518144/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787831174 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787831149.301370:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787831149.301374:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787831149.301377:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228518144:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787831149.301397:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228518272:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787831149.301417:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787831149.301439:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831149/real 1787831149] req@ffff985610f80000 x1874669228518272/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831174 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787831149.301450:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80000 x1874669228518272/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831174 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787831149.301457:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787831149.301466:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787831149.301469:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228518272:192.168.203.114@tcp:38:kworker.0 00000100:00080000:3.0:1787831149.301956:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787831149.301961:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787831149.301963:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787831149.301968:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7294 00000100:00080000:3.0:1787831149.301973:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787831149.301978:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787831149.301985:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787831149.302001:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f80b40 x1874669228518400/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787831149.302050:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228518400:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787831149.302080:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787831149.302152:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831149/real 1787831149] req@ffff985610f80b40 x1874669228518400/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831174 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787831149.302171:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f80b40 x1874669228518400/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831174 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787831149.302180:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787831149.302183:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787831149.302186:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f80b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228518400:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787831149.366981:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831150.263006:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787831150.389101:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831151.285400:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787831151.412950:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831152.309460:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787831152.436926:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831153.332805:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787831153.461037:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831154.356822:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:1.0:1787831154.427961:0:93375:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787831154.427971:0:93375:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787831154.427975:0:93375:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787831154.427985:0:93375:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 7300 00000100:00080000:1.0:1787831154.427996:0:93375:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787831154.428005:0:93375:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:1.0:1787831154.428457:0:93375:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a21c0 x1874669228518528/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787831154.428554:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228518528:192.168.203.114@tcp:250:kworker.0 00000800:00000100:0.0:1787831154.428614:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00080000:1.0:1787831154.428647:0:93375:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787831154.428650:0:93375:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787831154.428652:0:93375:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787831154.428657:0:93375:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7300 00000100:00080000:1.0:1787831154.428661:0:93375:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787831154.428665:0:93375:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787831154.428674:0:93375:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787831154.428688:0:93375:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a3480 x1874669228518656/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787831154.428935:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831154/real 1787831154] req@ffff9856085a21c0 x1874669228518528/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787831179 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787831154.428951:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a21c0 x1874669228518528/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787831179 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787831154.428961:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787831154.428966:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787831154.428971:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228518528:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787831154.428993:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228518656:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787831154.429018:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787831154.429041:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831154/real 1787831154] req@ffff9856085a3480 x1874669228518656/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831179 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787831154.429052:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a3480 x1874669228518656/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831179 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787831154.429065:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787831154.429068:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787831154.429071:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228518656:192.168.203.114@tcp:38:kworker.0 00000100:00080000:1.0:1787831154.429537:0:93375:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787831154.429542:0:93375:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787831154.429544:0:93375:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787831154.429549:0:93375:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7300 00000100:00080000:1.0:1787831154.429554:0:93375:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787831154.429560:0:93375:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:1.0:1787831154.429567:0:93375:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787831154.429583:0:93375:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856085a30c0 x1874669228518784/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787831154.429663:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856085a30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228518784:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787831154.429892:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787831154.429910:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831154/real 1787831154] req@ffff9856085a30c0 x1874669228518784/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831179 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787831154.429923:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856085a30c0 x1874669228518784/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831179 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787831154.429930:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787831154.429933:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787831154.429936:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856085a30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228518784:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787831154.485543:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831155.382446:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787831155.509227:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831156.405110:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787831156.533011:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831157.428803:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787831157.558240:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831158.452824:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787831158.580910:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831159.476832:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787831159.540923:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787831159.540938:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787831159.540943:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787831159.540956:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 7305 00000100:00080000:3.0:1787831159.540971:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787831159.540980:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787831159.541029:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f812c0 x1874669228518912/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787831159.541072:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787831159.541077:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787831159.541079:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787831159.541083:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7305 00000100:00080000:3.0:1787831159.541087:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787831159.541092:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787831159.541114:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787831159.541129:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81680 x1874669228519040/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787831159.541138:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787831159.541143:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787831159.541145:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787831159.541149:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7305 00000100:00080000:3.0:1787831159.541152:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787831159.541157:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787831159.541162:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787831159.541172:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f81e00 x1874669228519168/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787831159.541397:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228518912:192.168.203.114@tcp:250:kworker.0 00000800:00000100:0.0:1787831159.541484:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787831159.541529:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831159/real 1787831159] req@ffff985610f812c0 x1874669228518912/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787831184 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787831159.541548:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f812c0 x1874669228518912/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787831184 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787831159.541560:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787831159.541565:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787831159.541570:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228518912:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787831159.541578:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228519040:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787831159.541604:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787831159.541617:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831159/real 1787831159] req@ffff985610f81680 x1874669228519040/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831184 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787831159.541628:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81680 x1874669228519040/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831184 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787831159.541636:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787831159.541639:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787831159.541642:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228519040:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787831159.541648:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228519168:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787831159.541669:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787831159.541679:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831159/real 1787831159] req@ffff985610f81e00 x1874669228519168/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831184 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787831159.541720:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f81e00 x1874669228519168/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831184 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787831159.541730:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787831159.541733:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787831159.541736:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f81e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228519168:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787831159.606616:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831160.500823:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787831160.629575:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831161.525931:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787831161.653243:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831162.548817:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787831162.676980:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831163.572839:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787831163.701381:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831164.596884:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:3.0:1787831164.660859:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787831164.660872:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787831164.660878:0:54101:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787831164.660891:0:54101:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 7310 00000100:00080000:3.0:1787831164.660905:0:54101:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787831164.660915:0:54101:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:3.0:1787831164.660962:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f83840 x1874669228519296/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787831164.661000:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787831164.661004:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787831164.661007:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787831164.661011:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7310 00000100:00080000:3.0:1787831164.661015:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787831164.661036:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787831164.661049:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787831164.661062:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f803c0 x1874669228519424/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787831164.661071:0:54101:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787831164.661075:0:54101:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787831164.661076:0:54101:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787831164.661080:0:54101:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7310 00000100:00080000:3.0:1787831164.661084:0:54101:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787831164.661088:0:54101:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:3.0:1787831164.661092:0:54101:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787831164.661103:0:54101:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff985610f82d00 x1874669228519552/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787831164.662076:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228519296:192.168.203.114@tcp:250:kworker.0 00000800:00000100:0.0:1787831164.662164:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787831164.662208:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831164/real 1787831164] req@ffff985610f83840 x1874669228519296/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787831189 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787831164.662227:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f83840 x1874669228519296/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787831189 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787831164.662244:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787831164.662249:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787831164.662253:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f83840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228519296:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787831164.662262:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228519424:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787831164.662289:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787831164.662301:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831164/real 1787831164] req@ffff985610f803c0 x1874669228519424/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831189 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787831164.662313:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f803c0 x1874669228519424/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831189 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787831164.662321:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787831164.662324:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787831164.662327:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f803c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228519424:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787831164.662334:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228519552:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787831164.662350:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787831164.662359:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831164/real 1787831164] req@ffff985610f82d00 x1874669228519552/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831189 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787831164.662377:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff985610f82d00 x1874669228519552/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831189 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787831164.662385:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787831164.662388:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787831164.662392:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff985610f82d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228519552:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787831164.726086:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831165.622954:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787831165.749536:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831166.644807:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787831166.772886:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831167.668855:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787831167.797837:0:79887:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831168.692955:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:2.0:1787831168.822421:0:15163:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831169.717127:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00000100:00080000:2.0:1787831169.782683:0:93375:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3c946890-15ca-497a-b838-fe1cba61f4dd->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787831169.782744:0:93375:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787831169.782751:0:93375:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787831169.782763:0:93375:0:(import.c:510:import_select_connection()) MGC192.168.203.114@tcp: connect to NID 192.168.203.114@tcp last attempt 7315 00000100:00080000:2.0:1787831169.782781:0:93375:0:(import.c:589:import_select_connection()) MGC192.168.203.114@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787831169.782790:0:93375:0:(import.c:624:import_select_connection()) MGC192.168.203.114@tcp: import ffff98561118e000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000100:00100000:2.0:1787831169.782842:0:93375:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140d8f00 x1874669228519680/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787831169.782875:0:93375:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787831169.782879:0:93375:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787831169.782881:0:93375:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787831169.782885:0:93375:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7315 00000100:00080000:2.0:1787831169.782900:0:93375:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787831169.782904:0:93375:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff985602bc9800: import ffff985611182000 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787831169.782916:0:93375:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787831169.782929:0:93375:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140da1c0 x1874669228519808/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787831169.782938:0:93375:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7b4e8535-9db7-485d-abf6-bda966b63320->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787831169.782941:0:93375:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787831169.782943:0:93375:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787831169.782946:0:93375:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: connect to NID 192.168.203.114@tcp last attempt 7315 00000100:00080000:2.0:1787831169.782950:0:93375:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787831169.782953:0:93375:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff985602bc9800: import ffff985611185800 using connection 192.168.203.114@tcp/192.168.203.114@tcp 00000008:00100000:2.0:1787831169.782957:0:93375:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787831169.782966:0:93375:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9856140dbc00 x1874669228519936/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787831169.783040:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228519680:192.168.203.114@tcp:250:kworker.0 00000800:00000100:0.0:1787831169.783181:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787831169.783227:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831169/real 1787831169] req@ffff9856140d8f00 x1874669228519680/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787831194 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787831169.783245:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140d8f00 x1874669228519680/t0(0) o250->MGC192.168.203.114@tcp@192.168.203.114@tcp:26/25 lens 520/544 e 0 to 1 dl 1787831194 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787831169.783257:0:2382:0:(import.c:65:import_set_state_nolock()) ffff98561118e000 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787831169.783261:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787831169.783265:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3c946890-15ca-497a-b838-fe1cba61f4dd:2382:1874669228519680:192.168.203.114@tcp:250:kworker.0 00000100:00100000:0.0:1787831169.783272:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228519808:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787831169.783296:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787831169.783309:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831169/real 1787831169] req@ffff9856140da1c0 x1874669228519808/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831194 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787831169.783319:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140da1c0 x1874669228519808/t0(0) o38->lustre-MDT0000-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831194 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787831169.783601:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611182000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787831169.783606:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787831169.783609:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228519808:192.168.203.114@tcp:38:kworker.0 00000100:00100000:0.0:1787831169.783644:0:2382:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228519936:192.168.203.114@tcp:38:kworker.0 00000800:00000100:0.0:1787831169.783672:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.114@tcp 00000100:00100000:0.0:1787831169.783684:0:2382:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787831169/real 1787831169] req@ffff9856140dbc00 x1874669228519936/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831194 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787831169.783733:0:2382:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9856140dbc00 x1874669228519936/t0(0) o38->lustre-MDT0001-mdc-ffff985602bc9800@192.168.203.114@tcp:12/10 lens 520/544 e 0 to 1 dl 1787831194 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787831169.783741:0:2382:0:(import.c:65:import_set_state_nolock()) ffff985611185800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787831169.783744:0:2382:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.114@tcp failed (-110) 00000100:00100000:0.0:1787831169.783747:0:2382:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9856140dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7b4e8535-9db7-485d-abf6-bda966b63320:2382:1874669228519936:192.168.203.114@tcp:38:kworker.0 00010000:00010000:2.0:1787831169.846417:0:33341:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831170.740848:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:1.0:1787831170.870894:0:15164:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787831171.765198:0:92880:0:(client.c:2715:ptlrpc_set_wait()) set ffff985607c5bcc0 going to sleep for 0 seconds 00010000:00010000:0.0:1787831171.893213:0:33342:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) Debug log: 82677 lines, 82677 kept, 0 dropped, 0 bad.